In several hours there will be some storage maintenance on going.
We need to reboot all machines. Reason is on the storage side several drives have dropped in speed, which means the whole array is crawling right now. We have no idea why the SATA link is being renegotiated for lower speed, ALL sata cables are high quality and new. Further, Samsung 840 PRO SSD drive is again giving grief with hugely degraded performance, doing only 28M/s read peaks!
At the go, we will likely also update kernel, and likely finally change over to IET iscsi target.
All of this will take several hours and occur over multiple reboots etc. production will be restored occasionally momentarily as we prep the next change.
These storage maintenance breaks *will end* as soon as the cluster is up & running. We estimate that to happen early august.
At that point we can work on individual nodes without interference on the user facing side, as long as it's not something to do with the SAN gateway machine.
Tuesday, July 16, 2013
