When you are doing a build out like ours - so many new things, and taking certain things to extremes there is bound to be unexpected things happening.
After several days of issues with a storage brick we noticed the SATA Raid controller was not compatible with this specific SSD model and make. Then basic SATA power connectors failing one after another, a need for not just Y-cables from Molex to couple sata power connectors, we actually soldered our own tri-cables as no shop was open (past 9PM) and wanted to get it up & running ASAP.
Then we get to the fun part: Switches. Those mystical black boxes. We are starting to understand why people pay a premium for support contracts and keep a tight check that their model is still supported. A bunch of our switches are Brocade / Woven Systems 48xGbit + 4xInfiniband 10G, and whoa, has it been a job to get them funciton at any level!
After tedious search with very little results finding a supposedly correct manual it turns out half the example commands and half the documentation is completely wrong - but not totally off the base. Hours to get the switch even to respond to it's IP via management etc.
Just to notice that our HP Infiniband adapter is not compatible with this type of switch, the HPs are compatible with the Cisco Core switches (those beasts of a switches with 960Gbps fabric), but not with the Brocades. Fortunately, we had some Myricom cards which are not really even Infiniband - sure they utilize the same switches, the same fabric, but only support IPoIB, but these atleast work with the Brocades :)
Lots of things like this, even small pieces like rack nuts can give you a headache: We had a bunch of nuts which used smaller bolts, and huge pile of nuts which were way too loose to be used oO;
Personally, i've never used fiber links anywhere, yes, i've seen that stuff, but actually using -> a little bit of research and wondering What the ??? am i supposed to do with this connector :) Thing is, i like to learn this stuff, even tho we got people in our team who knows this stuff, but as "The Man"/"The Boss", i ought to know at least the basics :)
An all to the way of Brocade being very sensitive to cable quality, and not bringing up interfaceto 1Gbps if the cable in question is not top notch. Well I guess that's a good thing! :)
Plenty of progress, long nights at the colo room but things are moving, expecting that our transit link is up & running in a few days.
Br,
Aleksi
יום רביעי, יולי 3, 2013
