Adventures with Cumulus Linux & Mellanox 100G Switches
We decided to replace our Juniper EX series (in a Virtual Chassis configuration) that had caused us some issues over the last couple of years in 4 separate datacenters. Ok, issues is an understatement, we had entire switch clusters crash based on how long they were running, we completely lost confidence in them.
In researching our needs and our desire to use this as an opportunity to upgrade our backbone (and not just admit we made a bad decision with Juniper), we decided on Mellanox’s Spectrum SN2010 series switches with a mix of 25G and 100G ports in a half-width chassis; Yes, 2 switches per 1U of rack space, each with redundant PSUs. Mellanox has a great reputation in the HPC realm, and are focused on just switches and NICs. To prove out our solution, our vendor, Vology, was able to get us a couple of demo units before shelling out the money for our 8 new switches.