Migrating to Microsoft Storage Spaces

The Decision

A couple of years ago (late 2014) we decided to ditch our two Dell Equalogic PS6000xv in favor of something new.  Microsoft Storage Spaces in windows 2012R2 with a Scale Out File Server (SOFS).  We were already a Microsoft Hyper-V only environment.  We had the licensing to do it.  We just needed some new equipment.

The Promise of Scale Out File Servers

Highly available storage for Hyper-V and SQL on commodity hardware.  We went with SuperMicro Servers and JBODs and were able to get 121TB of raw storage / 40-60 usable depending on your resiliency setting per virtual disk.  We went with SuperMicro JBODs because they were on the hardware compatibility list for Storage Spaces and are enclosure aware.  Enclosure awareness allows data to be place intelligently across multiple JBODs in such a way you can lose an entire JBOD and still be up and running.

The Gamble

We used to pay about $6,000 per year in SAN support.  Plus we would pay a premium when we bought our servers to have 24×7 4hr support from Dell/Cisco/etc.  This time we went all in with SuperMicro.  We replaced all of our servers and SAN with a Windows Hyper-V environment with Storage Spaces. We would stock parts on site for any repairs and rely on the manufacturer parts for warranty.  To date, we have had just 1 dimm fail and it was replaced by the dimm manufacturer.

The Configuration

  • 2 – Scale Out File Servers
  • 3 – JBODs
    • 45 – 1TB Drives per JBOD
  • 3 – Hyper-V Compute Nodes w/ 256GB RAM Each
  • 2 – 10GB Switches

The Parts

JBODs

  • 45 – 1TB SAS Hard Drives Each / Total of 135
  • 3 – Enclosures
  • 6 – SAS Cables

 

Scale Out File Servers

  • 2 – Barebones SuperMicro 2U Chassis & Motherboard
  • 4 – Processors
  • 16 – DIMMs (128GB each) – Want high amounts for CSV Caching
  • 4 – LSI HBAs (Non-Raid Mode)
  • 2 – Mellanox ConnectX-3

Compute Nodes

  • 3 – Barebones SuperMicro 2U Chassis & Motherboard
  • 6 – Processors
  • 48 – DIMMs (256GB each)
  • 6 – Mellanox ConnectX-3 (1 card for dedicated storage traffic, 1 for VM traffic)

Networking

  • 2 – Mellanox SX1012 Top of Rack Switches (40GB)