A while back I finally made the decision to replace the backbone in my home tech stack, an i7 based ESXi machine built many many years ago featuring 32GB of RAM and about 10GB hardware raided storage.
After seeing some all-chinese motherboards in my various feeds I was curious as to how they would hold up, especially one of them Dual socket Xeon boards. The thing is, there are a lot of second hand/development Xeon CPUs on eBay for relatively cheap… see where I am going with this? Core galore! Perfect for a white box/homelab machine.
From the get go this was to be one of those “if it works it works” exercises and I went into this with fairly low expectations as to what would eventually show up. The ingredients were (prices at time of purchase):
- HUANANZHI X99 DUAL-T8D Motherboard (eBay) $295
- Matching pair of Intel Xeon E5-2673V3 CPU 12-Core 2.4GHz SR1Y3 LGA 2011-3 CPUs (eBay) $140
- Samsung 128GB 4x32GB DDR3-14900L 1.5C DDR3-1866 MHz ECC LRDIMM RAM (eBay) $305
Apart from the above, I’ve used some bits and pieces I had lying around:
- Inter-Tech 4F28 Mining Rack 4U 19″
- EVGA 500 BR 500W PSU
- IBM ServeRAID M5014 (running LSI 9260-8i Firmware)
- Intel PRO 1000-family NIC (for ESXi compatibility)
…and since the drives in my current array were pretty old, I ordered a set of Toshiba N300 NAS HDWG180UZSVA drives from my goto shop, planning to simply migrate the vmdks. This tuned out to be a mistake…
Motherboard
Honestly, this was the part I was most worried about. I had no experience buying no-name motherboards, let alone one that at most had a few youtube mentions from fringe adopters like myself. I was (seriously) fully expecting to set BIOS values using translate in Google Lens.
The one criteria I had (besides processor compatibility of course) was DDR3 support, more on that later.
During installation everything worked like a charm and just like any other motherboard I’ve installed before. The BIOS is in plain english and has the most insane amount of tweaking capabilities I’ve ever seen. Practically every mode and timing throughout every part of the MB can be set, but luckily everything seems happy at it’s default. I guess the one thing to complain about is the very sparse manual, but if you’ve build a few systems before the install should present no challenges what so ever.
CPUs
Choices here were a bit dictated by what was available at the Evil bay on the day that I looked. I knew I was in for a hefty TDP either way, and a Ryzen or similar would probably be better long term (especially with energy prices in Europe being what they are). But the goal here was to build something with an abundance or cores on a budget. When running dual-CPU it’s always best to get a matching pair and eventually I settled for a pair of E5-2673V3 from a seller with descent reviews. At 12 cores a pop using HT, they give me a whopping 12x2x2=48 threads to play with at 2.4Ghz for a mere $140. Under $3 per thread 🙂
They eventually arrived and both worked as expected. Buying QS/ES CPUs (not saying mine are) can be a little iffy, so make sure you read up on what that entails if you want to go down that road.
Memory
Since I was going to run a bunch of VMs I wanted a fair bit of memory, and this on a budget is why I went down the DDR3 route. They are dirt cheap second hand on the Evil bay (even ECC), and plenty fast for what I intend to use my rig for. Not a lot of motherboards support DDR3 these days, so if that is what you are eyeing make sure to double check that.
Conclusion
This was a build mostly for fun, even though I also had the need to upgrade my old white box. There are some limitations to the concept, like the ESXi license only allowing for 8 cores per guest. Also, I can not go up above ESXi v6.x due to my Raid controller not being supported any longer. Oh, and those Toshiba N300 disks… I actually ended up ordering 10 of them, 5 for me and 5 for a friend. Out of those, 5 failed catastrophically and within a few hours of each other after about 6 months of use. The other 5 lasted 6-7 months before they started showing pre-failure signs and all 10 were eventually returned under warranty. Goes to show, never build an array using the same batch, model or maybe even manufacturer drives.
As for durability, this machine has been on 24/7 for close to a year now and has during that time performed flawlessly. Onboard USB controllers and everything but the NIC works fine with VMware and it’s an awesome setup for quickly spinning up guests and trying stuff out, as well as running some more rigid guests such as file servers and docker hosts.
Total costs for what I actually really had to upgrade from my old i7 rig was about $800 with import taxes. I then ended up buying new disks and a few other bits and pieces that technically was not really needed for my upgraded box to function but that is another story.
Every part purchased second hand and/or from China ended up arriving and working as expected. Apart from somewhat lengthy transportation times (as always) everything worked out great. This is an expensive piece of kit, second hand parts or not. But its also the only machine constantly powered on in my home and it basically runs everything from home automation to movie night. Happy geeking!
