Supermicro SuperServer mini-tower ordered with 64GB of memory for $1900, the ultimate home virtualization lab?
My Supermicro SuperServer 5028D-TN4T is now on order, see status updates below.
JUN 30 2015 Update - New "deep dive inside" 38 minute video now available.
Back in 2013, we broke past the 32GB RAM restrictions for VMware ESXi 5.5, even the free version. But the hardware costs of getting a 64GB system was prohibitive, as was the cost of keeping all 8 of those watt-burning DIMMs running 24x7.
Memory (efficient and sort-of affordable)
That's all changing now.
32GB isn't Cool. You know what's cool? 128GB. At least when that DDR4 SDRAM is a relatively low 1.2 Volts. Now that's a big jump forward that was worth waiting for. 4x the memory, with 2x the CPU cores.
If all goes well, I'll be blessed with 2 32GB low voltage (1.2V) DDR4 DIMMS in a single efficient server within a few days. Hopefully I'll make my way to 128GB of memory, occupying all 4 slots, by the end of this year, without ripping and replacing any DIMMs.
Storage (wicked fast)
If you've followed TinkerTry for any part of these past 4 years, you'll note I've had my eye on the super-speedy M.2 SSD interface, with the addition of NVMe variants coming soon, such as the incredibly fast 2,260 MB/s read and 1,550 MB/s write Samsung SM951 NVMe. Another review here. I'll be able to leave that AHCI protocol / SATA 6Gbps interface for my low tier spinning drives and mid tier SSDs, savoring (future) NVMe for my top tier VM's primary storage needs. For now, I'll just drop my tZilla's RunCore mSATA drive into this apparently "quakeproof" SMAKN mSATA to M.2 adapter.
Yes, it's time for me to finally move away from my April 2011 moderately efficient and very successful build called vZilla, toward this tinier but way-more-powerful overall monster I'm dubbing...wait for it...
Mini vZilla! (aka, my SuperServer)
Future articles at TinkerTry will cover the basic un-boxing, installing of those 2 32GB DIMMs (with room for 2 more), and the adding of some drives to those 2 internal 2.5" bays and those 4 hot-swap 3.5" bays. Of course, powering up and getting ESXi 6.0 going (off USB) will be a blast, and you can be sure I'll be measuring the actual watt burn. Yeah, I'm looking forward to this summer fun!
CPU - Why Xeon D?
Why did I choose the System-on-a-Chip Intel Xeon D-1540?
Rarely does a leap forward like this come along, and the 4+ year wait for something worth my hard-earned dollars looked bleak at times, with the beloved Intel NUC and HP MicroServer Gen8 just not quite fitting the bill for my memory-hungry virtualization needs. So the minute I read about the long-awaited release of this new SoC (System on a Chip) back on March 9th, I couldn't wait to share the big news with all my TinkerTry readers, highlighting the efficiency (CPU+mobo using 45 watts max), power, and impressive versatility:
Intel Xeon D-1500 crams incredible virtualization lab specs into tiny mITX size, codename Broadwell-DE.
To my amazement, that article wound up garnering the 2nd highest single-day views in TinkerTry's 4-year/588-article history. Kind of gave me a sense of the pent-up demand for moving forward with our virtualization home labs. Clearly, I'm not alone, this is good! See also: Superguide: Home virtualization server enthusiasts' colorful variety of ESXi whiteboxes.
There's considerable appeal to getting these 4 components pre-assembled with a 3 year warranty for about $1200. Bought individually from Amazon today, the total cost would be about the same, $1220:
- Chassis/case Supermicro CSE-721TQ-250B (worth ~$170) discussed here, comes with the...
- Power Supply 250 watt Flex ATX 80+ power supply
- Motherboard Supermicro X10SDV-TLN4F (worth ~$1,050) comes with the...
- CPU/fan combo permanently attached to the motherboard
What are the details on that CPU? Why, it's the Intel® Xeon® processor D-1540, 8-Cores with a beefy 12M Cache running at 2.00 GHz/2.6 GHz Turbo. Intel lists the Recommended Customer Price at $581.
Yes, that's twice as many cores as my 2011 vintage $299 Intel® Core™ i7-2600 Processor, with 4-Cores and 8M Cache running at 3.4 GHz/3.80 GHz Turbo. See detailed comparison at cpuboss here. Yeah, I've lost some GHz for more cores, that's the trade-off, and a good match for a virtual workload, with many VMs asking for compute power, simultaneous.
Yes, this is no Atom (Avoton/Rangely). It's an efficient little rip-roaring beast. On paper anyway. Arrives barebones, a dark grey "white box," so I'll need to add memory and disk. But I like it that way, and I already have some SSDs and 4TB and 6TB drives.
A very recent scare I had with vZilla really upped the urgency of getting something newer. While doing some tinkering with a bunch of VMs, I found they were all suddenly getting very slow, no matter which datastore they happened to be on. Like 1/5th speed. Then the whole server abruptly powered off. Not good. Powered up, same deal, it powered right back off in about 15 minutes. So I waited for a bit, and this time, watched over it. Powered up. Barely enough time to boot, it powered off again. No PSOD, just an abrupt nothingness. Uh oh.
Quick visit to the BIOS clearly showed that the Intel Core i7 was quickly climbing to 92 Celsius / 198 Fahrenheit. Yowza. Yeah, my Corsair H60 CPU cooler pump had failed after 4 1/3 years of 24x7 use. The ASRock BIOS and CPU combo was simply doing the right thing, providing me thermal protection. Averting the meltdown of me and my CPU. Within a couple of Amazon-Prime days, I had a fresh Arctic Silver tube in hand, so I could replace the water cooler with the stock Intel Core i7 fan for now, time-lapse video of the surgery here.
This incident was a sign of this system's age. That scare, coupled with knowing I have some certification studying to do, meant it was time to think about what's next. I also really want to get back to some more ESXi and Hyper-V nesting testing, and know full well that my current 32GB cap would really be a pain point for the balance of 2015.
So with the Why behind us, it becomes:
How did I get my Supermicro SuperServer 5028D-TN4T on order?
I've been watching for these to start shipping since April, the original shipping month that various articles cited. Then last week, I started to see more listings show up, all showing back-ordered, unavailable, or pre-order. So I dove in to try to find out what was really going on here.
Starting with the Supermicro SuperServer 5028D-TN4T Product Page, it still simply says "Coming Soon." So it was off to the Where to Buy tab, I then chose North America, US East, and noticed it seemed to be the same list for the other US geos.
I then used Chrome, held down Ctrl, then opened a tab for each of the listed System Integrators/Resellers, and searched each site for 5028D-TN4T or some variant. Here's the only sites I located that had some sort of listing (prices are as of June 23 2015, and are subject to change at any time, of course):
AVADIRECT ($1,715.35 starting price for bundles)
PC Connection ($1,200.34, Availability: Temporarily Out-of-Stock)
PCM ($1,249.99, Availability: Call)
THINKMATE High Performance Computing ($1,740 starting price for bundles)
- Wiredzone ($1,199.62, Availability: Out-of-Stock)
(on June 24, this changed to "Built To Order", explained below)
I then tried to chat, email, or call those companies, to see if anybody could give me definitive ship dates. To my surprise, Wiredzone not only replied quickly, excerpt below:
Sent: Monday, June 22, 2015 4:18 PM
Subject: RE: Online customer inquiry
The ETA for this model is tomorrow 06/23 in California.
but they also quickly handled my follow-on questions, and took my phone call. Wiredzone takes credit cards or PayPal, ships to many countries, and seems to have an excellent reputation, with a ResellerRatings 9.87 out of 10 here. As far as customer service, I did find it easy to add memory to my original order, with the additional PayPal payment handled over chat with ease, in the evening.
Wiredzone currently offers no affiliate links. That means I make no commission on sales through their site. I'm simply passing along my own story about one of the many worldwide Supermicro Authorized Resellers, which you'll also likely notice doesn't include Amazon or Newegg.
Place your order:
Supermicro SuperServer 5028D-TN4T
Here's Wiredzone's SKU 10025066 MFG Part Number: SYS-5028D-TN4T listing:
Wiredzone states here that the 3 year warranty is provided by Supermicro.
Samsung 32GB ECC DDR4 1.2V-2133 SDRAM Memory
(Qty. 2 or Qty. 4 recommended)
Turns out that the Supermicro tested 32GB DIMM is the Samsung M393A4K40BB0-CPB, which Wiredzone also has at a reasonable in-stock $355.00, under-cutting Amazon's $367.95 non-Prime listing. The same memory is currently $429.99 on Newegg here.
Here's Wiredzone's SKU: 10024305 MFG Part Number: MEM-DR432L-SL01-ER21 listing:
Note that I'm aware that page 4-4 of the Supermicro 5028D-TN4T User's Manual doesn't specify 32GB DIMMs. [JUN 29 2015 - I reached out to Supermicro directly, giving feedback that the manual really needs to include 32GB DIMM info, including clarifying whether there is any difference in speeds between using just the 2 recommended blue slots for 2 32GB DIMMs, or using all 4 slots]
Of course, you can go with the free ground shipping option, from California for US customers, and save yourself the large FedEx 2 day fee I incurred getting this item cross-country quickly:
JUN 24 2015 Update
The memory has arrived. The SuperServer is due to arrive Thursday, June 25th by 8pm eastern, 15 lbs, apparently drop-shipped from Supermicro in San Jose, CA, which explains the "Built To Order (Ships in 3-4 Days)" designation on Wiredzone's product listing. Delivery exactly when Wiredzone promised.
Your order fulfillment and shipment times may vary, of course.
Considering doing a live event of the unboxing and power-up tests tomorrow evening. Please let us know if you're interested by dropping a comment below!
See also my related Google Plus post here
JUN 25 2015 Update
I'm also glad I'll have Supermicro Intelligent Management with many IPMI features including remote video over IP that'll really help when recording installation videos. Also means I won't need my handy but clumsy Lantronix Spider for this system.
08:00 am - It's getting closer!
11:00 am - It's here! Not the most subtle box on the front-door step, but I was home, and could have asked for delivery signature, if I had chosen to.
The live un-boxing and initial power-up will happen tonight, June 25th, likely at 8pm Eastern time. I'll likely be using a Google Hangout On Air, and will likely create a new post right here on TinkerTry, to share all the details.
JUN 25 2015 Update #2
The live broadcast happened and went pretty well. Full replay of the unboxing and power-up-ing video now available here, with more much-clearer videos coming soon.
See also at TinkerTry
Deep dive inside video of the Supermicro SuperServer 5028D-TN4T including component installation
JUL 02 2015
Supermicro SuperServer 5028D-TN4T Photo Gallery
JUL 01 2015
Installing Windows on the Supermicro SuperServer 5028D-TN4T
JUL 01 2015
Supermicro SuperServer 5028D-TN4T accessories and aspirations
JUN 29 2015
Supermicro SuperServer 5028D-TN4T first impressions
JUN 26 2015
TinkerTry's Xeon D-1540 fueled ESXi 6.0 home lab build begins LIVE!
JUN 25 2015
NVMe uses PCI Express to finally unleash the true power of NAND flash memory
APR 22 2015
- Intel Xeon D-1500 crams incredible virtualization lab specs into tiny mITX size, codename Broadwell-DE
MAR 09 2015
Nice little USB flash drive choice for that ESXi in your home lab
JAN 31 2015
Haswell Z97 limited to 32GB memory max, an unfortunate reality for virtualization enthusiasts
JUN 7 2014
Superguide: Timeline of USB, SATA, and Thunderbolt speeds
AUG 24 2013
Intel's Haswell-E X99S and Atom can handle 64GB of memory, affordability TBD
AUG 11 2013
- Looking forward to breaking past 550MB/sec SATA3 SSDs, using PCIe, PCI NGFF (M.2), and software caching
JUN 23 2013
- Supermicro SuperServer 5028D-TN4T User's Manual
JUN 01 2015, by Supermicro
SUPERMICRO SYS-5028D-TN4T REVIEW SMALL BOX – BIG POWER
MAY 13 2015, by Patrick Kennedy at Serve The Home
SUPERMICRO X10SDV-TLN4F REVIEW – A MUST GET PLATFORM
APR 13 2015, by Patrick Kennedy at Serve The Home
Common Information Model (CIM) Data for Home Lab Server Hardware
MAR 16 2015, by Chris Wahl at Wahl Network
Intel® Xeon® Processor D-1540 (12M Cache, 2.00 GHz)
Q1'15, by Intel
- Buying An M.2 SSD? How To Tell Which Is Which
APR 07 2014, by MARSHALLR at Republic of Gamers