Supermicro SuperServer mini-tower ordered with 64GB of memory for $1900, the ultimate home virtualization lab?

Posted by Paul Braren on Jun 23 2015 (updated on Jul 27 2015) in

My Supermicro SuperServer 5028D-TN4T is now on order, see status updates below.
JUN 30 2015 Update - New "deep dive inside" 38 minute video now available.

Back in 2013, we broke past the 32GB RAM restrictions for VMware ESXi 5.5, even the free version. But the hardware costs of getting a 64GB system was prohibitive, as was the cost of keeping all 8 of those watt-burning DIMMs running 24x7.

Memory (efficient and sort-of affordable)

That's all changing now.

32GB isn't Cool. You know what's cool? 128GB. At least when that DDR4 SDRAM is a relatively low 1.2 Volts. Now that's a big jump forward that was worth waiting for. 4x the memory, with 2x the CPU cores.

If all goes well, I'll be blessed with 2 32GB low voltage (1.2V) DDR4 DIMMS in a single efficient server within a few days. Hopefully I'll make my way to 128GB of memory, occupying all 4 slots, by the end of this year, without ripping and replacing any DIMMs.

Storage (wicked fast)

If you've followed TinkerTry for any part of these past 4 years, you'll note I've had my eye on the super-speedy M.2 SSD interface, with the addition of NVMe variants coming soon, such as the incredibly fast 2,260 MB/s read and 1,550 MB/s write Samsung SM951 NVMe. Another review here. I'll be able to leave that AHCI protocol / SATA 6Gbps interface for my low tier spinning drives and mid tier SSDs, savoring (future) NVMe for my top tier VM's primary storage needs. For now, I'll just drop my tZilla's RunCore mSATA drive into this apparently "quakeproof" SMAKN mSATA to M.2 adapter.

Yes, it's time for me to finally move away from my April 2011 moderately efficient and very successful build called vZilla, toward this tinier but way-more-powerful overall monster I'm dubbing...wait for it...

Mini vZilla! (aka, my SuperServer)

Future articles at TinkerTry will cover the basic un-boxing, installing of those 2 32GB DIMMs (with room for 2 more), and the adding of some drives to those 2 internal 2.5" bays and those 4 hot-swap 3.5" bays. Of course, powering up and getting ESXi 6.0 going (off USB) will be a blast, and you can be sure I'll be measuring the actual watt burn. Yeah, I'm looking forward to this summer fun!

CPU - Why Xeon D?

Why did I choose the System-on-a-Chip Intel Xeon D-1540?

Rarely does a leap forward like this come along, and the 4+ year wait for something worth my hard-earned dollars looked bleak at times, with the beloved Intel NUC and HP MicroServer Gen8 just not quite fitting the bill for my memory-hungry virtualization needs. So the minute I read about the long-awaited release of this new SoC (System on a Chip) back on March 9th, I couldn't wait to share the big news with all my TinkerTry readers, highlighting the efficiency (CPU+mobo using 45 watts max), power, and impressive versatility:
Intel Xeon D-1500 crams incredible virtualization lab specs into tiny mITX size, codename Broadwell-DE.

Supermicro-X10SDV-TLN4F-Motherboard-Inside — Supermicro X10SDV-TLN4F Motherboard Inside

To my amazement, that article wound up garnering the 2nd highest single-day views in TinkerTry's 4-year/588-article history. Kind of gave me a sense of the pent-up demand for moving forward with our virtualization home labs. Clearly, I'm not alone, this is good! See also: Superguide: Home virtualization server enthusiasts' colorful variety of ESXi whiteboxes.

There's considerable appeal to getting these 4 components pre-assembled with a 3 year warranty for about $1200. Bought individually from Amazon today, the total cost would be about the same, $1220:

Chassis/case Supermicro CSE-721TQ-250B (worth ~$170) discussed here, comes with the...
Power Supply 250 watt Flex ATX 80+ power supply
Motherboard Supermicro X10SDV-TLN4F (worth ~$1,050) comes with the...
CPU/fan combo permanently attached to the motherboard

What are the details on that CPU? Why, it's the Intel® Xeon® processor D-1540, 8-Cores with a beefy 12M Cache running at 2.00 GHz/2.6 GHz Turbo. Intel lists the Recommended Customer Price at $581.

Yes, that's twice as many cores as my 2011 vintage $299 Intel® Core™ i7-2600 Processor, with 4-Cores and 8M Cache running at 3.4 GHz/3.80 GHz Turbo. See detailed comparison at cpuboss here. Yeah, I've lost some GHz for more cores, that's the trade-off, and a good match for a virtual workload, with many VMs asking for compute power, simultaneous.

Yes, this is no Atom (Avoton/Rangely). It's an efficient little rip-roaring beast. On paper anyway.

Xeon-D-1540-Advanced-Features — Xeon D-1540 Advanced Virtualization Features including VT-x and VT-d

Arrives barebones, a dark grey "white box," so I'll need to add memory and disk. But I like it that way, and I already have some SSDs and 4TB and 6TB drives.

A very recent scare I had with vZilla really upped the urgency of getting something newer. While doing some tinkering with a bunch of VMs, I found they were all suddenly getting very slow, no matter which datastore they happened to be on. Like 1/5th speed. Then the whole server abruptly powered off. Not good. Powered up, same deal, it powered right back off in about 15 minutes. So I waited for a bit, and this time, watched over it. Powered up. Barely enough time to boot, it powered off again. No PSOD, just an abrupt nothingness. Uh oh.

Quick visit to the BIOS clearly showed that the Intel Core i7 was quickly climbing to 92 Celsius / 198 Fahrenheit. Yowza. Yeah, my Corsair H60 CPU cooler pump had failed after 4 1/3 years of 24x7 use. The ASRock BIOS and CPU combo was simply doing the right thing, providing me thermal protection. Averting the meltdown of me and my CPU. Within a couple of Amazon-Prime days, I had a fresh Arctic Silver tube in hand, so I could replace the water cooler with the stock Intel Core i7 fan for now, time-lapse video of the surgery here.

This incident was a sign of this system's age. That scare, coupled with knowing I have some certification studying to do, meant it was time to think about what's next. I also really want to get back to some more ESXi and Hyper-V nesting testing, and know full well that my current 32GB cap would really be a pain point for the balance of 2015.

So with the Why behind us, it becomes:

How?

How did I get my Supermicro SuperServer 5028D-TN4T on order?

I've been watching for these to start shipping since April, the original shipping month that various articles cited. Then last week, I started to see more listings show up, all showing back-ordered, unavailable, or pre-order. So I dove in to try to find out what was really going on here.

Starting with the Supermicro SuperServer 5028D-TN4T Product Page, it still simply says "Coming Soon." So it was off to the Where to Buy tab, I then chose North America, US East, and noticed it seemed to be the same list for the other US geos.

I then used Chrome, held down Ctrl, then opened a tab for each of the listed System Integrators/Resellers, and searched each site for 5028D-TN4T or some variant. Here's the only sites I located that had some sort of listing (prices are as of June 23 2015, and are subject to change at any time, of course):

AVADIRECT ($1,715.35 starting price for bundles)
PC Connection ($1,200.34, Availability: Temporarily Out-of-Stock)
PCM ($1,249.99, Availability: Call)
THINKMATE High Performance Computing ($1,740 starting price for bundles)
Wiredzone ($1,199.62, Availability: Out-of-Stock)
(on June 24, this changed to "Built To Order", explained below)

I then tried to chat, email, or call those companies, to see if anybody could give me definitive ship dates. To my surprise, Wiredzone not only replied quickly, excerpt below:

Sent: Monday, June 22, 2015 4:18 PM
Subject: RE: Online customer inquiry

Hi Paul,

The ETA for this model is tomorrow 06/23 in California.

but they also quickly handled my follow-on questions, and took my phone call. Wiredzone takes credit cards or PayPal, ships to many countries, and seems to have an excellent reputation, with a ResellerRatings 9.87 out of 10 here. As far as customer service, I did find it easy to add memory to my original order, with the additional PayPal payment handled over chat with ease, in the evening.

Wiredzone currently offers no affiliate links. That means I make no commission on sales through their site. I'm simply passing along my own story about one of the many worldwide Supermicro Authorized Resellers, which you'll also likely notice doesn't include Amazon or Newegg.

Place your order:

Supermicro SuperServer 5028D-TN4T

Here's Wiredzone's SKU 10025066 MFG Part Number: SYS-5028D-TN4T listing:

wiredzone.com/supermicro-servers-compact-embedded-processor-sys-5028d-tn4t-10025066

Wiredzone states here that the 3 year warranty is provided by Supermicro.

Samsung 32GB ECC DDR4 1.2V-2133 SDRAM Memory

(Qty. 2 or Qty. 4 recommended)

Turns out that the Supermicro tested 32GB DIMM is the Samsung M393A4K40BB0-CPB, which Wiredzone also has at a reasonable in-stock $355.00, under-cutting Amazon's $367.95 non-Prime listing. The same memory is currently $429.99 on Newegg here.

Here's Wiredzone's SKU: 10024305 MFG Part Number: MEM-DR432L-SL01-ER21 listing:

Supermicro 32GB DDR4 PC4-17000 (2133MHz)288p LRDIMM

32GB-Samsung-M393A4K40BB0-CPB-DDR4-SDRAM — Samsung M393A4K40BB0-CPB 32GB DDR4 SDRAM Module

Note that I'm aware that page 4-4 of the Supermicro 5028D-TN4T User's Manual doesn't specify 32GB DIMMs. [JUN 29 2015 - I reached out to Supermicro directly, giving feedback that the manual really needs to include 32GB DIMM info, including clarifying whether there is any difference in speeds between using just the 2 recommended blue slots for 2 32GB DIMMs, or using all 4 slots]

Of course, you can go with the free ground shipping option, from California for US customers, and save yourself the large FedEx 2 day fee I incurred getting this item cross-country quickly:

TinkerTry-SuperServer-June-22-2015-Order-information

JUN 24 2015 Update

The memory has arrived. The SuperServer is due to arrive Thursday, June 25th by 8pm eastern, 15 lbs, apparently drop-shipped from Supermicro in San Jose, CA, which explains the "Built To Order (Ships in 3-4 Days)" designation on Wiredzone's product listing. Delivery exactly when Wiredzone promised.
Your order fulfillment and shipment times may vary, of course.

Considering doing a live event of the unboxing and power-up tests tomorrow evening. Please let us know if you're interested by dropping a comment below!

See also my related Google Plus post here

JUN 25 2015 Update

I'm also glad I'll have Supermicro Intelligent Management with many IPMI features including remote video over IP that'll really help when recording installation videos. Also means I won't need my handy but clumsy Lantronix Spider for this system.

08:00 am - It's getting closer!

11:00 am - It's here! Not the most subtle box on the front-door step, but I was home, and could have asked for delivery signature, if I had chosen to.

The live un-boxing and initial power-up will happen tonight, June 25th, likely at 8pm Eastern time. I'll likely be using a Google Hangout On Air, and will likely create a new post right here on TinkerTry, to share all the details.

JUN 25 2015 Update #2

The live broadcast happened and went pretty well. Full replay of the unboxing and power-up-ing video now available here, with more much-clearer videos coming soon.