Intel Xeon D-1540 gets 5% speed bump as D-1541 with SR-IOV in a Supermicro SuperServer Feb. 2016, comparison with Core i7 6700K (Skylake), should I wait?

Posted by Paul Braren on Dec 15 2015 (updated on Feb 10 2015) in

Jan 08 2016 Update - New Xeon D Product Line-up info now available below, and in Patrick Kennedy's wonderful article, New Supermicro Intel Xeon D Motherboards – Even with 12 and 16 core chips!.

It's that time of year, when many an IT pro or PC enthusiast huddles up in their home labs and prepare for the long winter ahead by refreshing their home lab configuration. Especially if they're scheduled for time off from work. Or trying to get certified. Or backup all family PCs and VMs. Mini-tower SuperServer Bundle 2 or Bundle 3 is great for either, ready for VMware, in-stock, and shipping now from the US or EU.

With the 2015 holidays now very close, several TinkerTry visitors have asked me if I think they should wait for the minor CPU speed bump and/or SR-IOV compatibility expected to arrive with Intel's Xeon D-1541 CPU. For most people, that answer has been no, but of course, that decision is very individual. Let's start with the background on the innovative Xeon D chipset, arriving late 2Q2015 in the roughly $850 CPU/mobo SoC (System on Chip) Xeon D-1540:

CPU/GHz - Xeon D has twice the cache and cores of Skylake, at half the watts

This is a gross simplification, but a good starting point (costs will be higher, depends on how many drives are in there, how loaded the CPUs are, etc. Stepping back the GHz when there's little load doesn't tend to work under ESXi, so these numbers are actually decent average estimates for a system with an SSD or two, no GPU, and no extreme CPU workload most of the time:

Intel Xeon D-1540 system consuming 45 watts 24x7, $73.38 per year to operate at 14 cents per kW/hr, $293.52 for 4 years.
Intel Core i7 6700K system consuming 95 watts 24x7, $116.19 per year to operate at 14 cents per kW/hr, $464.76 for 4 years. That's a $171.24 difference, plus the additional thermal load on the home, which can cost roughly as much in AC costs too to counteract that, and the potential for higher noise levels. See also video about noise (with watt burn featured to) here.

5% speed boost, at most

It's hard to claim a 5% boost is expected to be very noticeable for most use-cases. So let's move on and look at SR-IOV. I admit, I actually never got around to enabling it on my Intel I350T2 PCI card in my older vZilla build, nor did I have a way to really test it back then. Worry not, Patrick Kennedy of Serve The Home has an excellent article all about SR-IOV:

THE BROADWELL-DE INTEL XEON D-1500 SERIES AND SR-IOV

Here's an excerpt:

SR-IOV is away to have a Single Root Function (e.g. an Ethernet Port) appear to be multiple separate physical devices. This is useful for virtualization because one can reduce virtualization overhead using a single port rather than adding ports and assigning them to each VM. A useful case here is adding 10GbE to a server than having the NIC shared with multiple VMs.

For me, the virtual switch you can set up under ESXi 6.0 seems to work fine, attaching it to my 1GbE or 10GbE physical interfaces, as needed. Speeds given my modest workloads seem tip top too. My initial tests done with a 2nd borrowed SuperSuper went well using just a CAT6a cable, with auto-crossover avoiding the need for a pricey 10GbE switch. With a few VMs, do I really expect SR-IOV to make a difference, and be worth the significant drawback of not being able to vMotion any VMs that leverage this feature? For me, the answer happens to be no.

All the differences between Xeon D-1540 and Xeon D-1541

Finally, let's move on to the other API features that Intel added to the 1541. I believe you'll find Intel ARK Compare Intel Products feature very handy for this:

compare-1540-1541-circled-cropped — Note the Networking Specifications section shows some new APIs, but accidentally leaves the left column blank for items that should say Yes, such as 10GBase-T. Click the image to jump to the ARK page with an added 3rd column for Skylake.

Yeah, not sure why I'd want DDR3 at this point, especially given it's less than $1000 for 4 of those tested Samsung 32GB DDR4 low voltage 1.2V modules, totaling 128GB of RAM. That's a LOT of VMs you can juggle, for a price that's likely similar to likely higher watt burn DDR3, but tested DDR3 modules aren't listed by Supermicro yet.

You may notice that Supermicro's X10SDV-TLN4F motherboard product page already shows:

Intel® Xeon® processor D-1540/1541,
Single socket FCBGA 1667;
8-Core, 16 Threads, 45W

sr-iov-screenshot — This BIOS screenshot taken from a video I had done that featured BIOS 1.0a.

We'll have to wait until the Xeon D-1541 actually arrives to know whether SR-IOV will work. I plan to have the opportunity to do exactly that, and I'm working hard to try to make sure that actually happens.

It's worth noting that activating SR-IOV requires a BIOS setting, which actually appeared (but didn't function) in the 5028D-TN4T BIOS 1.0a, then went away in 1.0b. Likely because it was reported to not work correctly. This situation was also outlined at Serve The Home forum thread, and this TinkerTry discussion.

In summary

So for folks who would answer yes, they require full SR-IOV for absolute network scalability, then yeah, I would recommend they wait until the Supermicro SuperServer SYS-5028D-TN4T arrives with the Xeon D-1541 on board.

For everybody else, especially those who only have time to tinker over the holidays, then by all means, there's no compelling reason to wait 2 more months. We have a mature BIOS 1.0b released months ago that seems to work great for all sorts of use cases I've tested, including Windows 10, boot from NVMe, and VMware ESXi 6.0 Update 1a with full NVMe and 10GbE support.

Reader mail

A European TinkerTry visitor wrote in recently:

Hello, still the bundle 3 (Supermicro Mini Tower Intel Xeon D-1540 Bundle 3 BV) of your sponsored server avaliable from Europe? I don't want to pay expensive shipping to get one.

Also I'm thinking to wait for Xeon-D 1541 version as long it adds SR-IOV. What's your recommendation?

Thanks for your time.

Here's my response, unedited:

Yes, Bundle 3 is now available and ships from The Netherlands to all points in Europe (including Spain) for $65 USD:
https://tinkertry.com/superservers#Bundle3

I have tested the hardware, and paid for the extras (USB memory, stickers, extra SATA cable, speaker), so the profit for me is extremely tiny. It’s more about building a community of owners, with more announcements coming to TinkerTry soon. I have even personally flown down to Wiredzone in Miami last week:
https://twitter.com/paulbraren/status/672732331581579264
to be sure I’m comfortable putting my reputation on the line for this very affordably priced reseller (all other US resellers charge more, usually much more, especially when adding disk or RAM). I’m so glad Wiredzone is now taking orders to EU too.

Admittedly, Intel never did get SR-IOV working with 1540, so if you really need that SR-IOV feature, waiting until Feb. 2016 for the 1541 to arrive would be best for your happiness:
https://tinkertry.com/intel-grows-xeon-d-platform#Below2
the performance boost will be tiny (<5%), and the pricing will likely be the same.

Intel often lets dates slip, original 1541 announcement indicated Nov 2015, but that never came true.

I hope you’re now a more informed potential buyer, and hope the EU pricing is favorable, for this fully authorized Supermicro reseller.

Where you wind up buying, you will likely have a hell of a good tinkering with it!
https://twitter.com/paulbraren/status/671907722137370625

Jan 06 2016 Update

See also full Xeon D line-up from Supermicro here:

Supermicro Solutions based on Intel® Xeon®-D processors

and the closely related Google+ by Jeroen Wiert Pluimers here:

that refers to wonderful article with all sorts of great information at wiert.me:

D-1541 with increased speed plus support for SR-IOV and DDR3 memory might end up in Supermicro SYS-5028D-TN4T somewhere in februari

Thank you, Jeroen, for all this great information, and for the EU Bundle 3 shout out!

New TinkerTry'd Supermicro SuperServer Bundle 2 and Bundle 3 - lower cost of entry for your VERY upgradable virtualization home lab, now shipping from US and EU!

So yes, we now know for sure that the same form factor Mini-ITX is arriving in 12 and 16 core variants. Rather likely it would also be quite happy in the same X10SDV-TLN4F chassis, unless power draw is signficantly more, we'll know soon.