Hands-on the new Supermicro SuperServer SYS-E300-9D featuring 4 core Intel Xeon D-2123IT: testing and analysis

Posted by Paul Braren on May 24 2018 (updated on May 29 2018) in
  • CPU
  • ESXi
  • HomeLab
  • HomeServer
  • Virtualization
  • Top-rear-view-Supermicro-SuperServer-SYS-E300-9D-pics-by-TinkerTry
    The optional PCIe riser adapter and OC-SLG3-2M2 2x M.2 NVMe adapter seen pictured in the PCIe HHHL bay do not come with the system.

    Thanks to your readership, TinkerTry has been able to feature world-first Xeon D unboxing and testing articles and videos these past 3 years. It all started back in June of 2015, when I live streamed the unboxing and testing of the original Xeon D-1540 based SYS-5028D-TN4T Supermicro SuperServer. This was soon followed by the the first look inside the tiny 1U half and third width SYS-E300-8D and SYS-E200-8D SuperServers, inclusing with dB and watt measurements featured in the comparison table.

    Later on, the voted-for then-Bundled 12 core Xeon D-1567 variant of the beloved SYS-5028D-TN4T mini-tower were revealed. And now, the world's first inside look at the (evaluation/loaner) SYS-E300-9D Supermicro SuperServer!

    Floating-front-view-Supermicro-SuperServer-SYS-E300-9D-pics-by-TinkerTry.JPG
    Supermicro SuperServer SYS-E300-9D

    I don't normally start an article with assigning my readers some homework, but this time around, it's important to have a bit of context, before we get into the weeds about this tiny new SuperServer.

    Let's start with Step 1 that features my initial impressions that were solely based on the spec sheets and Intel PR, long before my eval unit arrived.

    Step 1 - Read this

    supermicro-sys-e300-9d-superserver-is-the-only-xeon-d-2100-for-home-labs

    Next up, Patrick Kennedy's fantastic look at the performance of the 8 core motherboard this new SuperServer is based on.

    Step 2 - Read this

    supermicro-x11sdv-4c-tln2f-review-with-intel-xeon-d-2123it

    Step 3 - Watch this

    • Unboxing, with every component shown.
    • A sneak peek at a portion of my new home lab workbench making its debut, with new capabilities for having some fun together that I'm pretty darn enthused about!
    • Some sense of how the HHHL PCIe bracket and slot works.
      (later videos will feature details on installing the right-angle PCIe adapter)
    • Let's get a U.2 Optane 900P drive and a M.2 Samsung 960 EVO 1TB SSD installed in there
    • Time to boot up VMware ESXi 6.7, and have a look around the vSphere Host Client.
    • Some rough measurement of watt burn and noise levels, at idle

    Like most of my videos, there are some little surprises lurking in there, and some mistakes. I need to leave some mystery for you now, and fodder to make fun of me, don't I? Get some popcorn out, it's an hour long, oh my!

    Finally, without further ado, I present to you the world's first Supermicro SYS-E300-9D unboxing and preliminary testing video. Brought to you in exquisite 4K 30fps glory. It's a rather rough cut, with only basic edits to speed things along, and a few annotations. I've removed most of the uneventful parts.

    World's First Supermicro SuperServer SYS-E300-9D unboxing & BIOS settings [4K]

    Several more videos below, but you might want to keep reading first.

    Step 4 - Read this

    Is it an engineering sample, or is this considered production ready?

    Here's where things get a little strange.

    Front-view-Supermicro-SuperServer-SYS-E300-9D-pics-by-TinkerTry.JPG
    Supermicro SuperServer SYS-E300-9D

    I cannot be sure this isn't an engineering sample. That means that in all honestly, until I know for sure, I'm not going to spend a whole lot of my time away from my constant vSphere 6.7 tinkering to measure this unit super carefully. Why? Because the product that ships could differ in meaningful ways, such as changed IPMI settings and fan speeds, which would invalidate any dB meter measurements and power usage measurements I make now.

    Thanks for the memories

    Also worth noting that every time I go to produce a video where I do any sort of testing, I need to rip all the RAM from my SYS-5028D-TN4T 8 core system, leaving it useless for ESXi 6.7 tests I'm also doing. Why? Supermicro didn't have any memory to loan me. Since the product pages doesn't list compatible DIMM part #s yet, I was relieved when my DIMMs "just worked," at the full, rated 2400MHz speed. But having 2666MHz 32GB modules would have been preferred, with the product page very recently updated to include recommended memory, which is only about 3% more expensive than 2400MHz 32GB modules right now, at least at Wiredzone. Of course, having 4 128GB modules would have been really fun, to reach the Xeon D-2100's maximum of 512GB total RAM, and to tax that power supply a little further. Very important to note that this particular system's tested memory page maxes out at 32GB currently, not sure that will ever budge.

    2018-05-24_14-44-38-supported-memory-Xeon-D-2100--TinkerTry

    84 watt power supply

    SYS-E300-9D-84-watt-power-supply--TinkerTry

    I've reached out to Supermicro for comment on the status of the power supply that was shipped to me a full two weeks ago. Is the power supply, and the system, production ready? I still haven't heard back. So it's high time I tell you exactly what I observed so far, intending to update this article once I learn more.

    So far, it sure appears that Supermicro simply took the two year old SuperChassis E300 (part # CSE-E300), and shipped it with the larger of the two listed/tested power supplies: listed on the system's product page:

    Optional 60W/84W, DC-DC Power Adapter or DC/DC Power converter board

    and stuck the new Super X11SDV-4C-TLN2F motherboard in there and sent it to me. Why is that and issue? The video below will clearly demonstrate why it's insufficient. I was able to get the server to abruptly power down after exerting a bit of load to all of the CPU cores, bringing the watt use above 100 watts, for just a few seconds before things got real quiet, myself included. Turns out my discover of this was on a clip that was lost. How? Well, after recording 90 minutes straight of 4K 30fps goodness, the iPhone X apparently got mad at all the accumulating heat, and turned itself off, without warning. I only noticed this later. Too late, the "surprise" moment was lost forever. Oh, the irony.

    Here's the thing. I wasn't that surprised, as I knew that even at idle, this sucker was pretty power hungry, and loud too, just as I expected. It's just physics people.

    Let's compare the TDP for the two CPUs.

    From the Supermicro Product Pages:

    • SYS-E300-9D - Intel® Xeon® processor D-2123IT, CPU TDP support 60W
    • SYS-E300-8D - Intel® Xeon® processor D-1518, CPU TDP support 35W
    compare-91201-136429
    Click to view Intel ARK comparison page.

    From the Intel ARK Comparison Pages:

    Now let's get a little definition of TDP for you:
    TDP

    Thermal Design Power (TDP) represents the average power, in watts, the processor dissipates when operating at Base Frequency with all cores active under an Intel-defined, high-complexity workload. Refer to Datasheet for thermal solution requirements.

    So yeah, Xeon D system can go well beyond the CPUs TDP values for a long while, roughly double the TDP value actually. I've done that sort of abuse successfully with my Xeon D-1567 12 core powerhouse Bundle many times, banging away at all 12 cores for over 2 hours straight with 4K video render, such as the unboxing video above.

    OBSERVATIONS/COMPARISONS

    SYS-E300-9D v SYS-E300-8D

    SYS-E300-9D Similarities to SYS-E300-8D

    • Same chassis, at least in the review unit.
    • Same 4 core count for both, 8 threads (Hyper-Threading on by default)
    • Same easy install of ESXi 6.7, with no hack for proper health monitoring, and no separate Intel 10GbE driver downloads required. This ease of install is evident in my Xeon D-1500 install article/video recorded on ESXi 6.7 launch day, and in this STH Xeon D-2100 article.
    • You can only mount a 2.5" drive on the underside of the cover (directly over the PCIe slot) if the PCIe slot is not occupied with a HHHL card

    SYS-E300-9D Advantages over SYS-E300-8D

    • More memory bandwidth for true server workloads, such as virtualization workloads where many VMs hit the memory hard, concurrently.
    • Included Supermicro CBL-SAST-0956 - U.2 OCuLink cable. This innovative and tiny locking OCuLink connector, but that part isn't on the parts list of the E-300-9D Product Page, at least not yet. Click the Product Page's Parts List link to see for yourself.
    • If you're trying to get into BIOS settings, the presence of an Intel Optane drive in the PCIe slot doesn't generate an BIOS hang at A9 error, an issue the Xeon D-1541 system exhibited. I tried to report this to Supermicro, but based on their response, they have little interest in fixing this. It's possible that they're a little tapped out with all this Spectre/Meltdown patching,and it ain't over yet.
    • 2400MHz DDR4 32GB DIMMs run at full 2400MHz (Intel Xeon D-1500 series limited that ability to ONLY 8 core models).
    • 2666MHz should work took, with SMC Module Part # MEM-DR432L-SL02-ER26/M393A4K40BB2-CTD/K4A8G045WB-BCTD listed on the newly updated Compatible Memory Page

    SYS-E300-9D Disadvantages versus SYS-E300-8D

    • More power hungry, about double the power at idle.
    • Considerably more noise, even at idle (perception/judgment involved).
    • Much larger heat sink, takes up space on Mini-ITX motherboard, resulting in fewer SATA and network connections, and NO M.2 slot. Also prevents installation of a 2.5" drive above it.
    • Just 4 SATA3 ports (versus 6).
    • Just 2 RJ45 10G NIC ports (versus the 6 RJ45 1G ports and 2 10G SFP+ ports)
      (both have a dedicated IPMI port).
    • PCIe slot is only 8 lanes (half length), and the BIOS doesn't appear to support 8x8 bifurcation of that slot, at least not yet. This means the handy and affordable passive Supermicro [AOC-SLG3-2M2](AOC-SLG3-2M2 Supermicro Add-On Card for up two M.2 NVMe SSDs - Internal, PCI-E 3.0 x8, Low-Profile, ) 2 x M.2 HHHL card isn't going to work.
    • Can't get as much extra versatility plucking this system from it's chassis, since it's still limited to 1 PCIe slot, because it's a Mini ITX design, and you'll likely need that slot for more NIC ports.

    SYS-E300-9D Unknowns

    • Is this 84 watt power supply the one they'll be shipping?
      Two-Intel-X722-10GbE-NICs--TinkerTry
      Two Intel X722 10G ports on board, here's one of them.
    • Are the two Intel X722 RJ45 1GbE/10GbE NICs susceptible to the same issues that affected a small subset of mostly 12 core and higher Xeon D-1500 system with RJ45 Intel X557 10GbE?
    • In theory, according to the Intel Xeon D-2123IT ARK page, this system should support up to 512GB of RAM total, but currently only 32GB modules are listed as compatible by Supermicro for this particular system. That means the current supported maximum is 128GB total right now, exactly the same as the Xeon D-1500 series of CPUs.
    • Is the Intel(R) VROC with VMD Technology 5.2.4.1000 implemented?
      Note that the photo below says "No Intel VMD Controllers on the system"
      2018-05-23_22-55-16
    • Will there ever be official Intel Optane P4800X support for such systems, or will that forever be preserved for only the Intel Scalable Processors. It's always sad when such boundaries preclude the use of something that likely works so well, perhaps for market differentiation reasons more than technical reasons. I don't currently have a P4800X for testing, unfortunately.
    • I don't believe this system will have NVDIMM support.
    • Assuming 512GB support arrives, I don't know whether buying 4 128GB DDR modules will ever be affordable during this product's lifetime, that's a gamble.

    Bottom line

    Maybe - if you really need more than 128GB of RAM, higher memory bandwidth for extreme workloads, and you don't mind the extra power use and noise.

    Unless I hear that a better supply will be shipping with generally available SYS-E300-9D systems, I would not currently recommend anybody buy one of these SYS-E300-9D systems. The ability to rapidly lose power to the system by simply exerting a high CPU load is not acceptable. This behavior should only be possible when the Intel CPU itself powers off the system to protect itself from high CPU temperatures, which I believe I witnessed once at around 95°C/203°F for the Xeon D 1541 CPU. During my initial SYS-E300-9D tests, I barely got past 60°C/140°F, then the power went off.

    x11sdv-12c-tp8f_front
    Supermicro's pic of the larger Flex ATX X11SDV-12C-TP8F, featuring 12 cores.

    This issue doesn't mean the Xeon D-2100 family of processors don't hold a lot of promise for a certain enthusiast. In a bigger and better enclosure and power supply, with more cores, more PCIe slots, and more connectors including M.2, all in a Flex ATX design like the
    X11SDV-12C-TP8F, the newly expanded storage and networking capabilities could really shine. But in this tiny enclosure, the compromises from stuffing this 4 core variant into such a small space are just too many.

    If you don't mind 1U noise, the 8 core Xeon D-2146NT full-width rack-mount Supermicro SuperServer SYS-5019D-FN8TP is likely a much better choice for more workloads, once it's available. It will have a 200 watt power supply, and it has the 64GB DIMMs listed as compatible, so at least we know that 256GB of RAM is fully supported on it now. See the 64GB module at Wiredzone.

    I do wish there were a mini-tower SuperServer all assembled and warranted and ready to go, featuring Intel Xeon D-2100 inside. But alas, there are not.

    Closing thoughts

    As my previous, closely-related article stated, this new Xeon D family feels more like a modest spec bump that affects a few datacenter users, but for the home lab, it's a step backward in pretty much every way that matters. It appears to be buying us some time, as Intel continues to delay getting 10 nm products to market. A for me personally, given my expensive electric rates and delicate ears, I likely be skipping the Xeon D-2100 family entirely, saving my pennies for the next generational release, which could be quite a while. Likely at least a year. Remember that it took 3 years for the Xeon D-1500 to be joined by the power hungrier Xeon D-2100 family, which is looking to be more of a supplement than a replacement.

    Given how much you might spend on M.2 NVMe SSDs and 10GbE networking, honestly, it would feel a bit like cheaping out to go with just a 4 core CPU like this system has. Ok if all you want to do is just get certified or do proof-of-concepts/self training, but even then, you may want to go with a higher core count than any of the mini-1U models, see also TinkerTry.com/compare for what it means to get the most bank for your buck.

    Meanwhile, I'll keep on trucking with what I have. Remember that 4K render story above? My Xeon D-1567 12 core CPU only rose to about 82C, but it's designed for that, and it wasn't very noisey. There was no GHz throttling. Remember, that's in a mini-tower, which is a much quieter and more versatile design. That's part of why it's the only Bundle I TinkerTry'd, then created detailed build-procedures for Wiredzone to fulfill. And that's the issue here, Supermicro chose to not make a mini-tower variant of the Xeon D-2100, at least not yet. I won't be holding my breath, given the minimal effort they put into launching just a handful of Xeon D-2100 based systems, in stark contrast to how all-in they were with the Xeon D-1500. Even the full 1U rack mount Xeon D-2100 system merely re-uses an existing chassis too, so just not a lot of Supermicro innovation there either.

    It's interesting to note that sales of the Bundles from Wiredzone remain quite strong, just as strong now as they were before the Xeon D-2100 was announced earlier this year. The BIOS release for Spectre/Meltdown was prompt, and the encouragement and support by the community of hundreds of happy Bundle owners has been awesome!

    Lid-off-top-view-Supermicro-SuperServer-SYS-E300-9D-pics-by-TinkerTry
    Click to start image gallery viewer, then right-arrow your way through. Click again or pinch to zoom in closer
    IPMI-SYS-E300-9D--TinkerTry
    SYS-E300-9D-IPMI-main-screen--TinkerTry

    Video

    World's First Supermicro SuperServer SYS-E300-9D unboxing & BIOS settings [4K]
    Supermicro SuperServer SYS-E300-9D PCIe angle bracket and PCIe card for M.2 install [4K]
    Supermicro SuperServer SYS-E300-9D DIMM install and cable rerouting for improved airflow [4K]
    Supermicro SuperServer SYS-E300-9D powers off once 84 watt power supply is easily pushed to 110 watts [4K]

    May 29 2018 Update

    The SYS-E300-9D Product Page has now been updated (thank you, commenter Sybreeder)! If you look down near the bottom of that Product Page, you'll find the See Parts List link, clicking on it reveals the part #:

    Power Adapter   MCP-250-10127-0N    1   120W Lockable Power Adapter with US power cord

    which currently goes for about $70 USD at Wiredzone.

    The CSE-E300 SuperChassis E300 Product Page hasn't been updated yet though, not listing the 120 watt power supply or any X11SDV motherboards as compatible.

    Early today, I reached out to Supermicro to see if they can quickly ship me one of these 120 watt power supplies, but no response yet.


    Disclosure: TinkerTry makes a modest commission on each Wiredzone sale only if you use one of the affiliate links found at TinkerTry. Wiredzone is an authorized reseller that charges very competitive prices. Please consider sharing the URL TinkerTry.com/superservers. This source of web site funding goes directly into delivering more value to enthusiastic fans, and it sure beats complete dependency on advertisements. No sponsored posts, and all relationships with any vendors disclosed. All hardware and software was purchased, and any rare exceptions (loaners) are clearly mentioned. I'm a very discerning buyer who is relieved to finally have a highly-upgradeable virtualization server that is also widely available. I even worked hard with Wiredzone to help make these available in EU at one time, and we're trying to bring that offering back. This Xeon family of CPUs is a solid foundation for years of fun and interesting articles to come. This common platform helps owners to help each other more effectively, reaping maximum benefit from such a significant mutual hardware investment.


    See also at TinkerTry

    superservers

    supermicro-superservers-vcg-updated-to-6-7

    how-to-install-esxi-on-xeon-d-1500-supermicro-superserver

    supermicro-sys-e300-9d-superserver-is-the-only-xeon-d-2100-for-home-labs

    tinkertry-xeon-d-1500-home-lab-links

    connecticut-vmug-usercon-2018-homelab

    meltdown-and-spectre-info

    amd-epyc-embedded-3000-series-launched

    intel-xeon-d-2100-announced

    intel-optane-ssd-dc-p4800x-series-is-storage-or-memory-with-intel-memory-drive-technology

    my-tinkertry-d-xeon-d-bundle-2-supermicro-superserver-bundle-2-of-joy

    pcie-to-m2-nvme-accessories-overview

    compare

    supermicro-superserver-sys-e200-8d-and-sys-e300-are-here

    supermicro-superserver-sys-e200-8d-and-sys-e300-pictures

    supermicro-debuts-new-compact-intel-xeon-processor-d-based-superservers
    TinkerTry First Look - world's first Supermicro SuperServer SYS-E300-8D and SYS-E200-8D unboxing!
    Supermicro SuperServer SYS-E200-8D - added 3rd 40mm fan improves cooling and reduces noise [4K]

    See also

    intel-xeon-d-2183it-benchmarks-and-review-16c-soc

    intel-xeon-d-2100-works-with-vmware-esxi-and-ubuntu-16-04-on-supermicro-x11sdv

    platform-power-consumption-first-benchmarks-intel-xeon-d-2100-series

    intel-xeon-d-2100-series-launch-central

    part-2-supermicro-superserver-e300-8d-deep-dive