Benchmark Optimized BIOS Settings for Supermicro SuperServer SYS-5028D-TN4T

Posted by Paul Braren on Jan 13 2016 (updated on Jan 16 2016) in
  • ESXi
  • Virtualization
  • HomeLab
  • HomeServer
  • Windows
  • HowTo
  • Back in early November of 2015, I came up with a set of performance optimized BIOS settings that seemed to squeak out the best performance from this particular system:

    Given there are so many permutations and use cases, no BIOS setting combination is perfect for every benchmark, and the User Manual doesn't exactly help. But I believe this article is a solid start. Benchmarking is a timesink, and I was really just trying to come up with a way to get close to top speeds, and a repeatable procedure for others to replicate my results, as they establish whether their new SuperServer Bundle1/2/3 purchase is performing up to speed.

    Changing to a configuration that will likely burn a bit more power probably isn't right for everybody. I'd still recommend you stick with the same BIOS settings that Bundle 1/2 came to you with from Wiredzone, detailed here:

    2016-01-13_23-03-31

    Yes, you may lose about 5% performance overall, but you'll also enjoy lower fan noise and watt burn, especially if left running 24x7, which is a reasonable thing to do, since this system generally uses roughly 45 watts with Windows 10 at idle, ~60 watts with VMware ESXi and a VM or two left idling, and ~110 watts under a busy CPU load, fully decked out with 8 drives.

    Remember, the BIOS isn't everything, you may have some easy tweaks to your OS to do as well. See for example what changing one setting in VMware ESXi 6.0 Update 1b can do for you, pictured at right. This was with only 2 VMs running, vCenter, and Windows 10, with the default VM settings of 2048MB of RAM and 2 vCPUs, and a thin provisioned C: drive, with that VMFS datastore on the Samsung 950 PRO M.2 NVMe SSD.


    Benchmark Optimized SYS-5028D-TN4T BIOS Settings - Video

    BIOS configuration video that follows along with the step-by-step instructions below, in realtime. You can pause/unpause at each step, to make it even easier to follow along.

    Benchmark Optimized SYS-5028D-TN4T BIOS Settings - Step-by-step

    This procedure applies to any owner of the X10SDV-TLN4F motherboard.

    Note, this procedure asks you to take a moment to save your existing BIOS settings. This way, you can quickly and easily revert later on, such as when you're done with benchmarking using the temporary benchmark BIOS settings below. If you forget and reset to defaults, no problem, you can manually get back to the Recommended BIOS Settings.

    1. Open Console Redirect session in IPMI iKVM interface (point browser to management IP)
    2. Select "Power Control" and "Set Power On"
    3. Press "Del" key when prompted, to enter BIOS configuration
    4. Go to "Save & Exit" tab
      Save as User Defaults
      Save configuration / Yes
      Restore Optimized Defaults
      Load Optimized Defaults? / Yes
      Save Changes and Reset
      Save configuration and reset? / Yes
    5. After system restart, Press "Del" key when prompted, to enter BIOS configuration
    6. Advanced / Boot Feature
      Bootup NumLock State [Off]
    7. Advanced / Boot Feature
      Power Button Function [4 Seconds Override]
    8. Advanced / Boot Feature
      Restore on AC Power Loss (Stay Off)
    9. Press ESC
    10. Advanced / CPU Configuration / Advanced Power Management Configuration / CPU C State Control
      CPU C State [Disable]
    11. Advanced / CPU Configuration / Advanced Power Management Configuration / CPU Advanced PM Tuning / Energy Perf BIAS
      Energy Performance Tuning [Disable]
    12. Advanced / CPU Configuration / Advanced Power Management Configuration / CPU Advanced PM Tuning
      Energy Perf BIAS Setting [Performance]
    13. Advanced / CPU Configuration / Advanced Power Management Configuration / CPU Advanced PM Tuning / CPU Advanced PM Tuning / Program PowerCTL_MSR
      Energy Efficient Turbo [Disable] - (source)
    14. Advanced / PCIe/PCI/PnP Configuration
      M.2 PCI-E 3.0 X4 [EFI]
      Slot 7 PCI-E 3.0 x16 [EFI]
    15. Advanced
      Onboard Video Option ROM [EFI]
    16. Security
      CSM Support [Disabled] (it may warn you about video, if so, save changes and reboot, then resume at this step, and try again)
    17. BOOT
      Boot Mode Select [UEFI]
    18. Save & Exit
      Save Changes and Reset
      Save configuration and reset? / Yes
    19. Use a browser to login to IPMI, choose Configuration / Fan Mode
      Fan Mode "Set to Full Speed"
    20. Save and Exit
    21. Wait to see that your preferred device begins to boot, then you're done!

    See also at TinkerTry

    Supermicro SuperServer 5028D-TN4T booting VMware ESXi 6.0, showing watts used. Jul 06 2015.
    Supermicro SuperServer 5028D-TN4T booting Windows 8.1, showing watts used. Jul 06 2015.
    Power efficiency of a Z68 Motherboard System, using a CyberPower UPS's LCD Display to measure watts. Aug 05 2011 - yeah, we've come a long way, in terms of efficiency!