World's fastest consumer SSD - Samsung 950 PRO M.2 NVMe benchmark results

Posted by Paul Braren on Nov 7 2015 (updated on Jan 11 2016) in
  • Storage
  • Virtualization
  • ESXi
  • HomeServer
  • HomeLab
  • HowTo
    • B01639694M
    Available now on Amazon in 256GB and 512GB sizes, 1TB not expected until early next year.

    These past few hours, I've been quite enjoying doing some basic tests of my two new 950 PROs, 512GB capacity, ensuring I'm getting expected performance levels. I short, I am!

    This first series of tests use Windows 10, using two 950 PRO 512GB model MZ-V5P512BW SSDs at the latest firmware 1B0QBXX7. The next series will cover a Windows 10 VM running on VMware ESXi 6.0 Update 1a. I won't be using this Samsung 950 for exclusive Windows 10 use for very long. Its fate is a quick secure erase, then combined-use as both vSphere Flash Read Cache, and as a VMFS datastore. It's all explained in my storage plan, along with my Samsung 850 EVO 2TB for laptop/VM multi-boot.

    Table of Contents

    TotalBytesWrittenOver3TBin1dayFirmwareIsLatest
    Just one day of many tests. Lifetime of SSD is 400TB written. I would suggest you avoid benchmarks that do writes and run for hours or overnight. Just don't go nuts with those artificial write tests, and you'll be fine for years.

    1. Hardware
    2. Software
    3. UEFI BIOS Benchmark Settings
    4. Test method
    5. ATTO Disk Benchmark Results
    5. AS SSD Benchmark Results
    6. CrystalDiskMark Results
    7. Best of each of the 3 runs, Drive C M.2 socket and Drive D PCIe slot
    8. Video of BIOS changes
    9. Video of thermal test and fix
    10. Video of ATTO run on C: and D: concurrently
    11. Video of thermal behavior
    12. Video of iperf speed over 10GbE
    13. Thank You
    14. Disclosure

    These results are looking quite favorable, highlighting the power of the efficient Intel Xeon D-1540 SoC. This is a compact motherboard at 6.7"x6.7" (17cm x 17cm), with an integrated octacore Xeon that operates at around 45 watts. For these tests, I'm just using one tiny drive, the Samsung 950 PRO. It can only consume at most 7 watts, but sips only 70mW when idle. Idle is where most of its time is normally spent. Think about how little time SSDs are used during normal use, given how fast they read and write those tiny files touched as you use Windows for productivity tasks. These synthetic benchmarks are exactly that, synthetic, but are very useful to help us know if we're getting the expected performance out of our hardware investments.

    I realize these aren't perfect tests, no benchmark is. I also realize many purists would want to change to higher queue depths or even more repeated runs. The thing is, I need to use this hardware very soon, and I don't plan to gobble up too many TBs of those 400TB lifetime of writes, just because. This is not an evaluation. This drive is a keeper.

    My goal began mostly just to determine if the Intel 750 Series or this drive is better for my needs. But it quickly became apparent that I should go ahead and document my simple tests, so that other SuperServer Bundle 1, 2, or 3 owners can easily obtain very similar results, knowing they too are getting the expected performance that matches or exceeds the manufacturer's claims. These instructions are only valid for this particular system, your results will likely vary on other systems.

    I tried to keep things simple, using each benchmark utility at defaults, for a minimum of fuss. Rest assured that if you wish to leave your BIOS at the more efficient settings I carefully outline here:

    your performance will be more than 90% of what you see here, but your CPU will also be able to step down to 0.78GHZ when the system is idle, saving you watt burn. That's what I'll be doing, turning those conservation settings back on, as soon as I'm done with these tests.

    Now that I have these tests done, I'll also be re-running these same 3 utilities against an Intel 750 Series 400GB that I (temporarily) still have.

    I also tinkered with likely scenarios such as running 2 950 PROs through 2 instances of ATTO Disk Benchmark, simultaneously. That PCI 3.0 bus held up quite well, with both 950s getting about 75% of the speed they'd get if they were the only tests running. Not something you'd likely see in the old SATA3 days. NVMe is light on CPU too.

    So far, comparing this drive to my recent Intel 750 Series and Samsung SM951 tests, it's sure looking like this one is tops for my needs. That "Best Consumer SSD in the world" phrase I've seen seems well deserved.

    Let's cover the basics, the way I set up my tests of the Samsung 950 PRO.

    If you just want pictures/results, jump below.

    Hardware

    Software

    Benchmark Optimized SYS-5028D-TN4T BIOS Settings - Video Instructions

    Benchmark Optimized BIOS configuration video that follows along with the Step-by-Step instructions below.

    Benchmark Optimized SYS-5028D-TN4T BIOS Settings - Step-by-Step Instructions

    1. Shut down your system
    2. Open Console Redirect session in IPMI iKVM interface (point browser to management IP)
    3. Power on the system
    4. press "DEL" to run Setup (enter BIOS)
    5. Go to "Save & Exit" tab
      Restore Optimized Defaults
      Load Optimized Defaults? / Yes
      Save Changes and Reset
      Save configuration and reset? / Yes
    6. After system restart, Press "Del" key when prompted, to enter BIOS configuration
    7. Advanced / Boot Feature
      Bootup NumLock State [Off]
    8. Advanced / Boot Feature
      Power Button Function [4 Seconds Orverride]
    9. Advanced / Boot Feature
      Restore on AC Power Loss (Stay Off)
    10. Advanced / CPU Configuration / Advanced Power Management Configuration / CPU C State Control
      CPU C State [Disable]
    11. Advanced / CPU Configuration / Advanced Power Management Configuration / CPU Advanced PM Tuning / Energy Perf BIAS
      Energy Performance Tuning [Disable]
    12. Advanced / CPU Configuration / Advanced Power Management Configuration / CPU Advanced PM Tuning
      Energy Perf BIAS Setting [Performance]
    13. Advanced / CPU Configuration / Advanced Power Management Configuration / CPU Advanced PM Tuning / CPU Advanced PM Tuning / Program PowerCTL_MSR
      Energy Efficient Turbo [Disable] - (source)
    14. Advanced / PCIe/PCI/PnP Configuration
      M.2 PCI-E 3.0 X4 [EFI]
      Slot 7 PCI-E 3.0 x16 [EFI]
    15. Advanced
      Onboard Video Option ROM [EFI]
    16. Security
      CSM Support [Disabled]
      It will warn you about video settings needing to be saved first.
      Go to "Save & Exit" tab, Save Changes and reboot, then resume at this step.
      CSM Support [Disabled]
    17. BOOT
      Boot Mode Select [UEFI]
    18. Save & Exit
      Save Changes and Reset
      Save configuration and reset? / Yes
    19. Use a browser to login to IPMI Web UI, choose Configuration / Fan Mode
      Fan Mode "Set to Full Speed"
      Save and Exit

    Test Method

    • Windows system was rebooted about an hour before the start of these tests
    • Samsung Magician is removed from Startup
    • SuperDoctor was used only to see if there was anything interesting going on with component temperatures during the trial-runs/setup, with a roughly 15 second refresh interval. You could do this via IPMI and browser refresh too. SuperDoctor was not used for the actual benchmark tests.
    • Onedrive is removed from Startup
    • verified CPU showing > 2.0GHz even though idle
    • three runs of each test were completed
    • at least 30 seconds of cool-down (idle system) between test runs
    • no other utilities were used for the screenshots below, the thermal measurements from Supermicro SuperDoctor 5 seen in the top-most picture above this article were from my early tests, monitoring the system while benchmarking just to learn a bit more about how the system responds.

    Results

    ATTO Disk Benchmark Results

    Simple, graphical test that shows sequential transfer rates at various file sizes. The speeds at 4K are most important for a Windows boot drive.

    In short, Samsung Specifications state:
    2,500 MB/sec reads and 1,500 MB/sec writes.
    According to ATTO, my highest numbers from 3 runs:
    2,598 MB/sec reads and 1,576 MB/sec writes.
    Nice!

    TinkerTry-Tested-Samsung_950_PRO_ATTO_on_SYS-5028D-TN4T_Run1of3
    ATTO Disk Bench trial 1 of 3, using methods described at bit.ly/950pro
    TinkerTry-Tested-Samsung_950_PRO_ATTO_on_SYS-5028D-TN4T_Run2of3
    ATTO Disk Bench trial 2 of 3, using methods described at bit.ly/950pro
    TinkerTry-Tested-Samsung_950_PRO_ATTO_on_SYS-5028D-TN4T_Run3of3
    ATTO Disk Bench trial 3 of 3, using methods described at bit.ly/950pro

    AS SSD Benchmark Results

    by Alexej Schepeljansk (Google Translate of author's description)

    The AS SSD Benchmark determines the performance of solid state drives (SSD). The tool contains six synthetic and three copy-testing. The synthetic tests determine the sequential and random read and write performance of the SSD. These tests are conducted without using the operating system caches.

    TinkerTry-Tested-Samsung_950_PRO_AS_SSD_on_SYS-5028D-TN4T_Run1of3
    AS SSD Benchmark trial 1 of 3, using methods described at bit.ly/950pro
    TinkerTry-Tested-Samsung_950_PRO_AS_SSD_on_SYS-5028D-TN4T_Run2of3
    AS SSD Benchmark trial 2 of 3, using methods described at bit.ly/950pro
    TinkerTry-Tested-Samsung_950_PRO_AS_SSD_on_SYS-5028D-TN4T_Run3of3
    AS SSD Benchmark trial 3 of 3, using methods described at bit.ly/950pro

    CrystalDiskMark Results

    Single click test reveals the read and write performance of incompressible data.
    Source.

    TinkerTry-Tested-Samsung_950_PRO_CrystalDiskMark_SSD_on_SYS-5028D-TN4T_Run1of3
    CrystalDiskMark trial 1 of 3, using methods described at bit.ly/950pro
    TinkerTry-Tested-Samsung_950_PRO_CrystalDiskMark_SSD_on_SYS-5028D-TN4T_Run2of3
    CrystalDiskMark trial 2 of 3, using methods described at bit.ly/950pro
    TinkerTry-Tested-Samsung_950_PRO_CrystalDiskMark_SSD_on_SYS-5028D-TN4T_Run3of3
    CrystalDiskMark trial 3 of 3, using methods described at bit.ly/950pro

    Best of each tools 3 runs - 950 PRO Drive C: in M.2 socket

    TinkerTry-Tested-Samsung_950_PRO_Best_of_3_runs_based_on_peak_read_speeds_on_SYS-5028D-TN4T-2015-11-07
    Drive C: - of each of the 3 runs of the 3 tools, I took the best ones, shown here, based only on the single highest read speed. It's apparent after yesterday's tests that optimizing of BIOS helped a bit, gaining somewhere under a 10% boost. For me, it's not worth running CPU at near full GHz 24x7 for that small boost.

    Best of each tools 3 runs - 950 PRO Drive D: in Lycom DT-120 M.2 to PCIe adapter

    TinkerTry-Tested-Samsung_950_PRO_Drive_D_Best_of_3_runs_based_on_peak_read_speeds_on_SYS-5028D-TN4T-2015-11-07
    Drive D: - of each of the 3 runs of the 3 tools, I took the best ones, shown here, based only on the single highest read speed. The results are nearly identical, so using the PCI adapter doesn't seem to affect speed in any substantial way. Fan speed was kept high, and there were no thermal slowdowns noted.

    See also full details about this adapter at:

    Videos

    How to setup the UEFI BIOS in Supermicro SYS-5028D-TN4T for benchmarking

    excerpt from The Samsung 950 PRO PCIe SSD Review byt Billy Tallis at AnandTech:

    even when showing symptoms of possible thermal throttling, the 512GB sustained respectable performance and in tests that were representative of interactive use it performed extremely well. Users waiting on a full range of Skylake systems to come to the market may need a PCIe to M.2 adapter in order to put the drive in a slot that provides four lanes at PCIe 3.0 speed, but with the added benefit that such adapters can be bought with heatsinks to reduce the chance of triggering thermal throttling.

    In this video you will see a continuous video with no cuts, demonstrating a speed dip in the middle of the 2nd ATTO disk speed benchmarking test, which Samsung calls Dynamic Thermal Throttling Protection. Quickly remedied by simply changing the chassis fan speed from low to medium just for benchmarking. I put mine back to the default low setting for 24x7 routine/normal use.

    For the bigger transfers, speed barely impacted at all. CPU is at 17% for small files near the beginning of the test, spiking to 26% very briefly, then below 5% for the bigger files. NVMe on PCIe 3.0 x 4 really shines here. CPU temp. rose from 38C to 46C, no other motherboard component rose more than roughly one degree. These findings are impressive.

    Third party video showing Samsung 950 Pro vs Samsung SM951 Thermal Comparison.

    See also the single M.2 to PCIe 3.0x4 adapter that I have just tested, at a cost of roughly $20 USD, that doesn't come with any heatsinks. Would I really want to peel off that big sticker, and what would that do to my warranty anyway? This particular adapter is one of the "Frequently Bought Together" accessories for buyers of the 950 PRO on Amazon. There are pricier options with heatsinks, such as this Bplus M2P4A on Amazon or Newegg.

    I'm going with a ~$20 adapter I wrote about here:

    If you wish to read a little more about my initial testing, head to my How to boot from NVMe article here.

    Brief iperf3 demo, showing 2.53 Gbps speeds moving data from one SuperServer to another, running Windows 10. This is a network test, not a disk test.

    If you're interested in trying iperf yourself, read more about it then download.

    950PRO-to-950PRO-over-10GbE-iperf_just_finished_TinkerTry
    The iperf test just finished, but the Task Manager hasn't quite noticed yet. Notice the nicely shaped histogram, a good sign of network health.
    5028-TN4T-rear-showing-10GbE-active.JPG
    You can see the CAT6e cable going directly across [X552 10GbE NIC does auto-crossover]. The other 2 network cables are for management via iKVM for remote GUI used in the video above.

    Thank you

    This article and video, and my articles about the Intel 750 Series NVMe SSD, wouldn't have been possible without the temporary loan of a second Samsung 950 PRO and a 128GB equipped second Supermicro SuperServer Bundle 2 by Trond Eirik Haavarstein aka Eric @xenappblog of xenappblog.com

    xenappblog

    You might even spot a familiar name among the presenters at xenappblog's upcoming virtual expo. Be sure to sign up early, because hundreds typically attend his events ;-)

    Disclosure

    This is not a sponsored post. I had been planning to do this testing for months now. The 2nd 950 PRO that I borrowed for this test was identical to the first, also from Amazon. My systems and my SSDs were all purchased, I just didn't have a 128GB maxed out system with a 2nd 950 PRO of my own to do these tests. Your purchase of this this same exact same SuperServer uses a Wiredzone affiliate link.

    See also at TinkerTry

    tinkertry-superserver-home-lab-storage-configuration

    See also

    TinkerTry - PCs, EVs, home tech, efficiency and more, including virtualization. My opinions here, not my employer's.

    Paul Braren, MCSE in 1998, VCP #2681 in 2005, vExpert 2014 to 2024, Veeam Vanguard Emeritus, 2015 to 2019.

    Support TinkerTry

    PayPal | Patreon | VMware Store

    After 6 successful years testing then shipping well over 1,000 Xeon D Bundles, Wiredzone had to stop selling them in mid-2021 due to cost, supply, and logistics challenges. So far, Xeon D-1700/2700 (Ice Lake D) was a minor refresh for 2023, with Xeon D-1800/2800 (Granite Rapids D) refresh looking a little better for 2024. More cores, but still just PCIe Gen4 and DDR4. Looking ahead, I'm glad it's Pat Gelsinger at Intel's helm. I'm also grateful to have had the honor of working at VMware when Pat was the CIO there. I'll leave it at that, given the whole Broadcom thing.

    Easily update VMware ESXi at TinkerTry.com/easy-update-to-latest-esxi