Copying about 50GB from a Samsung 950 PRO NVMe SSD back to itself is slower than copying to another 950 PRO over 10GbE
Yesterday, I said:
Next torture test? Abuse of that 10GbE connection to the loaner Supermicro SuperServer right next to this one...
Well, that's what I went and done. Since the moment the idea entered my brain, until the test was done, I couldn't get the unanswered question out of my head. I just had to try it out. It's not like I can find anybody else out there who had done this, which only served to increase my interest in obtaining an answer.
First, you got your Samsung 950 PRO NVMe M.2 drive, 512GB size, specs here. Next you got another identical 950 PRO (borrowed), installed in another identical Supermicro SuperServer SYS-5028D-TN4T. NVMe is able to shine here, with the PCIe 3.0 x 4 lane M.2 socket in each system, paired with a 10GbE connection between the two. No jumbo frames, just install the Intel X552 Windows 10 driver and go. No pricey 10GbE switch required, a simple 10GbE capable 1' length of CAT6A cable will do.
Finally, you got yourself a fresh build of Windows 10, and a BIOS set up like I used for benchmarking, explained here.
Now you ask yourself, how long does it take to copy about 50GB of data, consisting of 17 copies of this 4.4GB openSUSE-12.3-DVD-x86_64.iso?
The answer, it turns out, depends highly on whether you're doing a local copy, where the same Samsung 950 is doing both reads and writes. Or whether you're doing it over the 10GbE network connection. Guess which wins?
Hands-down winner is the network copy. Yep, plain old SMB network sharing in Windows 10 can really hum along, given a fast enough network pipe, and fast enough SSD. Geez, my 10GbE is a bottleneck, when it comes to my Samsung 950 PRO? I'm ok with that ;-)
Let's think for a minute together here. Yeah, just a minute, I can't wait to get the video out, and hear your feedback with comments right here below the article. So I'll just put my "draft" thoughts out there. Yesterday, reading and writing to the same drive seemed to fill a write buffer, then keep speeds down around 700MB/sec. Today, the same test over ethernet, I get back to 1200MB/sec, pretty near the 1576MB/sec of my best ATTO Disk Bench result. And very nearly identical to the 1250MB/sec I get when copying from one Samsung 950 PRO to another when inside the same system.
So it would seem more like the drop-off I noticed is just the way this drive and controller and firmware handles concurrent reads and writes that are sustained. I can't know for sure, this is just a conversation starter. I can't dwell on this, because in some sense it doesn't matter. This is simply the best damn SSD experience I've ever had, at work, or at home.
Enjoy the video, hopefully it's half as fun to watch as it was to record.
This article and video, and my articles about the Intel 750 Series NVMe SSD, wouldn't have been possible without the temporary loan of a second Samsung 950 PRO and a 128GB equipped second Supermicro SuperServer Bundle 2 by Trond Eirik Haavarstein aka Eric @xenappblog of xenappblog.com
You might even spot a familiar name among the presenters at xenappblog's upcoming virtual expo. Be sure to sign up early, because hundreds typically attend his events ;-)
This is not a sponsored post. I had been planning to do this testing for months now. The 2nd 950 PRO that I borrowed for this test was identical to the first, also from Amazon. My systems and my SSDs were all purchased, I just didn't have a 2nd identical system with a 2nd 950 PRO of my own to do these tests. If you purchase of this this same exact same SuperServer, it uses a Wiredzone affiliate link, and the Amazon's Samsung 950 PRO is also an affiliate link, helping quality articles like this be possible.
See also at TinkerTry
How does copying over 200GB from one Samsung 950 PRO SSD to another in under 3 minutes sound? (42 copies of a 4.4GB Linux distro)
Nov 17 2015
- World's fastest consumer SSD - Samsung 950 PRO M.2 NVMe benchmark results
Nov 05 2015
How to install Samsung 950 PRO M.2 SSD in a PCIe slot - tested with Supermicro 5028D-TN4T & Lycom DT-120 M.2 to PCIe adapter
Nov 05 2015
Superguide: Supermicro SuperServer 5028D-TN4T - THE Ultimate Home Virtualization Lab
Nov 01 2015
- Supermicro Superserver 5028D-TN4T UEFI BIOS 1.0.b and IPMI 02.14 released - improves boot from M.2 or NVMe support
Sep 23 2015
Problems with using SSDs for write caching and how to avoid them
Jul 23 2013 by Marc Staimer at TechTarget - SoldState Storage
- Flash Write-Back Caching Limitations
Sep 26 2013 by Howard Marks at InformationWeek Network Computing