Intel Optane SSD 900P Series is the latest in 3D XPoint NVMe hotness, should be great for home virtualization lab enthusiasts
Disclosure: The U.2 version of the 900P 280GB was the only model available anywhere, so I purchased one to test. Intel later provided me a loaner AIC version of the 900P as well, in the 280GB capacity. There were no stipulations, I continue to simply share my experiences.
Intel's October 27 2017 press release:
- Blazing-Fast Gaming with Intel’s First Client Optane SSD
Intel today announced the launch of the Intel® Optane™ SSD 900P Series, the first SSD for desktop PC and workstation users built on Intel® Optane™ technology.
The Intel® Optane™ SSD 900P Series is ideal for the most demanding storage workloads, including 3D rendering, complex simulations, fast game load times and more. Up to 22 times more endurance than other drives also gives the heaviest users peace of mind.
- Amazon? Probably best to choose only what's available on Amazon Prime.
Nov 01 Update - Note, at 4:10pm eastern, Newegg auto-notifier informed me that the 280GB PCIe version might be available for pre-order, but by the time I checked later that evening, pre-orders were no longer being accepted, for any of Newegg's 900P versions.
Nov 03 Update - Only the U.2 model with the U.2 connection became available, so I ordered it
Nov 19 Update - Inventory at Newegg looking better, Amazon doesn't seem to have Prime orders in stock, not sure how I feel about the many 3rd party sources.
This was a great week for home virtualization enthusiasts! A more affordable PCIe NVMe and U.2 drive is arriving soon, featuring the spectacular specs of the enterprisey P4800X that you saw here at TinkerTry first:
- World's First Close Look at Intel Optane SSD DC P4800X Series - PCIe NVMe arrives with 375GB of 3D XPoint!
Aug 11 2017
What's the big deal here with the 900P then? Well, it's pretty much the same as a P4800, but at a greatly reduced price of $599.00 retail at Newegg for 480GB of Optane. Yes, still a hefty premium over SATA3 SSDs, but still less than half the price of the 375GB P4800X. This great-for-enterprise-workloads Optane P4800X might as well been called the Unobtainium P4800X, given the realities of most lab budgets. The 900P promised to bring most of that goodness down to earth, just be sure you have a PCIe 3.0 x4 lanes slot available to obtain top speeds.
This sucker is begging to be driven hard, see values highlighted values. yes, those aren't ms (milliseconds), those are µs (microseconds). What does that mean?
Let's go further. How about the Endurance Ratings? How's 8.76 PBW (Petabytes Written) sound? Well, that converts to 8760 TBW (Terabytes Written). Compared with the 750 Series that has a 127 TBW (Terabytes Written) rating, that’s about a 68 times the write endurance.
What if we stick a DataCenter product in the mix, the Intel SSD DC P4600 Series 1.6TB. Not a fair comparison, but fun nonetheless:
See also the main Intel Optane SSD 900P Series landing page.
Finally, a fun look at how involved proper testing is done in the Intel Optane Solid State Drive 900P Series Evaluation Guide.
Now you know why the positive press is pouring in over this new little hotness in NVMe SSD, here's a select few:
- Intel Optane SSD 900P 480GB and 280GB NVMe HHHL SSD Review - Lots of 3D XPoint!
Oct 27 2017 by Allyn Malventano at PC Perspective
Optane’s strongest benefits are the very performance traits that do not effectively scale with additional drives added to an array. The preferred route is to just get a larger single SSD with more 3D XPoint memory installed on it, and we have that very thing today (and in two separate capacities)!
- Intel Optane 900P Low Price NVMe might Be a Perfect Fit For a Home Lab
Oct 30 2017 by Vladan Seget at ESX virtualization.
If you remember, a few months back VMware announced a support for latest Optane drives from Intel with 3D XPoint memory, and the solution was the fastest one tested, on vSAN. This has had of course a price (I saw a $1,791.00 on Amazon), which is quite pricey for a 375GB PCIe NVMe card. Not something a homelabber can afford, right?
- The Intel Optane SSD 900P 280GB Review
Oct 27 2017 by Billy Tallis at AnandTech
However this year's revolution from Intel will be very hard for the competition to match anytime soon. All of Intel's previous record-setting SSDs have relied on the drive's controller to stand out from the crowd. This time, Intel's advantage comes from the storage medium: its 3D XPoint memory technology, a new nonvolatile memory that offers much higher performance than flash memory.
VMware Compatibility Guide - I/O Device Search for Optane only shows the two P4800X drives, AIC (Add in Card) and U.2 (hot swap NVMe in a 2.5" enclosure).
I have reached out for comment on whether this 900P will ever appear on the VCG (VMware Compatibility Guide), but likely the answer will be no. Even less likely to ever be on vSAN VCG. This is really a consumer drive here, and VMware's vSAN testing is on the supported gear for the enterprise, for very good reasons.
Here's the thing. The 900P has PLP (Power Loss Protection), also called Supercapacitors. PLP was actually just discussed on the Virtually Speaking Podcast, right at this spot. Yes, scary stories just in time for Halloween! PLP capability could mean that the 900P is the best somewhat-affordable choice as a caching layer for vSAN lab tests. Completely unsupported by VMware mind you, but a lot less likely to need support in the first place. Consumer drives don't usually make sure uncommitted writes make it out of the buffer and into the non-volatile NAND, but this 3D XPoint sure does! Nobody wants to hear storage array and corruption in the same sentence, no matter how backed up you are.
Let's be real clear here. In no way am I recommending that such consumer drives be used for vSAN, especially when it comes to production workloads where having a support contract with completely supported enterprise rated components is essential.
There is no telling whether using the 900P for vSAN is a good idea for a home lab, with hands-on testing certainly needed. C'mon now, the press release even says "Blazing-Fast Gaming."
So, as a thought experiment, let's just say you were to format a 900P as VMFS 6 using VMware vSphere / ESXi 6.5 Update 1. You then use this datastore for nested ESXi 6.5U1 instances, creating a 4 or even 6 node vSAN. Alternatively, maybe you get even more "creative" and try NVMe pass through of each NVMe device you have, which isn't hard at all anymore, see my recent How to configure VMware ESXi 6.5.x for VMDirectPath I/O pass-through of any NVMe SSD, such as Windows Server 2016 installed directly on an Intel Optane P4800X booted in an EFI VM. Got your attention yet? Maybe you've got yourself one very fast vSAN caching tier in your lab tests for your other more affordable SSDs. Maybe...
So where's that leave TinkerTry? Well, for now, I can't know how well this works or performs until I can get my hands on one of these. Please know that I sure am trying to get one, as is nearly every other blogger out there.
Decent capacities in M.2 sizes are a long way off
Even more versatile would be a 900P in the M.2 form factor, like a much better version of the original M.2 Optane module, see:
- Intel Optane M.2 16GB/32GB consumer NVMe SSDs for Windows caching are not that fast as VMware VMFS or NTFS, save up for bigger PCIe versions
Jun 11 2017
What I'm getting at here is that you can stuff 4 M.2 drives into a single PCIe 3.0 x 16 slot, and pass each NVMe device through to a VM independently. That would be a much more practical way to handle cache testing scenarios such as vSAN POCs.
As for cosmetics and market differentiation? Nice job on the distinctive backplate Intel, good move. Smart looking, and flow-through practicality, demonstrating your apparent mastery of AIC cooling that was so evident in my FLIR thermal imaging tests of the similarly designed P4800X that easily handled my no air flow tests. I strongly suspect the 900P will perform very similarly, can't wait to get my hands on one to try it.
Sorry, these are just renders, not quite like taking close-ups of a production sample, such as I did with the loaner P4800X here. Still, you'll get a pretty good idea of how the stickered GA product will look. After clicking on image, don't forget to go full screen with the icon at top right. Enjoy!
In the original article above, I neglected to mention that you should look closely at that photo gallery, specifically, the last photo. See the caption? It's the Intel Optane SSD 900P Series U.2 Box Contents M.2 Cable that has my attention for folks that own the SYS-5028D-TN4T system. While host swap U.2 drives, which look like extra thick 2.5" laptop drives, wouldn't normally do much good in this SATA3 system. But remember, there's 2 bays in there for 2.5" drives that get screwed into a bracket, and both have extra space for taller drives. As for the 1U height systems like the SYS-E300-8D and SYS-E200-8D, I strongly suspect there just isn't near enough clearance in there, where the 7mm SSDs I tried on my loaner 1U systems barely fit, seen here and here.
Special thanks to a TinkerTry reader who wrote in on November 1st, bringing this oversight to my attention.
Internally mounted U.2 thermals
What about airflow? Admittedly, the 4 hot-swap SATA drive bays get the airflow, but there isn't much air across the two internal drive bays. That said, even with no airflow, the similar P4800X did fine, even with quite a bit of stress, as already mentioned above, and seen here. So I suspect the aluminum heat sinks integrated into the U.2 900P design will be sufficient, with Intel learning from years of making U.2 drives for the enterprise, see also my visit to Intel at VMworld 2015 featured here. That said, let's not forget that U.2 dives are typically mounted where there's active airflow in a noisy 2U server. Such airflow is not easily possible with this particular system.
Intel U.2 to M.2 Adapter Cable (18" length confirmed by Intel)
Installation should be this simple:
- remove either of the 2.5" drive brackets by removing the 2 chassis screws.
- secure the U.2 900P to the bracket using 4 2.5" screws that you'll apparently need to provide.
- reinstall the bracket into the SuperServer again with the same 2 chassis screws you removed earlier.
- running the included U.2 to M.2 cable down to the motherboard's M.2 slot that runs at full PCIe 3.0 x4 lane speeds.
That's it! Now you've got an additional 900P U.2 drive added to your mini-tower system, leaving your precious PCIe slot open for a good cause, such as:
- another P900, the HHHL PCIe form-factor version.
- a GPU.
- a 4 way full speed M.2 PCIe card that leverages this system's 4x4x4x4 bifurcation feature that's easily enabled in the BIOS.
How could I fit two 900P drives into my SYS-5028D-TN4T mini-tower?
- one U.2 900P in the side internal 2.5" bay, connected to the motherboard's one M.2 slot using the included cable
- one PCIe 900P in the one PCIe slot
How could I fit three 900P drives into my SYS-5028D-TN4T mini-tower?
You'd need the extra expense of an SFF NVMe adapter card, such as the Supermicro OC-SLG3-2E4 (see User's Guide that's available at Wiredzone, and two U.2 P900 SSDs that include the SFF 8643 cable, used to connect to the OC-SLG3-2E4 in the PCIe slot, and a third U.2 P900 with the included M.2 cabled, used to connect to the motherboard's M.2 slot.
Physical Installation Instructions for Intel® Optane™ SSD 900P Series
Oct 27 2017 by Intel Support, excerpt:
Installation steps for the Intel® Optane™ SSD 900P Series 2.5-inch U.2 drive
- Mount the SSD in the chassis using screws to hold the SSD securely
- Connect the cable to the U.2 SSD connector (SFF 8639)
- Depending on the cable that came in your box at purchase, either connect the:
M.2 card in the PCIe supported M.2 motherboard connector
SFF 8643 to your PCIe supported motherboard connector
- Connect SATA power to the connector on the cable
- What’s in the Box for Intel® Optane™ SSD 900P Series
Oct 27 2017 by Intel, excerpt:
Article ID: 000025739
The Intel® Optane™ SSD 900P Series is available in three different retail kits.
Note - This type of solid-state drive does not receive data from SATA / AHCI connectors.
Intel® Optane™ SSD 900P Series Retail Box for PCIe* U.2 2.5” with SFF 8643 Cable
Intel® Optane™ SSD 900P Series (U.2 2.5” form factor)
U.2 to SFF 8643 Adapter Cable
Intel® Optane™ SSD 900P Series Retail Box for PCIe* U.2 2.5” with M.2 Cable
Intel® Optane™ SSD 900P Series (U.2 2.5” form factor)
U.2 to M.2 Adapter Cable
Intel® Optane™ SSD 900P Series Retail Box for PCIe* Add-In Card
Intel® Optane™ SSD 900P Series (add-in card form factor)
Half Height Bracket
It's here! I now have my "Intel® Optane™ SSD 900P Series Retail Box for PCIe* U.2 2.5” with SFF 8643 Cable." I don't yet have the U.2 Adapter, such as the OC-SLG3-2E4, so I can't test it yet, but I did testing mounting it in the SYS-5028D-TN4T chassis, and it went well. Unboxing video, recorded in 4K, is being produced now, hopefully ready to share by Nov 4th.
Added 900P unboxing video above.
It's here! My Supermicro AOC-SLG3-2E4 - Supermicro NVMe PCIe Host Bus Adapter - 2 internal NVMe ports 6.4GB/s Gen-3 and were both home tonight, and got a chance to unbox and install the PCIe card, on camera of course. Kind of a cool moment when I quickly realize that the same exact BIOS tweaks that were needed for my earlier P4800X tests were needed for the 900P as well.
Of course, I learned the hard way again that I'd get a BIOS POST hang that simply says
A9 as soon as the AOC-SLG3-2E4 is installed in the PCIe slot with a 900P attached, just like I saw with the very similar P4800X.
Remove the AOC-SLG3-2E4, and the boot hang went away. Not sure what would have happened had I simply removed the U.2 connection cable from the drive.
I used my own exact BIOS settings fix for this hang, right at this spot in the original P4800X article. Short version is be sure that:
SLOT7 PCI-E 3.0 X16 Bifurcationis set to default
SLOT7 PCI-E 3.0 X16 OPROMis changed from the default of
and now I was good to go, boots up and works just fine. I have a lot of basic changes to attend to next, including:
- upping the number of Paravirtual SCSI adapters and vCPUs, as outlined by Intel here
- revisiting my BIOS settings, as Intel describes here
- revisiting my NVMe driver, note that there are no ESXi downloads for the 900P, only for the similar P4800X, this isn't at all surprising, given the consumer focus of the 900P.
- check on the 900P firmware, by booting over to Windows and trying out Intel's utility
- check out performance on Windows 10 + NTFS using the 18.104.22.1684 NVMe driver, from:
NVMe* Drivers for Intel Optane SSD 900P Series.
- view the Supermicro AOC-SLG3-2E4 PCIe card and the 900P U.2 drive itself with my FLIR thermal imaging camera while under extended load
That's just for starters. Once the PCIe version of the 900P arrives, and/or the U.2 to M.2 cable, I'll have some more tests up my sleeve to share with you, stay tuned!
New video now available above, "Installing the Supermicro AOC-SLG3-2E4 U.2 PCIe adapter with the Intel Optane 900P [4K]".
Video production underway of initial testing under ESXi 6.5U1, to be published right here soon. Done! Added to the video gallery above.
Intel has provided me with one of each of the following:
• Qty 1: Intel U.2 to M.2 Adapter Cable
• Qty 1: Intel Optane SSD 900P 280GB (U.2)
which enabled me to proceed with testing that both 900P drives work properly in the SYS-5028D-TN4T Supermicro SuperServer system, concurrently.
First, I removed the Supermicro AOC-SLG3-2E4 PCIe adapter card. I then wired up my 900P U.2 drive to the motherboard's M.2 slot using the Intel U.2 to M.2 cable. Finally, I was able to use the PCIe slot for the loaner 900P AIC (Add-In Card), as pictured below.
Problems were unearthed that slowed my progress a bit:
- BIOS tweak needed to avoid BIOS hang at
A9error, setting EFI for both the M.2 and PCIe slots, recommended BIOS settings article updated
- if both M.2 and PCIe are occupied by Optane, I can't get back into the BIOS without removing the PCIe card (upcoming video I'll publish shows me figuring this out)
- coincidentally timed bump in the road explained here.
Testing continues to go fairly well overall, more videos and additions to this article to come.
Next up will be booting over to Windows to check in on firmware levels, and getting my video published of the 900P AIC unboxing and install, along with my VMFS 6 formatting and initial testing of the new 900P AIC under VMware ESXi 6.5U1.
As expected, the stated absolute throughput values of 2500Mbps reads and 2000 MBps writes the can be exceeded slightly by other M.2 SSDs like the Samsung 960 PRO, and my initial testing slightly exceeding these values when using simple ATTO Disk Benchmark tests in a Windows 10 VM that's running on ESXi 6.5U1, with the Optane 900P formatted VMFS 6.
There's more to the story here. Don't forget the latency, which I haven't really tested at all yet. There's also the lack of any need for typical SSD housekeeping like garbage collection, so performance is consistent, all the time. Don't forget the ability to fill this sucker up to 99% without worrying about performance degradation, not something that can be said about older SSDs where performance tanked once you accidentally filled the SSD accidentally, which isn't exactly hard to do quite by accident in these days of thin provisioned VMs.
From a benchmarking standpoint, Optane is a dream to test, you don't have to
worry about pre-conditioning...the performance is always stable, it doesn't matter what you do to it.
I compared the 4 drive Optane array to a 4 drive 960 PRO 512 gig drive array, so 2 terabytes worth of Samsung's fastest SSD that they make...how high does the throughput go?...Looking at random access, now you would think that the 960 PROs would be pretty quick, but the Optane array, even though it's 1/20th of the capacity...just walks over it in random access...how high are the IOPs when you're at low queue depths, because that's, you know, where you'll actually be.
Power Loss Protection.
According to the Intel spec sheets, the Intel® Optane™ SSD 900P Series doesn't offer what Intel calls Enhanced Power Loss Data Protection.
According to the Intel spec sheets, the Intel® Optane™ SSD DC P4800X Series does offer what Intel calls Enhanced Power Loss Data Protection.
Is this a concern, especially if you're considering using the 900P in a completely unsupported use-case, such as in the vSAN caching layer? This warrants further investigation.
Absolutely none of Intel's Optane parts (including the $3000+ enterprise drives) include large capacitors for power loss prevention.
BUT: EVERY Optane product on the market now including the consumer drive Phoronix tested and even the cheap 16GB cache drives provides power loss protection.
That's because none of these drives use a RAM cache at all and the data are stored fast enough that PLP is an inherent feature of the drive that doesn't need extra circuitry.
With the impending holiday that I quite enjoy, with family and friends all around, I wanted to get as much as I could documented for you, given I'll be falling behind on the follow-along-with-me video production:
I updated my ESXi 6.5U1 to Intel's 22.214.171.124 version of NVMe driver, from:
Download VMware ESXi 6.5 intel-nvme 126.96.36.199 NVMe Driver for Intel(R) DC Series NVM Express Solid-State Drives
I dual booted over to Windows 10, installed the latest toolbox from
Download Intel® Solid State Drive Toolbox
only to discover that both of my 280GB 900P Optane SSDs (U.2 and AIC) are already at the latest firmware level E2010325.
I am finding performance using ATTO Disk Benchmark to be as expected
the BIOS A9 hangs continue to be a problem only when I want to get in to the BIOS to change something, work-around is to pull the Optane drives out, make my changes, then put them back in
- I do not yet have an answer from Intel to help get clarity on Enhanced Power Loss Data Protection on the 900P versus the P4800X Optane parts
Today, I downloaded Intel SSD Data Center Tool 3.0.4, which is the latest version for VMware. Then I followed the installation instructions in the filename
Intel_SSD_DCT_Install_Guide_3_x.pdf included in the downloaded
DataCenterTool_3.0.4_ESXi.zip file, and unfortunately, it doesn't seem to pick up that two 900P drives are installed in my system with the 188.8.131.52 NVMe Driver. This thwarted my efforts to see the NVMe firmware levels, and my usual procedure here doesn't work either:
I also tweeted some ATTO Disk Benchmark early tests, but Twitter blurs screenshots pretty badly. Here's the full resolution pic.
Tomorrow, I'll be eating turkey. See you right back here on Friday!
All P4800x articles.
All NVMe articles.
- Intel announcements include details about benchmarking Optane P4800X in VMware ESXi, new 750GB capacity, and 3D XPoint fab expansion
Nov 13 2017
- Intel's new Ruler form factor allows up to 1PB of NVMe in 1U, along with a new 2.5" 3.8TB SATA SSD and Dual Port SSDs for Data Centers
Aug 15 2017
- M.2 expansion for your NVMe SSDs - EZDIY adds, Angelbird Wings adds and cools, Amfeltec Squid adds, cools, and quadruples
Dec 06 2016
- Samsung SZ985 Z-NAND SSD - Upcoming Competition for Intel's P4800X?
Nov 20 2017 by Allyn Malventano at PC Perspective
While the SZ985 runs at ~1/3rd the latency of Samsung's own NAND SSDs, it has roughly double the latency of the P4800X. For the moment that is actually not as bad as it seems as it takes a fair amount of platform optimization to see the full performance benefits of optane, and operating slightly higher on the latency spectrum helps negate the negative impacts of incorrectly optimized platforms.
Image Source: ExtremeTech
- Intel Optane SSD 900P Review (480GB) – Understanding Disruptive Technology
Nov 12 2017 by Les Tokar at The SSD Review
The Intel Optane SSD 900P, and specifically Intel 3D XPoint memory changes this, in that, this memory can add, delete or replace information at the most basic level without requiring the large movement of data to accommodate such. This results in significant performance and endurance increase and is very much considered disruptive as it has the potential to change the storage industry as we see it today.
- The Intel Optane SSD 900P 280GB Review
Oct 27 2017 by Billy Tallis at AnandTech
Who is the Optane SSD 900P for?
With a price per GB a little over twice that of the the fastest flash-based consumer SSDs, the Optane SSD 900P is an exclusive high-end product. For most desktop usage, drives like the 960 PRO are already fast enough to make storage no longer a severe bottleneck. The most noticeable delays due to storage performance on a 960 PRO are when moving around large files, and the Optane SSD doesn't offer any significant improvement to sequential transfer speeds. Random writes can be a challenge for flash-based SSDs, but volatile write caches and SLC caches allow them to handle short bursts with very high performance.
- New Optane disks appear on web shops' lists
280GB and 480GB beasts on PCIE 4.0, apparently
Oct 25 2017 by Team Register at The Register
SSDPE21D280GASM, a 2.5-inch drive with 280GB capacity and U.2 or M.2 connections
SSDPE21D280GASX, as above, but with PCIE 4.0 connector
SSDPED1D280GASX, a 280GB add-in card for PCIE 4.0
SSDPED1D480GASX, a 480GB add-in card, again for PCIE 4.0
Here's a really thorough look at the subtleties of M.2 NVMe benchmark testing, and an overview of why NVMe is better than SATA in very measurable ways.
- Benchmarking Samsung NVMe SSD 960 EVO M.2
Mar 24 2017 by Taras Shved at StarWind
Everyone knows that, currently, the SSDs are one of the best storage devices that allow you to upgrade your architecture and significantly accelerate the performance of the computer. SSD accelerates the loading speed of your PC, applications opening and files searching speed, and generally increases the performance of your system. Despite the fact that solid-state drives are more expensive than standard hard drives, the performance improvement can hardly be overlooked.
- How 3D XPoint Phase-Change Memory Works
Jun 02 2017 by Allyn Malventano at PC Perspective
While we were initially told at the XPoint announcement event Q&A that the technology was not phase change based, there is overwhelming evidence to the contrary, and it is likely that Intel did not want to let the cat out of the bag too early.
- Supermicro AOC-SLG3-2E4R and Supermicro AOC-SLG3-2E4 Differences
May 30 2015 by Patrick Kennedy at STH
The Supermicro AOC-SLG3-2E4R – $149 (Amazon or Ebay but I got from ebay) is a lower cost part with no PLX chip onboard. It also therefore consumes less power but has a much tighter compatibility matrix.