M.2 expansion for your NVMe SSDs - EZDIY adds, Angelbird Wings adds and cools, Amfeltec Squid adds, cools, and quadruples
Such adapter cards pass signals from your M.2 NVMe device right through to your motherboards PCIe slot, with no speed degradation. You can see these simple wire traces in the close-look video that's featured below, so there's no special drivers needed. Modern PCs and servers simply "see" the NVMe drive, whether it's in a motherboard socket, or installed in any of these adapters. A nice perk, if your system can boot from NVMe, it can likely boot just fine from any of the M.2 NVMe devices mounted in any of these cards. Note, these items were all purchased, and some of the shopping links below are affiliate links, see detailed disclaimer and disclosure below every article.
- PCI Express M.2 SSD NGFF PCIe Card to PCIe 3.0 x4 Adapter (Support M.2 PCIe 22110 ,2280, 2260, 2242
- Available at Amazon.
Simple and affordable (around $22 USD) way to add M.2 NVMe to your PCIe equipped system, best speeds if your PCIe is 3.0 x 4 lanes. Because it's longer than the Lycom DT-120 I wrote about a year ago, it can handle not just the common 2280, but also those supercapacitor-equipped enterprise 22110 length drives. You know, the ones you might need for those future vSAN tests you're pondering.
In the video, you may notice I've covered up the too-bright LED on this card, since I sometimes run the system with the lid off at demonstrations and VMUG events. The fix was simple, LightDims Original Strength LED covers.
Simple, looks good, and affordable, and it includes full height and half height backplates.
The Not So Good
That blue LED is way too bright, but that's easily fixed.
Note that the new Samsung 960 PRO/EVO (that one TinkerTry visitor already has!) is said to run cooler under load than the 950 PRO partly because of the new Polaris controller, and partly because of the heat-spreader copper layer under the sticker, which better spreads the heat out. The idea here is to prevent thermal throttling whenever possible, and even with throttled, it's still way faster than any 2.5" SATA3 SSD. Simply having your M.2 storage up off the motherboard can mean that this affordable card lets you run cooler longer.
- Available at Amazon and Newegg.
- Product Page and Specs.
- User Manual already references Samsung 960 PRO/EVO compatibility.
Haven't fully tested this yet, as the need for supplemental cooling for my beloved and (formerly) world's-fastest consumer SSD is only rarely, such as when I want to run extended duration benchmarks without cranking the chassis fan to max. I don't benchmark to show off, I benchmark to determine whether I'm getting roughly the expected performance of my investment. This does require application of a special adhesive strip to your NVMe, thermally connecting it to the large aluminum passive cooling surface.
But people love this thing, and the construction and look sure seems of high quality, have a look at the video below.
Looks good, great construction quality, haven't fully tested.
The Not So Good
No half-height backplate included.
- SKU-086-34 SQUID PCIe Gen 3 Carrier Board for 4 M.2 SSD modules (M.2 key M)
(full or half-height bracket; x16 or x8 PCI Express Adapter)
- Available at Amfeltec site which has you contact Amfeltec sales via email to get the latest pricing
- Hardware Manual
- Active cooling fan moves provides laminar airflow across all 4 slots, at the cost of ~16dB noise increase, but that's at an 8" distance with no PC cover in place.
Now this is pretty spectacular. A way to get 4 M.2 NVMe drives into one PCIe 3.0 x16 slot. Do the math. Samsung 960 PRO/EVO likes PCIe 3.0 x 4, and we have x 16 in those longer slots, such as those found in the Xeon D. No, it's not RAID, or a RAID controller, it's just a way for me personally to use my 128GB Samsung SM951 alongside my Samsung 950 PRO 512GB, and soon my Samsung 960 EVO 1TB.
They also make a more affordable 2 M.2 slot and Gen 2 versions, see the whole product line, Squid PCI Express Carrier Boards™ for M.2 SSD modules. , but if you want to be able to use several more M.2 devices over time, this device should serve you well for years to come. Note, if your motherboard supports Intel RST/RSTe, it's often for SATA only, so don't assume that means you can RAID M.2 NVMe. Check your motherboard or system documentation.
- initial tests, seen near the end of the video below, do indicate that the speeds I'm getting from my M.2 drives are just as fast as when they were mounted right on the motherboard, this is good
- no drivers needed, it just works
- the cooling fan does seem like it might be effective, but I haven't really tested it yet
The Not As Good
- unfortunately, you can't get pricing easily online, but hopefully this Canadian company will work on easier availability worldwide, see also their current list of distributors
- it is pricey (hundreds), you should , to ask for a quote shipped to your address
- fan noise can be a concern for some use cases, but it's easily detached (demonstrated in the video) for users who have already have adequate airflow
- no driver needed for any of these card, they pass signals through that seamlessly, so picky OSs like VMware "see" each mounted M.2 NVMe device natively
- did my brief testing under ESXi 6.5 (see video), but NVMe has been supported since 5.5, works fine
- there seems to be little reason to worry VMware HCL listing such adapters as compatible, because they're invisible to the ESXi hypervisor, nice!
I have found a US based reseller of Amfeltec products:
- PCI Express Gen 3 Carrier Board for 4 M.2 SSD Modules (SKU-086-34)
Without asking, Amfeltec in Canada decided to send me 2 more of their latest product offerings. Because there is no affiliate program (no commissions) and prohibitive return shipping fees, I decided to keep the 2 items, and disclose how they were obtained.
Recording the unboxing of the items while recording in 4K gives you a very detailed look, admittedly uncut and unedited. Since the recorded video is a massive 5GB in all, editing this for small gaffes and gaps just wasn't feasible in any reasonable amount of time.
- PCI Express Gen 3 Carrier Board for 4 M.2 SSD modules - original review above
- PCI Express Gen 3 Carrier Board for 2 M.2 SSD modules - new unboxing video below
- 1U PCI Express Gen 3 Carrier Board for M.2 SSD module - new unboxing video below
This last model features a metal retainer clip that could thankfully easily be removed (off camera), since for Xeon D systems like my Supermicro SuperServer SYS-5028D-TN4T, this could touch some nearby motherboard jumpers.
All of Amfeltec's PCI Express Gen 3 Carrier Boards can be found at:
- AOC-SLG3-2M2 Supermicro Add-On Card for up two M.2 NVMe SSDs - Internal, PCI-E 3.0 x8, Low-Profile, available at Wiredzone.
TinkerTry Udo Keller commented:
...as soon as you change the BIOS setting for PCIe bifurcation from x16 to x4x4x4x4, both slots are recognised.
This is a big development. I have now succesfully tested the Supermicro SuperServer SYS-5028D-TN4T with the HYPER M.2 X16 CARD! There's a catch: I only got this working by using a PCIe ribbon cable slot extender to run the Hyper card externally. It's a full height PCIe card, so there's no other way.
But because this Xeon D system doesn't need a card with a PLX on it, it works fine with this card by just turning on x4x4x4x4 Bifurcation in the BIOS. That's right, no RAID, just JBOD style passthrough of 4 M.2 x4 devices at full speed, seen as native NVMe devices under VMware ESXi 6.5U1, for example. Should work fine with any OS though, since it's just passing M.2 devices right through.
With Intel® Virtual RAID on CPU (VROC), unused CPU PCle® lanes can be assigned to storage, allowing you to create a bootable RAID array with multiple M.2 SSDs. Unlike motherboard PCH-based RAID, VROC isn't confined by the 32 Gbps DMI bottleneck, so multiple drives can be teamed to provide incredible throughput.
Thermaltake TT Gaming PCI-E x16 3.0 Black Extender Riser Cable 200mm AC-053-CN1OTN-C1
I can't seem to find the exact model number of the ribbon cable I used from 2 years ago, as featured in the video above. But this Thermaltake readily-available ribbon cable seems appears to be of the same specifications, and is likely to work just fine with the ASUS HYPER M.2 adapter, or whatever other PCIe device you are interested in testing.
Full article on this project is planned, with video. For now, here's a picture of the card situated inside my SYS-5028D-TN4T, and I was able to put the cover back on without any issues. I had to remove the aluminum cover/heatsink to get it to fit though. You'll notice the power lead for the fan is detached, since it won't do any good without the aluminum cover. I also used some 3M 03614 Scotch-Mount 1/2" x 15' Molding Tape to fasten it without any possibility of shorts, since the Hyper card's entire back surface is painted. The red foam tape cover that's visibile was used to protect the capacitors on the front from any inadvertent bumps, when installing/removing the lid. If you look closely, you'll see my 960 PRO still on the motherboard's M.2 slot. That's right, I now have FIVE M.2 NVMe devices, all running at full PCIe 3.0 x 4 lanes of speed, at a much lower price point. This is great, and things I'll keep in mind as I begin my quest for future Xeon D-1500 and Xeon D-2100 chassis designs, and beyond.
There's a new tiny PCIe to M.2 adapter now available, as David Chung shared here. Only 4 of those 16 lanes of connections are used, the first 8 pins near the M.2 connector actually. The rest of the pins are likely there to provide some extra friction and stability, since this isn't a card that can be screwed down and fastened in place. Given the way this adapter is notched, it should fit in most modern motherboards, even if those extra pins aren't plugged into anything.
Given it's also just another pass through device, I don't see how this would affect vSphere/vSAN compatibility or support.
Great twitter conversation with William Lam about the new:
- SYBA SI-PEX40142 5 Slot M.2 B-key SATA Base PCI-e 3.0 x2 Bandwidth Controller Card Require x16 Slot
- I/O CREST 5 Slot M.2 B-Key SATA Base Pci-E 3.0 X2 Bandwidth Non-Raid Controller Card Require X16 Slot
These appear to be 5 x M.2 card clones that use PCIe 3.0 x2.
Boot-from-NVMe has some special NVMe-aware BIOS/UEFI considerations that I wrote about back in November of 2015 here.
Nov 05 2015
- How to boot Windows 10 from NVMe based PCIe storage, featuring Samsung 950 PRO M.2 SSD in a Supermicro SYS-5028D-TN4T
Nov 05 2015