It's gotten very easy to install this VIB, with the easy method right up top, featuring the latest well tested Jul 19 2017 4.5.3 driver, with a full description of how I developed and tested an easy 4.5.3 install procedure below. For the original fully supported but more time consuming method for production environments, jump below for details.
Fast, easy, and effective 10GbE VIB installation method (unsupported)
Same command works for upgrades too!
This super fast download-and-install one liner takes under a minute to apply, partly because the VIB is only 187KB! This particular screenshot actually shows how it's the same command for an upgrade. Your 'Removed:' section may differ. This is an unaltered copy of the Intel VIB net-ixgbe_4.5.3-1OEM.600.0.0.2494585.vib
Warning/Disclaimer:
I've temporarily hosted this shortcut method at tinkertry.com, for convenience. There is no warranty implied, and there is no support given for anybody who finds they encounters issues. If you are at all concerned about 3rd party VIB source, your connection is https encrypted, and the MD5SUM to help detect data corruption is: 3c69bc962e0571a9e265b13f405d0451
You can also do an FC on this hosted file versus the VMware file. Of course, those concerned can simply use the fully VMware supported method below, which requires a few more steps, including authentication from "My VMware" just to get the VIB.
backup your ESXi, this will fully protect you from troubleshooting, especially if you have no VMware support, with peace-of-mind, since you'll have the option to roll-back your ESXi should your VIB install not go well
shutdown all your VMs, or go into maintenance mode first
login to your ESXi SSH shell, and paste in the following command:
Now you can reboot through the web GUI, or via SSH with this command:
reboot
That's it! You can see some sample pictures of how things should look in ESXi after you've made the upgrade and now have this new driver loaded.
Easy 10GbE install method used, note the use of the no-sig-check option, that's why the long method detailed below is recommended for production workloads, since it starts with logging into My VMware and downloading the zip directly from VMware.vSphere Client view of installed Intel X552/X557 10GbE.vSphere Web Client view of installed Intel X552/X557 10GbE.
Make sure you're ready to reboot soon, with VMs already shut down, or get into maintenance mode. That's because the end of this intruction set has you rebooting!
Open an SSH session (PuTTY) to your ESXi 6.0 (or ESXi 6.0 Update 1) server
(if you forgot to enable SSH, here's how)
Put your system into maintenance mode, or, shut down all your VMs now and ensure you've set your ESXi host to automatically gracefully shutdown all VMs upon host reboot.
Unzip the downloaded ixgbe-4.5.3-2494585-6925533.zip file, the VIB you want is named net-ixgbe_4.5.3-1OEM.600.0.0.2494585.vib and is found at the top level of the unzipped folder.
Upload net-ixgbe_4.5.3-1OEM.600.0.0.2494585.vib to one of your datastores using vSphere Web Client Datastore browser.
Open an SSH session (PuTTY) to your ESXi 6.x host/server, my example below is at ESXi 6.0 Update 1a
(if you forgot to enable SSH, here's how).
Paste the following commands into your PuTTY session, one line at a time.
(you will need to specify the full path to where you put your VIB, it will differ from what is seen as Intel_750_NVMe_400GB in my example screenshots, but using tab autocomplete, it's pretty easy to construct this command in your environment, on the fly).
This is my successful install screenshot on ESXi 6.0U2, click twice to zoom in fully. It should look very similar for ESXi 6.5.x, and the newer 4.5.3 on ESXi 6.xNow that's how it's supposed to look!
Kept for archival reasons, you can skip down to the many article updates below.
It's here, at last! The long wait since June for this 10GbE VIB ended when JBBERLIN77777 replied to my reports of a VMware X552/X557 driver download site with invalid links. He announced his finding of the download link, over in the huge VMware Communities thread:
Were's still not sure how he found that URL, but we're all glad it works! Can you tell I'm happy, especially now that I'll have 2 SuperServers directly connected over 10GbE using Cat6a cabling, and each will have a Samsung 950 PRO M.2 NVMe 512GB drive, on November 3rd. In other words, a way to actually push the speeds of that wire. Woo hoo, what timing, this is a great week! No, I don't own a 10GbE switch, and this second SuperServer 5028D-TN4T system is just a loaner from a very gracious Wiredzone customer. I only have it for another few weeks.
March 17 2017 Update - new VIB 4.5.1 is released. Here's the Release_Notes_ixgbe-4.5.1.txt and the actual driver download page for the new VIB, requires My VMware login (sign-up is free):
File size: 851.5 KB
File type: zip
Download Now Name: ixgbe-4.5.1-2494585-5197765.zip
Release Date: 2017-03-17
Build Number: 5197765
VMware ESXi 6.0 ixgbe 4.5.1 NIC Driver for Intel Ethernet Controllers 82599, x540, x550, and x552
The ESXi 6.0 driver package, also compatible with ESXi 6.5, includes version 4.5.1 of the Intel ixgbe driver. It supports the products based on the Intel 82599, x540, x550, and x552 10 Gigabit Ethernet Controllers. For detailed information and ESX hardware compatibility, please check the I/O Hardware Compatibility Guide Web application.
This VIB works fine with 6.0, as we are reassured that's OK at the Download VMware vSphere site:
> Important Note for Certified I/O drivers > ESXi 6.0 supports I/O drivers built and certified on ESXi 5.5. The VMware Compatibility Guide lists both ESXi 5.5-based and ESXi 6.0-based drivers as supported with ESXi 6.0. See Knowledge Base article 2111492.
I do not know how that URL was found, and there seems to be no 6.0 version of that download site, but if one does show up, it's pretty likely to turn out to be the same exact VIB file inside.
Nov 03 2015 9:00pm EST Update - Yep, they're all the same VIB. VMware has now fixed the URLs on the existing download site, and they use the same link for all the 5.5 and 6.0 versions.
Hi Paul, can the 10 GB NICS on the Supermicro SuperServer also operate at 1 GB speed under ESXi 6.0? Thank you.
Yep, no problem at all, auto speed negotiation on by default. So in a sense you have 4 1GbE ports, and one 1GbE dedicated to management.
All 4 ports handle auto-crossover as well, should you use direct attachment to another system, without a network switch in the middle. Just use normal CAT5e for the 1GbE ports, and normal CAT6a or CAT7 cables for those 10GbE ports, no cross-over cabling required. Easy under Windows, but issues with getting this to work under VMware.
VMware ESXi 6.5 came out on Nov 15 2016, recorded a video of the simple install method using the unchanged one-liner install, and added it to the article above. Firewall commands instructions are optional for most configurations, so I've left them only in the longer instruction set above.
Also worth noting that there are issues with trying to force data rates with this driver, so direct connect (auto-crossover RJ45 ports allow any normal CAT6a or CAT7 cable) gets tricky, discussed here.
Additional configuration options supported by the driver: None
Bug fixes:
b1208972122: Fix PSOD when CNA = 8 and VMDQ < 4
b1208935559: Network device not showing "Cable Type" information for DA connection on X520-DA2
b1208939075: Allow to force link speed using esxcli or vSphere Web Client
If you've gone to ESXi 6.5.0d Build 5310538 and found your 10GbE NICs have disappeared, don't worry, just re-install them. It's all explained right here.
4.5.2 is out, download and release notes here. Read those release notes, it would seem not everybody will want or need this particular update, which seems geared for FCoE customers, explained here in KB 2149835 as well.
This same driver works fine with the Intel X540 PCIe (AIC) cards as well! I tested 4.5.2 on my old Core i7 vZilla build, and there were no issues, 10000 speeds acheived.s
Testing of version 4.5.3 has begun contents of above article will be updated once I'm comfortable that this driver works well on Intel Xeon D-1541 and Xeon D-1567.
While this new Intel driver shows as October 23 2017, the download just became available today. Here's how I located the driver.
Let's choose to expand just the Relese - ESXi 6.5 U1, Device Driver(s) ixgbe version 4.5.3 section, where I find the actual download driver link in the section labeled Footnotes:
Now I have the file I want: /ixgbe-4.5.3-2494585-6925533/net-ixgbe_4.5.3-1OEM.600.0.0.2494585.vib
all ready for my testing, so I can soon call it TinkerTry'd. You may want to push a copy to your intranet host that is accessible via a URL. I've put a copy on my external host, as a convenience, use-at-your-own-risk for home labs. You can verify the checksum if you'd like, 3 types of checksums provided above.
After backing up the ESXi host, and after shutting down all VMs (as explained in detail in the full article above), I'm ready for the simple install two-liner, pasted in to my ssh session to my intended ESXi server:
this installs the driver the easy way, no My VMware login neeed.
After the reboot, I issued this command
esxcli network nic list
to determine which nice name to use from column one, then:
esxcli software vib list | grep gb
to see which driver is loaded for that nic, which should include the 4.5.3-1OEM.600.0.0.2494585
portion of the bundle called INT_bootbank_net-ixgbe_4.5.3-1OEM.600.0.0.2494585.vib
Now I'm done with the install, testing speeds and function over time can now commence, using my home's 10GbE network.
I've received some reports of some 12 core Xeon D systems having network outages as often as twice per day, which I've occasionally experienced in my home lab as well. The link layer LEDs go off on the attached 10G switch, regardless of the switch brand, cabling, or VIB used. I plan to be detailing this story in a separate post, stay tuned. Meanwhile, there is a workaround that really does works, albeit only temporarily:
gracefully shut down your 12 core (or greater) ESXi system
unplug the power cord for at least 15 second
plug the power cord back in
power up and boot ESXi
A proper fix is available by contacting Supermicro directly, but I'm coordinating with Supermicro on this to hopefully see a fix incorporated right into future BIOS releases.
After reading through the 4.5.3 release notes, it would seem some folks may want to try the older 4.5.2 or 4.5.1 versions out. To make this easier, I'm publishing the handy one-liner commands to do so here:
Installation Result
Message: The update completed successfully, but the system needs to be rebooted for the changes to be effective.
Reboot Required: true
VIBs Installed: INT_bootbank_ixgben_1.6.5-1OEM.600.0.0.2768847
VIBs Removed: VMW_bootbank_ixgben_1.4.1-2vmw.650.1.26.5969303
VIBs Skipped:
but you'll need to disable the native driver before it will work, explained here.
For reference, you can remove un-used VIBs. Here's how to remove 4.5.3, for example:
[root@xd-1567-5028d:~] esxcli software vib remove --vibname=ixgbe
[NoMatchError]
No VIB matching VIB search specification 'ixgbe'.
Please refer to the log file for more details.
[root@xd-1567-5028d:~] esxcli software vib remove -n net-ixgbe
Removal Result
Message: The update completed successfully, but the system needs to be rebooted for the changes to be effective.
Reboot Required: true
VIBs Installed:
VIBs Removed: INT_bootbank_net-ixgbe_4.5.3-1OEM.600.0.0.2494585
VIBs Skipped:
[root@xd-1567-5028d:~]
After 6 successful years testing then shipping well over 1,000 Xeon D Bundles, Wiredzone had to stop selling them in mid-2021 due to cost, supply, and logistics challenges. So far, Xeon D-1700/2700 (Ice Lake D) was a minor refresh for 2023, with Xeon D-1800/2800 (Granite Rapids D) refresh looking a little better for 2024. More cores, but still just PCIe Gen4 and DDR4. Looking ahead, I'm glad it's Pat Gelsinger at Intel's helm. I'm also grateful to have had the honor of working at VMware when Pat was the CIO there. I'll leave it at that, given the whole Broadcom thing.
