How to easily update your VMware Hypervisor from 6.5.x to 6.5 Update 1 Express Patch 4 (ESXi 6.5 U1 EP4)

Posted by Paul Braren on Oct 8 2017 (updated on Mar 31 2018) in
  • ESXi
  • Virtualization
  • HowTo
  • HomeLab
  • Important Update - On Mar 20 2018, VMware VMSA-2018-0004.3 announced that CVE-2017-5715 (Spectre-2) mitigation is now included in the latest patch that you should be using instead of the older patch featured in the original article below. You'll find the newer article that features an even easier update method here:

    Article below as it originally appeared.


    Once you've performed this patch and rebooted, your ESXi system will show as:

    • 6.5.0 Update 1 (Build 6765664)
    • (Updated) ESXi-6.5.0-20171004001-standard (VMware, Inc.)

    Even if you don't use vSAN, you may be interested in the new NVMe inbox (included) driver that arrived in 6.5 Update 1 and is still included in EP 04, it's named:

    • VMW_bootbank_nvme_1.2.0.32-4vmw.650.1.26.5969303

    Below, you'll find both the detailed Step-By-Step Instructions and the walk-thru Video, but you should really read the whole article first, including a way to backup your ESXi media before you begin described in the Prerequisites section, and a way to roll-back easily if things don't go right, see Reverting to a previous version of ESXi (1033604).

    VMware ESXI 6.5 U1 EP04

    Read both of these KB articles for details on what this Express Patch fixes:

    • VMware ESXi 6.5, Patch Release ESXi650-201710401 (2151081)
      kb.vmware.com/kb/2151081

      Release Date: October 5, 2017
      NOTE: This patch release contains a fix for a rare but highly critical vSAN bug. For more information, see KB 2151061.
      The vSAN issue outlined in KB 2151061 manifests itself only under highly specific operations and IO patterns.

    • VMware ESXi 6.5, Patch Release ESXi650-201710401-BG: Updates esx-base, esx-tboot, vsan and vsanhealth VIBs (2151061)
      kb.vmware.com/kb/2151061

      Patch Category - Bugfix
      Patch Severity - Critical
      Build - For build information, see KB 2151081.
      Host Reboot Required - Yes
      Virtual Machine Migration or Shutdown Required - Yes
      Affected Hardware - N/A
      Affected Software - N/A
      VIBs Included -

      • VMware_bootbank_esx-base_6.5.0-1.29.6765664
      • VMware_bootbank_esx-tboot_6.5.0-1.29.6765664
      • VMware_bootbank_vsan_6.5.0-1.29.6765666
      • VMware_bootbank_vsanhealth_6.5.0-1.29.6765667

      PRs Fixed - 1869931, 1886760, 1893508, 1966720
      Related CVE numbers - N/A

    See also details about the original 6.5 Update 1 Release below.

    VMware ESXI 6.5 U1

    Release Notes. The simple update method that this article details means you won't need the ISO Download Page for:
    ESXi 6.5 U1 | 27 JULY 2017 | Build 5969303

    This upgrade is also known as version 6.5.0 Build 5969303 or 6.5U1.

    More about this update in KB 2149910:


    Warning:

    1. vCenter/VCSA 6.5 should be upgraded to 6.5 Update 1 (aka U1) before upgrading your host(s) to ESXi 6.5 U1 EP04 Build 6765664, see:
      How to easily update your VMware vCenter Server Appliance from 6.5.x to 6.5 Update 1 (VCSA 6.5 U1)
      Jul 28 2017
    2. I have only tested this method when upgrading from 6.5.0a Build 5224934 to Build 5969303, your experience from earlier 6.x versions may vary.
    3. I have been able to replicate that the Xeon D 10GbE X552/X557 driver VIB needs to be re-installed right after the upgrade, simple one line workaround is documented here, with details below.
    4. This is not official VMware documentation, it's merely a convenient upgrade technique that may help in lab tests, a little simpler than the official procedure VMware documents and demonstrates in KB2008939. It's up to you to adhere to the backup-first advice detailed below, full Disclaimer found at below-left, at the bottom of very TinkerTry page.
    5. See also the Drawbacks section below.

    Why ESXCLI?

    All the background story on how this easy ESXCLI upgrade method came about was covered in my earlier articles about updating 6.0 U2 and 6.5.
    If you're in production, beware, this code just came out yesterday. This article is for the lab, where you may want to give this critical patch a try.

    Benefits

    1. No new license needed to go from 6.0.x or 6.5.x to 6.5 U1 Build 6765664
    2. Users of the free hypervisor and folks who can't download the GA Offline bundle now have a path forward as well, without needing to read TinkerTry's My VMware's "You either are not entitled or do not have permissions to download this product." error, and what to do about it.

    Drawbacks

    1. See also Differences Between vSphere Upgrades and Updates and Upgrade or Update a Host with Image Profiles:

      The esxcli software profile update command brings the entire contents of the ESXi host image to the same level as the corresponding upgrade method using an ISO installer. However, the ISO installer performs a pre-upgrade check for potential problems, and the esxcli upgrade method does not. The ISO installer checks the host to make sure that it has sufficient memory for the upgrade, and does not have unsupported devices connected. For more about the ISO installer and other ESXi upgrade methods, see Upgrade Options for ESXi 6.0.

    2. See also Upgrading Hosts by Using esxcli Commands and Overview of the ESXi Host Upgrade Process.
    3. Before proceeding, you should read Overview of the ESXi Host Upgrade Process. This article below is just about the quick and easy way, effective and safe for most folks. For those more interested in "clean installs", where you login to My VMware, download the ESXi 6.5U1 ISO, shut down the ESXi on USB that you're already running, eject that USB flash drive and label it and set it aside, then boot from another USB drive like the SanDisk Ultra Fit with a fresh install of 6.5U1 imaged onto it. This clean install is much more time consuming than the easy method outlined below. Why? This is because once ESXi 6.5U1 is freshly installed, at a minimum you'll also have to use Datastore Browser to locate your VMs on your VMFS datastores, then add those files with *.vmx extensions back into your inventory, then add the host back to your cluster that should already be at 6.5U1. While this extra work may help you be sure that you don't have any drivers or changes carried over from your previous build, for many users, that's not a concern.

    Prerequisites

    Once you've completed ALL of the following preparation steps:

    1. upgraded to VCSA 6.5 U1
    2. ensured your ESXi 6.5.x host has a working internet connection
    3. reviewed the release notes
    4. reviewed How to easily update your VMware Hypervisor to ESXi 6.0 Update 2 for the full back story that includes some warnings about potential gotchas/driver issues
    5. backed up the ESXi 6.5.x you've already got, if its on USB or SD, then use something like one of the home-lab-friendly and super easy methods such as USB Image Tools under Windows, as detailed by Florian Grehl here

    you can now continue with this simple approach to upgrading your lab environment. Unsupported, at your own risk, see the full disclaimer at below-left.

    You should wind up with the same results after this upgrade as folks who upgrade by downloading the full ESXi 6.5 U1 ISO / creating bootable media from that ISO / booting from that media (or mounting the ISO over IPMI/iLO/iDRAC/IMM/iKMV) and booting from it:

    File size: 332.63 MB
    File type: iso
    Name: VMware-VMvisor-Installer-6.5.0.update01-5969303.x86_64.iso
    Release Date: 2017-07-27
    Build Number: 5969303

    installing it, rebooting, then running the patch process described below.

    Upgrade Step-by-Step

    Download and upgrade to 6.5U1EP04 update using the patch directly from the VMware Online Depot

    The entire process including reboot is usually well under 10 minutes, and many of the steps below are optional, making it appear more difficult than it is. Triple-clicking on a line of code below highlights the whole thing with a carriage return, so you can then right-click and copy it into your clipboard, which gets executed immediately upon pasting into your SSH session. If you want to edit the line before it's executed, manually swipe your mouse across the code rather than triple-clicking the lines of code.

    1. Open an SSH session (eg. PuTTY) to your ESXi 6.0.x server
      (if you forgot to enable SSH, here's how)
    2. Turn on maintenance mode, or ensure you've set your ESXi host to automatically gracefully shutdown all VMs upon host reboot, or shutdown all the VMs gracefully that you care about, including VCSA.
    3. Firewall allow outbound http requests - Paste the one line below into into your SSH session, then press enter:
      esxcli network firewall ruleset set -e true -r httpClient

      More details about the firewall here.

    4. Pull down ESXi Image Profile using https and run patch script - Paste the line below into into your SSH session, then hit enter and wait while nothing seems to happen, taking somewhere between roughly 3 to 10 minutes before the completion screen (sample below) appears:
      esxcli software profile install -p ESXi-6.5.0-20171004001-standard -d https://hostupdate.vmware.com/software/VUM/PRODUCTION/main/vmw-depot-index.xml

      It warns you what VIBs will be removed if you proceed. Note that next command is the same, but with
      --ok-to-remove
      added at the end. This allows the upgrade to proceed, now that you've been properly warned. Be sure to make note of what VIBs it says will be removed, just in case the inbox (included) drivers it installs don't work for your system.

      esxcli software profile install -p ESXi-6.5.0-20171004001-standard -d https://hostupdate.vmware.com/software/VUM/PRODUCTION/main/vmw-depot-index.xml --ok-to-remove

      If you find they don't work, after rebooting, you'll need to locate the original VIB download site and install it, using the detailed install instructions usually found at the vendor's VIB download site. Now that the included AHCI/SATA driver has been fixed, home lab enthusiasts are likely to find such issues much less common.
      If these esxcli software profile install commands fails, you may want to try changing update to install, details below, see also Douglas' comment. Wait time for the successful install depending mostly on the the speed of the ESXi's connection to the internet, and somewhat on the write speed of the storage media that ESXi is installed on.

    5. OPTIONAL - Xeon D with 10GbE - If your system includes two 10GbE Intel X552/X557 RJ45 or SFP+ NICs ports, they can be used for 1GbE or 10GbE speeds, but you'll need to regain the 10GbE Intel driver VIB that the upgrade process replaced with an older one that doesn't work with your X557. Simply copy and paste the following one-liner fix:
      esxcli software vib install -v https://cdn.tinkertry.com/files/net-ixgbe_4.5.2-1OEM.600.0.0.2494585.vib --no-sig-check

      as described in detail here before proceeding.

    6. OPTIONAL - Xeon D-1567 - If your system uses the Xeon D-1567 (12 core) you may find the VMware ESXi 6.0 igb 5.3.3 NIC Driver for Intel Ethernet Controllers 82580, I210, I350, and I354 performs better for the service console on either ETH0 or ETH1 instead of the included-with-6.5U1EP4 VMware inbox driver for I-350 called
      VMW_bootbank_net-igb_5.0.5.1.1-5vmw.650.0.0.4564106. No need to download separately. Simply copy and paste the following one-liner fix:
      esxcli software vib install -v https://cdn.tinkertry.com/files/net-igb_5.3.3-1OEM.600.0.0.2494585.vib --no-sig-check

      before proceeding.

    7. OPTIONAL - Intel Optane P4800X - If your system has an Intel Optane P4800X NVMe SSD of either the PCIe or U.2 type, you'll need the Intel driver for proper support, find your NVMe firmware version, then reference this version to verify the exact VIB you should be using on the VMware HCL - IO Devices Keyword P4800X. If it's intel-nvme version 1.2.1.15, simply paste the easy one-liner fix:
      esxcli software vib install -v https://cdn.tinkertry.com/files/intel-nvme-1.2.1.15-1OEM.650.0.0.4598673.x86_64.vib --no-sig-check

      before proceeding.

    8. Firewall disallow outbound http requests - To return your firewall to how it was before this online upgrade, simply copy and paste the following:
      esxcli network firewall ruleset set -e false -r httpClient
    9. If you turned on maintenance mode earlier, remember to turn maintenance mode off.
    10. If you normally leave SSH access off, go ahead and disable it now.
    11. Type or paste
      reboot

      and hit return (to restart your ESXi server), or use your favorite ESXi UI to restart the host.

    12. After the reboot is done, it would be a good idea to test login using ESXi host client, pointing your browser to the IP or hostname of your just-graded server, to be sure everthing seems to be working right.

    You're done!

    Special thanks to VMware ESXi Patch Tracker by Andreas Peetz at the VMware Front Experience Blog. This upgrade test was performed on a TinkerTry'd VMware HCL system. Yes, on both the very popular 8 core and the rather special 12 core version of the beloved Supermicro SuperServer SYS-5028D-TN4T system.

    esxcli-update-successful-65U1-Build-6765664-TinkerTry-Oct-08-2017
    Here's how my upgrade from 6.5.0d to 6.5 U1 Build 5969303 looked, right after the 1 minute download/patch.
    DCUI-showing-6.5-build-6765664-TinkerTry-2017-OCT-08
    Yep, it worked! This is called the DCUI, using Supermicro's iKVM HTML5 UI to show you what my console looked like after the patch & reboot.
    ESXi-host-client-6.5U1-build-6765664-fulllength-TinkerTry-2017-OCT-08
    ESXi Host client view of Build 6765664.

    That's it! When the reboot is complete, you'll see for yourself that you now have the latest ESXi, Build 6765664, as pictured above. Now you have more spare time to read more TinkerTry articles!

    When the upgrade is complete, on the ESXi Host Client UI, under Host / Configuration, you should see the following "Image profile"
    (Updated) ESXi-6.5.0-20171004001-standard (VMware, Inc.)

    Potential gotchas

    1. Depending upon your ESXi firewall configuration, if the above command results in a network related error such as:
      'NoneType' object has no attribute 'close'
      then you skipped the firewall configuration step above, try again!

    2. Notice that the command recommended you use when clicking on the ESXi-6.5.0-20170304101-standard link at VMware ESXi Patch Tracker:
      esxcli software profile update -p ESXi-6.5.0-20170702001-standard -d https://hostupdate.vmware.com/software/VUM/PRODUCTION/main/vmw-depot-index.xml

      doesn't work, says:
      Message: Host is not changed.
      but simply changing from update to install worked for me, but your results may vary. See also the interesting comment below.

      host-is-not-changed
      Using the update parameter doesn't work, as seen above, but using install does.

    Video

    How to easily update your VMware Hypervisor from 6.5.x to 6.5 Update 1 Express Patch 04

    Closing Thoughts

    Alternatively, you could have used VMware Update Manager on a Windows system or VM, but for one-off upgrades typical in a small home lab, pasting these 3 or 4 lines of code is pretty darn easy.

    Looking ahead, since VUM is now built into VCSA 6.5, this adds another way to do future upgrades and patches, even in a small home lab environment.


    Nov 30 2017 Update

    This release is also known as:

    • ESXi 6.5 U1 Express Patch 4
    • ESXi 6.5 U1 EP04
    • ESXi 6.5 U1 EP4
    • ESXi 6.5 U1EP4
    • ESXi 6.5U1EP4

    All ESXi releases have been nicely documented by VMware here:

    From the above table, I created this summary of this 6.5U1EP4 release:

    • Version - ESXi 6.5 U1 Express Patch 4
      Release Date - 2017-10-05
      Build Number - 6765664
      Installer Build Number - N/A

    It's quite possible the above upgrade technique will work for all of the following 6.x versions of ESXi, but I haven't tested:

    Version | Release Date | Build Number | Installer Build Number
    ESXi 6.5 U1 Express Patch 4 2017-10-05  6765664 N/A
    ESXi 6.5 U1 2017-07-27  5969303 N/A
    ESXi 6.5.0d 2017-04-18  5310538 N/A
    ESXi 6.5. Express Patch 1a  2017-03-28  5224529 N/A
    ESXi 6.5. Patch 01  2017-03-09  5146846 5146843
    ESXi 6.5.0a 2017-02-02  4887370 N/A
    ESXi 6.5 GA 2016-11-15  4564106 N/A
    ESXi 6.0 Patch 6    2017-11-09  6921384 N/A
    ESXi 6.0 Express Patch 11   2017-10-05  6765062 N/A
    ESXi 6.0 Update 3a (ESXi 6.0 Patch 5)   2017-07-11  5572656 N/A
    ESXi 6.0 Express Patch 7c   2017-03-28  5251623 N/A
    ESXi 6.0 Express Patch 7a   2017-03-28  5224934 N/A
    ESXi 6.0 Update 3   2017-02-24  5050593 N/A
    ESXi 6.0 Patch 4    2016-11-22  4600944 N/A
    ESXi 6.0 Express Patch 7    2016-10-17  4510822 N/A
    ESXi 6.0 Patch 3    2016-08-04  4192238 N/A
    ESXi 6.0 Express Patch 6    2016-05-12  3825889 N/A
    ESXi 6.0 Update 2   2016-03-16  3620759 N/A
    ESXi 6.0 Express Patch 5    2016-02-23  3568940 N/A
    ESXi 6.0 Update 1b  2016-01-07  3380124 N/A
    ESXi 6.0 Express Patch 4    2015-11-25  3247720 N/A
    ESXi 6.0 U1a (Express Patch 3)  2015-10-06  3073146 N/A
    ESXi 6.0 U1 2015-09-10  3029758 N/A
    ESXi 6.0.0b 2015-07-07  2809209 N/A
    ESXi 6.0 Express Patch 2    2015-05-14  2715440 N/A
    ESXi 6.0 Express Patch 1    2015-04-09  2615704 2615979
    ESXi 6.0 GA 2015-03-12  2494585 N/A

    See also at TinkerTry


    See also


    Upgrade Log

    Below, I've pasted the full text of my upgrade, helps you see what drivers were touched, use the horizonal scroll bar or shift + mousewheel to look around, and Ctrl+F to Find stuff quickly:

    login as: root
    Using keyboard-interactive authentication.
    Password:
    The time and date of this login have been sent to the system logs.
    
    WARNING:
       All commands run on the ESXi shell are logged and may be included in
       support bundles. Do not provide passwords directly on the command line.
       Most tools can prompt for secrets or accept them from standard input.
    
    VMware offers supported, powerful system administration tools.  Please
    see www.vmware.com/go/sysadmintools for details.
    
    The ESXi Shell can be disabled by an administrative user. See the
    vSphere Security documentation for more information.
    [root@xd-1541-5028d:~] esxcli network firewall ruleset set -e true -r httpClient
    [root@xd-1541-5028d:~] esxcli software profile install -p ESXi-6.5.0-20171004001-standard -d https://hostupdate.vmware.com/software/VUM/PRODUCTION/main/vmw-depot-index.xml
     [Exception]
     You attempted to install an image profile which would have resulted in the removal of VIBs ['INT_bootbank_iavmd_1.2.0.1011-1OEM.650.0.0.4598673', 'INT_bootbank_intel-nvme_1.2             .1.15-1OEM.650.0.0.4598673']. If this is not what you intended, you may use the esxcli software profile update command to preserve the VIBs above. If this is what you intended             , please use the --ok-to-remove option to explicitly allow the removal.
     Please refer to the log file for more details.
    [root@xd-1541-5028d:~] esxcli software profile install -p ESXi-6.5.0-20171004001-standard -d https://hostupdate.vmware.com/software/VUM/PRODUCTION/main/vmw-depot-index.xml --ok-to-remove
    Installation Result
       Message: The update completed successfully, but the system needs to be rebooted for the changes to be effective.
       Reboot Required: true
       VIBs Installed: VMW_bootbank_nvme_1.2.0.32-4vmw.650.1.26.5969303, VMware_bootbank_esx-base_6.5.0-1.29.6765664, VMware_bootbank_esx-tboot_6.5.0-1.29.6765664, VMware_bootbank_vsan_6.5.0-1.29.6765666, VMware_bootbank_vsanhealth_6.5.0-1.29.6765667
       VIBs Removed: INT_bootbank_iavmd_1.2.0.1011-1OEM.650.0.0.4598673, INT_bootbank_intel-nvme_1.2.1.15-1OEM.650.0.0.4598673, VMware_bootbank_esx-base_6.5.0-1.26.5969303, VMware_bootbank_esx-tboot_6.5.0-1.26.5969303, VMware_bootbank_vsan_6.5.0-1.26.5912915, VMware_bootbank_vsanhealth_6.5.0-1.26.5912974
       VIBs Skipped: VMW_bootbank_ata-libata-92_3.00.9.2-16vmw.650.0.0.4564106, VMW_bootbank_ata-pata-amd_0.3.10-3vmw.650.0.0.4564106, VMW_bootbank_ata-pata-atiixp_0.4.6-4vmw.650.0.0.4564106, VMW_bootbank_ata-pata-cmd64x_0.2.5-3vmw.650.0.0.4564106, VMW_bootbank_ata-pata-hpt3x2n_0.3.4-3vmw.650.0.0.4564106, VMW_bootbank_ata-pata-pdc2027x_1.0-3vmw.650.0.0.4564106, VMW_bootbank_ata-pata-serverworks_0.4.3-3vmw.650.0.0.4564106, VMW_bootbank_ata-pata-sil680_0.4.8-3vmw.650.0.0.4564106, VMW_bootbank_ata-pata-via_0.3.3-2vmw.650.0.0.4564106, VMW_bootbank_block-cciss_3.6.14-10vmw.650.0.0.4564106, VMW_bootbank_char-random_1.0-3vmw.650.0.0.4564106, VMW_bootbank_ehci-ehci-hcd_1.0-4vmw.650.0.14.5146846, VMW_bootbank_elxnet_11.1.91.0-1vmw.650.0.0.4564106, VMW_bootbank_hid-hid_1.0-3vmw.650.0.0.4564106, VMW_bootbank_i40en_1.3.1-5vmw.650.1.26.5969303, VMW_bootbank_igbn_0.1.0.0-14vmw.650.1.26.5969303, VMW_bootbank_ima-qla4xxx_2.02.18-1vmw.650.0.0.4564106, VMW_bootbank_ipmi-ipmi-devintf_39.1-4vmw.650.0.0.4564106, VMW_bootbank_ipmi-ipmi-msghandler_39.1-4vmw.650.0.0.4564106, VMW_bootbank_ipmi-ipmi-si-drv_39.1-4vmw.650.0.0.4564106, VMW_bootbank_ixgben_1.4.1-2vmw.650.1.26.5969303, VMW_bootbank_lpfc_11.1.0.6-1vmw.650.0.0.4564106, VMW_bootbank_lsi-mr3_6.910.18.00-1vmw.650.0.0.4564106, VMW_bootbank_lsi-msgpt2_20.00.01.00-3vmw.650.0.0.4564106, VMW_bootbank_lsi-msgpt3_12.00.02.00-11vmw.650.0.0.4564106, VMW_bootbank_misc-cnic-register_1.78.75.v60.7-1vmw.650.0.0.4564106, VMW_bootbank_misc-drivers_6.5.0-1.26.5969303, VMW_bootbank_mtip32xx-native_3.9.5-1vmw.650.0.0.4564106, VMW_bootbank_ne1000_0.8.0-16vmw.650.1.26.5969303, VMW_bootbank_nenic_1.0.0.2-1vmw.650.0.0.4564106, VMW_bootbank_net-bnx2_2.2.4f.v60.10-2vmw.650.0.0.4564106, VMW_bootbank_net-bnx2x_1.78.80.v60.12-1vmw.650.0.0.4564106, VMW_bootbank_net-cdc-ether_1.0-3vmw.650.0.0.4564106, VMW_bootbank_net-cnic_1.78.76.v60.13-2vmw.650.0.0.4564106, VMW_bootbank_net-e1000_8.0.3.1-5vmw.650.0.0.4564106, VMW_bootbank_net-e1000e_3.2.2.1-2vmw.650.0.0.4564106, VMW_bootbank_net-enic_2.1.2.38-2vmw.650.0.0.4564106, VMW_bootbank_net-fcoe_1.0.29.9.3-7vmw.650.0.0.4564106, VMW_bootbank_net-forcedeth_0.61-2vmw.650.0.0.4564106, VMW_bootbank_net-igb_5.0.5.1.1-5vmw.650.0.0.4564106, VMW_bootbank_net-ixgbe_3.7.13.7.14iov-20vmw.650.0.0.4564106, VMW_bootbank_net-libfcoe-92_1.0.24.9.4-8vmw.650.0.0.4564106, VMW_bootbank_net-mlx4-core_1.9.7.0-1vmw.650.0.0.4564106, VMW_bootbank_net-mlx4-en_1.9.7.0-1vmw.650.0.0.4564106, VMW_bootbank_net-nx-nic_5.0.621-5vmw.650.0.0.4564106, VMW_bootbank_net-tg3_3.131d.v60.4-2vmw.650.0.0.4564106, VMW_bootbank_net-usbnet_1.0-3vmw.650.0.0.4564106, VMW_bootbank_net-vmxnet3_1.1.3.0-3vmw.650.0.0.4564106, VMW_bootbank_nhpsa_2.0.6-3vmw.650.0.0.4564106, VMW_bootbank_nmlx4-core_3.16.0.0-1vmw.650.0.0.4564106, VMW_bootbank_nmlx4-en_3.16.0.0-1vmw.650.0.0.4564106, VMW_bootbank_nmlx4-rdma_3.16.0.0-1vmw.650.0.0.4564106, VMW_bootbank_nmlx5-core_4.16.0.0-1vmw.650.0.0.4564106, VMW_bootbank_ntg3_4.1.2.0-1vmw.650.1.26.5969303, VMW_bootbank_nvmxnet3_2.0.0.22-1vmw.650.0.0.4564106, VMW_bootbank_ohci-usb-ohci_1.0-3vmw.650.0.0.4564106, VMW_bootbank_pvscsi_0.1-1vmw.650.1.26.5969303, VMW_bootbank_qedentv_2.0.3.29-1vmw.650.0.0.4564106, VMW_bootbank_qfle3_1.0.2.7-1vmw.650.0.0.4564106, VMW_bootbank_qflge_1.1.0.3-1vmw.650.0.0.4564106, VMW_bootbank_qlnativefc_2.1.50.0-1vmw.650.1.26.5969303, VMW_bootbank_sata-ahci_3.0-26vmw.650.1.26.5969303, VMW_bootbank_sata-ata-piix_2.12-10vmw.650.0.0.4564106, VMW_bootbank_sata-sata-nv_3.5-4vmw.650.0.0.4564106, VMW_bootbank_sata-sata-promise_2.12-3vmw.650.0.0.4564106, VMW_bootbank_sata-sata-sil24_1.1-1vmw.650.0.0.4564106, VMW_bootbank_sata-sata-sil_2.3-4vmw.650.0.0.4564106, VMW_bootbank_sata-sata-svw_2.3-3vmw.650.0.0.4564106, VMW_bootbank_scsi-aacraid_1.1.5.1-9vmw.650.0.0.4564106, VMW_bootbank_scsi-adp94xx_1.0.8.12-6vmw.650.0.0.4564106, VMW_bootbank_scsi-aic79xx_3.1-5vmw.650.0.0.4564106, VMW_bootbank_scsi-bnx2fc_1.78.78.v60.8-1vmw.650.0.0.4564106, VMW_bootbank_scsi-bnx2i_2.78.76.v60.8-1vmw.650.0.0.4564106, VMW_bootbank_scsi-fnic_1.5.0.45-3vmw.650.0.0.4564106, VMW_bootbank_scsi-hpsa_6.0.0.84-1vmw.650.0.0.4564106, VMW_bootbank_scsi-ips_7.12.05-4vmw.650.0.0.4564106, VMW_bootbank_scsi-iscsi-linux-92_1.0.0.2-3vmw.650.0.0.4564106, VMW_bootbank_scsi-libfc-92_1.0.40.9.3-5vmw.650.0.0.4564106, VMW_bootbank_scsi-megaraid-mbox_2.20.5.1-6vmw.650.0.0.4564106, VMW_bootbank_scsi-megaraid-sas_6.603.55.00-2vmw.650.0.0.4564106, VMW_bootbank_scsi-megaraid2_2.00.4-9vmw.650.0.0.4564106, VMW_bootbank_scsi-mpt2sas_19.00.00.00-1vmw.650.0.0.4564106, VMW_bootbank_scsi-mptsas_4.23.01.00-10vmw.650.0.0.4564106, VMW_bootbank_scsi-mptspi_4.23.01.00-10vmw.650.0.0.4564106, VMW_bootbank_scsi-qla4xxx_5.01.03.2-7vmw.650.0.0.4564106, VMW_bootbank_shim-iscsi-linux-9-2-1-0_6.5.0-0.0.4564106, VMW_bootbank_shim-iscsi-linux-9-2-2-0_6.5.0-0.0.4564106, VMW_bootbank_shim-libata-9-2-1-0_6.5.0-0.0.4564106, VMW_bootbank_shim-libata-9-2-2-0_6.5.0-0.0.4564106, VMW_bootbank_shim-libfc-9-2-1-0_6.5.0-0.0.4564106, VMW_bootbank_shim-libfc-9-2-2-0_6.5.0-0.0.4564106, VMW_bootbank_shim-libfcoe-9-2-1-0_6.5.0-0.0.4564106, VMW_bootbank_shim-libfcoe-9-2-2-0_6.5.0-0.0.4564106, VMW_bootbank_shim-vmklinux-9-2-1-0_6.5.0-0.0.4564106, VMW_bootbank_shim-vmklinux-9-2-2-0_6.5.0-0.0.4564106, VMW_bootbank_shim-vmklinux-9-2-3-0_6.5.0-0.0.4564106, VMW_bootbank_uhci-usb-uhci_1.0-3vmw.650.0.0.4564106, VMW_bootbank_usb-storage-usb-storage_1.0-3vmw.650.0.0.4564106, VMW_bootbank_usbcore-usb_1.0-3vmw.650.1.26.5969303, VMW_bootbank_vmkata_0.1-1vmw.650.1.26.5969303, VMW_bootbank_vmkplexer-vmkplexer_6.5.0-0.0.4564106, VMW_bootbank_vmkusb_0.1-1vmw.650.1.26.5969303, VMW_bootbank_vmw-ahci_1.0.0-39vmw.650.1.26.5969303, VMW_bootbank_xhci-xhci_1.0-3vmw.650.0.0.4564106, VMware_bootbank_cpu-microcode_6.5.0-0.0.4564106, VMware_bootbank_emulex-esx-elxnetcli_11.1.28.0-0.0.4564106, VMware_bootbank_esx-dvfilter-generic-fastpath_6.5.0-0.0.4564106, VMware_bootbank_esx-ui_1.21.0-5724747, VMware_bootbank_esx-xserver_6.5.0-0.23.5969300, VMware_bootbank_lsu-hp-hpsa-plugin_2.0.0-5vmw.650.1.26.5969303, VMware_bootbank_lsu-lsi-lsi-mr3-plugin_1.0.0-10vmw.650.1.26.5969303, VMware_bootbank_lsu-lsi-lsi-msgpt3-plugin_1.0.0-7vmw.650.1.26.5969303, VMware_bootbank_lsu-lsi-megaraid-sas-plugin_1.0.0-8vmw.650.1.26.5969303, VMware_bootbank_lsu-lsi-mpt2sas-plugin_2.0.0-6vmw.650.1.26.5969303, VMware_bootbank_native-misc-drivers_6.5.0-0.0.4564106, VMware_bootbank_rste_2.0.2.0088-4vmw.650.0.0.4564106, VMware_bootbank_vmware-esx-esxcli-nvme-plugin_1.2.0.10-1.26.5969303, VMware_locker_tools-light_6.5.0-0.23.5969300
    [root@xd-1541-5028d:~] esxcli software vib install -v https://cdn.tinkertry.com/files/net-ixgbe_4.5.1-1OEM.600.0.0.2494585.vib --no-sig-check
    Installation Result
       Message: The update completed successfully, but the system needs to be rebooted for the changes to be effective.
       Reboot Required: true
       VIBs Installed: INT_bootbank_net-ixgbe_4.5.1-1OEM.600.0.0.2494585
       VIBs Removed: VMW_bootbank_net-ixgbe_3.7.13.7.14iov-20vmw.650.0.0.4564106
       VIBs Skipped:
    [root@xd-1541-5028d:~] esxcli software vib install -v https://cdn.tinkertry.com/files/net-igb_5.3.3-1OEM.600.0.0.2494585.vib --no-sig-check
    Installation Result
       Message: The update completed successfully, but the system needs to be rebooted for the changes to be effective.
       Reboot Required: true
       VIBs Installed: INT_bootbank_net-igb_5.3.3-1OEM.600.0.0.2494585
       VIBs Removed: VMW_bootbank_net-igb_5.0.5.1.1-5vmw.650.0.0.4564106
       VIBs Skipped:
    [root@xd-1541-5028d:~] esxcli software vib install -v https://cdn.tinkertry.com/files/intel-nvme-1.2.1.15-1OEM.650.0.0.4598673.x86_64.vib --no-sig-check
    Installation Result
       Message: The update completed successfully, but the system needs to be rebooted for the changes to be effective.
       Reboot Required: true
       VIBs Installed: INT_bootbank_intel-nvme_1.2.1.15-1OEM.650.0.0.4598673
       VIBs Removed:
       VIBs Skipped:
    [root@xd-1541-5028d:~] esxcli network firewall ruleset set -e false -r httpClient
    [root@xd-1541-5028d:~]