ASRock, Bryce, and fkhsr team up across continents and deliver quite the Z77 VT-d/VMDirectPath article and test matrix, thank you!

Posted by Paul Braren on May 4 2012 (updated on Aug 14 2014) in
  • HomeLab
  • Virtualization
  • If you are keen on shopping for a Z77 motherboard, and really have your heart set on VMDirectPath, then this article is for you!

    Be sure not to miss the the Google Doc that Bryce has created, this thing is a masterpiece, Z77 ESXi 5 Board Matrix

    It goes way more in depth than my original, July 2011 spreadsheet about Z68 VMDirectPath successes and failures at TinkerTry.com/vmdirectpath, and related article TinkerTry.com/sandybridgeversusivybridge.

    VT-d capability is handy for when you'd like to be able to "pin" current and future PCI devices to a particular VM, side-stepping devices that the hypervisor itself doesn't support. Such as assigning HighPoint USB 3.0 card ports to a Windows VM, which works great with tinkertry.com/vzilla for example. Or perhaps a Thunderbolt card, assuming they ever become affordable.

    Below is a reformatted, but unedited, forum-like dialogue between site visitor Bryce and fkhsr, pulled from the active Disqus comments below this popular (13,000 views) TinkerTry.com/sandybridgeversusivybridge article. Enjoy!


    Bryce (US) April 25 2012 6:00pm eastern:

    I'm looking to replicate your ASRock Z68 build on Z77.  However, according the manual the ASRock Z77 Professional board no longer includes the option to enable VT-d in the Advanced North Bridge Sections.  Other than the upgraded Intel chip set its very similar to your Z68 board, which means it offers good Ethernet and SATA compatibly with ESXi 5.  I have an inquiry out to ASRock to see if the feature is truly gone.

    Bryce (US) April 26 2012 6:50pm eastern:

    I've been compiling a spreadsheet of some high end Z77 boards. Here is what I have so far. If you or anyone would like to contribute I think it would be a valuable resource. For each board in question I've been thru manuals or found speficications. I'll update links in the the spreadsheet comments.
    http://docs.google.com/spreadsheet/ccc?key=0AjymuQhfM0vYdHZtNThGSllMeU1SMU9ldVltUmp4NWc

    fkhsr (France) April 26 2012 8:38pm eastern:

    Thanks so much for the testing, I'm kind of in the same situation as Bryce (looking to replicate your setup on Z77, for gaming and other stuff) ; i have been searching quite a while to find out that manufacturers don't communicate that much about it..

    So I decided to send a mail to ASRock (asking about status for the support of VT-d on Z77 MBs) and here's (a snippet of) there answer :

    Dear customer,
    [...already known stuff preceeding...]
    Last but not least it is maybe best to also ask Intel if the Z77 chipset does support VT-D. I found that the Z77 Extreme6, Z77 Extreme4 and Z77 Pro4 all have an option in the BIOS for Intel virtualization Technology (VMM) based on the Vanderpool protocol.
    Last but not least it is maybe best to also ask Intel if the Z77 chipset does support VT-D.
    I found that the Z77 Extreme6, Z77 Extreme4 and Z77 Pro4 all have an option in the BIOS for Intel virtualization Technology (VMM) based on the Vanderpool protocol.
    Please give me some feedback, and let me know if I can be of assistance

    Kind regards/ mit freundlichen Gruessen,ASRock Support

    So maybe if we all spam Intel about this they will do something (kiding.. or not) ? I really wouldn't like to be forced to buy current gen-1 MB just because of this..What do you think about this guys ? And here a screenshot of the asrock z77 extreme6 UEFI :

    ASRockUEFI

    Bryce (US) April 27 2012 12:14am eastern:

    Intel is supporting VT-d on their flagship Z77 board, the DZ77GA-70K. http://www.intel.com/support/motherboards/desktop/sb/cs-030922.htm If you have an open line of communication with ASRock refer them to this link.

    Bryce (US) May 3 2012 12:28pm eastern:

    I've finally gotten a ASRock rep to confirm support. The Z77 Professional Manual will be updated to include VT-d Support.
    forums.tweaktown.com/asrock/48417-fatal1ty-z77-professional-vt-d-support.html

    Moderator (US) May 3 2012 8:04pm eastern:

    footnote, from above site, important to note
    The quad core K models do NOT support SIPP, vPro, VT-d and TXT

    Sounds similar to 2600 versus 2600k issue, no?  Discussed at _Best affordable Intel CPU to buy for your personal virtualization project, Core i7 2600, not the 2600K! article._

    See also the original source of this tip:
    www.anandtech.com/show/5771/the-intel-ivy-bridge-core-i7-3770k-review/2

    Best affordable Intel CPU to buy for your personal virtualization project, Core i7 2600, not the 2600K!

    fkhsr (France) May 5 2012 2:47pm eastern:

    Sorry for the delay, I've just passed this info onto asrock support in Netherlands (maybe tinkererguy passed it onto ASRock US and they may not have shared the info).
    Anyway I'l keep an eye on the web to find new info (especially at TinkerTry.com ;) ) and keep you informed if I find some info I think is relevant to us.
    Anyway one thing is sure : I'll be ready to build my rig on july this year ; hopefully more info is available till then.

    Thanks.

    first.last (US) May 5 2012 8:48pm eastern:

    I got the same information from ASRock tech support as Bryce regarding the Z77 Professional board supporting VT-d. Based on the email chain I had with them, it appears ASRock U.S. passed the question on to ASRock Taiwan, which responded with the following:

    Dear Xxxxx,Thanks for your reply. There is an option of "Vt-d" in Z77 Professional BIOS.Z77 Professional user manual don't mention the option, we are modifying the user manual. If any question, please kindly contact us.Thanks for your e-mail. Have a nice dayASRock TSD

    Bryce (US) May 21 2012 12:53pm eastern:

    Hi folks,

    Have recently finished building my own ASRock Z77 Extreme4 with LSI MegaRAID 2965 SAS controller (including CacheCade 2.0) thanks to the great advice on this site. I actually ended up building my own ESXi install image with all the drivers I needed in it and thought this might be useful to others so have included my notes on using VMware PowerCLI to build a custom install ISO from the latest ESXi release including security updates below.

    Note after downloading the extra ESXi drivers I had unziped them to my working directory.
    Note: the set of Get-ESXImageProfile commands is just to narrow down the latest release - probably a better way but it works :-)

    Source: http://alexander.walden.com.au/2012/01/18/making-your-own-esxi-5-installation-image/

    Download ESXi 5 drivers from LSI and from VMWare.
    (VMWare.com – Support and Downloads (Hover) – Product
    Downloads – Vmware vSphere – Drivers and Tools Tab – Expand Driver CDs - VMware
    ESXi 5.0 Driver CD for Broadcom NetXtreme I Gigabit Ethernet Driver -
    3.120h.v50.2)

    Add-EsxSoftwareDepot http://hostupdate.vmware.com/software/VUM/PRODUCTION/main/vmw-depot-index.xml
    Add-EsxSoftwareDepot ..\tg3-3.120h.v50.2-547149\tg3-3.120h.v50.2-offline_bundle-547149.zip
    Add-EsxSoftwareDepot ..\VMW-ESX-5.0.0-LSIProvider-500.04.V0.24-261033-456178\VMW-ESX-5.0.0-LSIProvider-500.04.V0.24-261033-offline_bundle-456178.zip
    Get-ESXImageProfile
    Get-ESXImageProfile -name "ESXi-5.0.0-201205*"
    Get-ESXImageProfile -name "ESXi-5.0.0-201205*"|format-list
    New-EsxImageProfile -CloneProfile ESXi-5.0.0-20120504001-standard -Name ESXi-5.0.0-20120511001-Custom
    Set-EsxImageProfile -ImageProfile ESXi-5.0.0-20120511001-Custom -AcceptanceLevel CommunitySupported
    (Get-EsxImageProfile -Name ESXi-5.0.0-20120511001-Custom).VibList
    Get-EsxSoftwarePackage
    Add-EsxSoftwarePackage -ImageProfile ESXi-5.0.0-20120511001-Custom -SoftwarePackage net-tg3
    Add-EsxSoftwarePackage -ImageProfile ESXi-5.0.0-20120511001-Custom -SoftwarePackage LSIProvider
    Export-EsxImageProfile -ImageProfile ESXi-5.0.0-20120511001-Custom -ExportToBundle -FilePath " ESXi-5.0.0-20120511001-Custom.zip"
    Export-EsxImageProfile -ImageProfile ESXi-5.0.0-20120511001-Custom -ExportToISO -FilePath "ESXi-5.0.0-20120511001-Custom.iso"
    Compare-EsxImageProfile ESXi-5.0.0-20111204001-standard ESXi-5.0.0-20120511001-Custom

    Sep 15 2012 Update:

    Here's a nice write-up on more Z77 success, and a little "shout-out" to TinkerTry that is greatly appreciated, where Kuonen says:

    Inspired by vZilla on tinkertry.com 

    Read all the build details, see photos, and a lot of screenshots over at vEsuvio 4.0 – ivy bridge whitebox. Enjoy!


    Jun 04 2013 Update:
    Nice write-up on some speed testing of SSDs under Hyper-V, using the GA-Z77X-UD5H, over at:
    Performance: Using SSD’s for VM Drives in Home Hyper-V Server on GA-Z77X-UD5H, June 3, 2013, by John Stutsman