VMware Appliance for Folding@home - help researchers unfold the mystery of diseases like Coronavirus/COVID-19 virtually

Posted by Paul Braren on Mar 21 2020 (updated on Mar 29 2020) in
  • Efficiency
  • ESXi
  • Virtualization
  • HowTo
  • HomeLab
  • Tips added since initial publication:

    1. There are server-side issues that result in failing to get work assigned to your VM, see ongoing discussions here, new automatic bigadv on 16 CPU or larger VMs seems to have helped.
    2. I have now published a short video of downloading/deploying/configuring 1.0.1 on one ESXi 6.7U3 server, it's pretty easy, and only took about 10 minutes!
    3. As of Tue Mar 24 2020, I'm starting to have some success with more consistently getting workloads, but only if I reboot idle VMs occasionally.
    4. Getting the optional passkey and entering it during OVF deployment or afterward via the Web Control UI gets more points.

    Article as it originally appeared below, with updates appended.


    I first spotted a few tweets about this ongoing effort by William Lam @lamw earlier this week, hearing he was working on an OVA file to make downloading and installing Folding@home much easier. Hey, I have a home lab, and 3 of my systems could be taken away from my incessant tinkering and put to an even better use for a while.

    I'm new to Folding@home, but managed to deploy a few copies of William Lam's OVA which had just been published earlier that day.

    Table of Contents

    1241111572962889728

    Given all the novel Coronavirus 2019 / COVID-19 concern out there, what could be better than a good excuse to have a little distraction (and fun) putting my vSphere 6.7 Update 3 cluster back together, and a chance to use my lab's smooth-running DNS and certificate-error-free HTML5 based vSphere Client.

    So when this nudge from my work colleague Matt Webb @dinotheroller arrived, I had already been working to clear my plate of day job and personal responsibilities so I'd be able to invest some of my time into this project Friday night. Yes, responsibilities like getting food when there's less people shopping, got that done first.

    Announcement

    Amanda's words are excellent, I encourage you to read her whole article, with just an excerpt below. I've even had the chance to meet her in person at a VMUG a while back, you know, back when we could meet people face-to-face.

    vmware-appliance-fah
    • AMANDA BLEVINS
      Mar 20, 2020
      A Force for Good: VMware Appliance for Folding@Home

      At VMware, our EPIC2 values are a core component of who we are. We manifest our values in all aspects of our work and through being a Force for Good in our communities. When I initially read an article about Folding@Home and the project’s impact in helping researchers find treatments for diseases such as Coronavirus, Cancer, Alzheimer’s, and more, I knew that I had to contribute to the project! To quote foldingathome.org:

      The Folding@Home project (FAH) is dedicated to understanding protein folding, the diseases that result from protein misfolding and aggregation, and novel computational ways to develop new drugs in general.
      ...

    TinkerTry Datacenter's 4 system CPU summary:

    • Xeon D 1541 8 core / 16 threads, 16 vCPUs assigned to 1 Folding@home VM
    • Xeon D 1541 8 core / 16 threads, 16 vCPUs assigned to 1 Folding@home VM
    • Xeon D 1567 12 core / 24 threads, 8 vCPUs assigned to 1 Folding@home VM
    • Xeon D 2123IT 4 core / 8 threads, 8 vCPUs assigned to 1 Folding@home VM

    TOTAL - 24 CPU cores / 48 vCPUs dedicated to Folding@home

    Given my SuperServer Workstation is also my daily driver for all TinkerTry content creation and 4K video editing and rendering, and also the RDP client for doing my day job, I needed to avoid using any more than about a third of this systems available threads for VMs.

    folding at home 4 nodes

    Watts

    I'm currently estimating that if I get workload to all my cores, the maximum impact I'll be seeing by firing up 3 additional nodes that I normally leave powered down will be in the in the range of 125 to 175 watts.

    Download

    vmware-appliance-for-folding-home-101

    This is the page where you'll want to download both:
    VMware-Appliance-FaH_1.0.1.ova
    VMwareApplianceforFAHDeploymentSteps.pdf
    from, this Photon OS based appliance is a mere 291MB in size, so it shouldn't take very long.

    While that's downloading, be sure to check out the requirements page to see what else you'll need to know, including port #s that VM will be using, just in case you have outbound firewall rules to deal with. See also security discussion initiated by Chris Colotti here, but he and team 241525 at Cohesity have jumped in as well.

    Configure

    Using vSphere Client to deploy the OVA, you'll simply need to answer some basic questions, and configure your network. It's easiest if you just go with DHCP, and note that this VM doesn't do well if cloned, winding up with a duplicate IP address even if you do remove the virtual NIC to get a new MAC address. It's best to just deploy each VM from the OVA, to as many systems and cores as you'd like to give it.

    You can join:

    • VMware Team ID 52737
    • Dell|EMC Team ID 236797
    • Cohesity Team ID 241525
    • Some VMs on one team, some on the other
    • Some other Team ID of your choosing

    Here's the step-by-step.

    1) Deploy OVF Template

    Steps 1 to 6 are done the usual way you deploy an OVA to a vSphere cluster, using the vSphere client HTML5 UI, see VMware's excellent Deploy an OVF or OVA Template.

    Step 7 is Customize template, this is the screen with the most questions.

    In my lab's DHCP network, I have IPs from 10.10.1.1 to 10.10.1.255 with netmask of 255.255.255.0. Substitute your network IPs appropriately. I noticed that entering a password in the "Password to configure for remote F&H management" doesn't result in a prompt for a password when I use my browser to have a look at status, so I've removed it from my summary table below.

    Here's the only fields I touched, then it works:

    Hostname                      folding5
    DNS                           10.10.1.1
    DNS Domain                    lab.local
    OS Credentials root password  VMware1!
    F&H Team ID                   52737 (leave as-is for VMware, or 236797 for DellTechnologiesNA)
    Remote networks allowed to connect to F@H Web Management
    (http://[FQDN]:7396)          10.10.1.0/24

    2) Customize the VM you just deployed

    • Change the CPUs from 2 to however many you want to give it, I chose 16
    • Change the Memory from the default to 12GB, if you chose 16 CPUs (or more)

    You may want to alter the default "Medium" setting for CPU usage in the Folding Power setting seen in the deployment template, or create a separate Resource Group, especially if you cluster is busy doing other workloads. This can also be changed later on easily using the web client.

    3) Power Up V

    Power on the VM, opening a console so you can see if it successfully starts work. If you get "Failed to get assignment" errors, don't worry, it will automatically keep trying again. If it sits without work assigned for many hours, I have found a reboot of the VM can often get workload assigned within about 5 minutes.

    4) Manage/Monitor VM

    folding-at-home-status-via-Chrome--TinkerTry
    Web control UI

    From a system on the same network as your VM, open a browser and type in:
    http://[IP ADDRESS or HOSTNAME]:7396
    then press enter. There should be no prompts for a password, and you are able to click on the link under 'I'm folding as" to Change Identity, making sure your name, Team Number, and Passkey are in there, using the requested passkey which takes just seconds.

    Troubleshoot

    A variety of errors might be seen on your console view of your VMs, including:
    WARNING Failed to get assignment from
    Empty work server assignment
    Exception: Could not get an assignment

    At this time, there are some difficulties in getting any work done with the VM during the daytime here in the US. In my 18 hours or so of running so far, the errors might resolve themselves eventually if you simply leave them running. Note, the CPU cycles aren't being used during those waiting periods, which will mitigate the effects on your electric bill.

    It's to be expected that the back end servers and employees will need some time to scale up to the increased demand, in part due to this new much-easier-to-deploy VM that even Linux novices should feel comfortable using.

    1240619123664654336

    Greg Bowman @drGregBowman Mar 19
    Please be patient if you experience idle time on @foldingathome as we face the #COVID19 pandemic together! We are working like mad to start simulations of COVID-19 proteins. The enthusiastic response to our effort has been tremendous and has sometimes emptied our queues.

    1240250014204596224

    Jeremiah Miller @perpterds Mar 18
    @foldingathome I think something is up with your stats page/server... Getting constant "bad gateway" errors. Confirmed with a friend.

    Also, we're both getting a LOT of "Failed to get assignment" errors.

    In the Folding Forum, see Temporary server outages for more information on the behind-the-scenes.


    Mar 22 2020 Update

    Last night, I heard William Lam is working on a new release of the OVA, he tweeted the details about the planned change in his tweet:

    automatic configuration of “bigadv” is going to be in 1.0.1 release. In fact, just released it few seconds ago internally for validation before we publish


    Mar 23 2020 Update

    New release announced by William Lam at 10:30am ET! From the Changelog tab:

    March 23, 2020 - v1.0.1

    • Support for VMware Fusion and Workstation
    • Support for vHW 11
    • Automatic detection of CPU >= 16 and configuration of "bigadv" in config.xml (during initial setup and upon each reboot)
    • Fixed HTTP Proxy configuration required in config.xml
    • ClientIdentifier for DHCP has been update to use MAC Address preventing issues when cloning from powered on FAH Appliance
    • Updated Setup Instructions, FAQ and vGPU configuration using NVIDIA GRID
      VMware-Appliance-FaH_1.0.1.ova
      MD5: 3b526544be33c8dc95355e54fddbe2b9

    March 20, 2020 - v1.0.0 (Initial Release)
    VMware-Appliance-FaH_1.0.0.ova
    MD5: fe03d681c8ed747561df8639381b84f5


    Mar 24 2020 Update

    Title changed from:
    VMware Appliance for Folding@home - join the fight against diseases such as Coronavirus virtually
    to:
    VMware Appliance for Folding@home - help researchers unfold the mystery of Coronavirus/COVID-19 disease virtually


    Mar 25 2020 Update

    Title changed from:
    VMware Appliance for Folding@home - help researchers unfold the mystery of Coronavirus/COVID-19 disease virtually
    to:
    VMware Appliance for Folding@home - help researchers unfold the mystery of diseases like Coronavirus/COVID-19 virtually

    Created a shortcut to get to this article that is easily memorized, to share with others. It's tinkertry.com/fah. Changed my name on the Folding@home status page from:
    @paulbraren_tinkertry.com/fah
    to:
    tinkertry.com/fah
    since my first 2,735 credit was apparently not showing my name at all, just blank where the name should be.


    Mar 26 2020 Update

    It's happened 3 times now where I have a VM booted that has had no work assigned for many hours. I heard the interval between subsequent requests for work gets longer the longer the system has been booted. This makes sense, from a scaling perspective. But if I reboot my VM, I find that within about 5 minutes, it usually gets work assigned.


    Mar 27 2020 Update

    Tweeted this important update, and updated the article accordingly:

    1243594059152789504

    Thx! Just moved from old/legacy:
    Team: Dell|EMC Team ID 245895
    https://stats.foldingathome.org/team/245895
    to top-ranked name variant:
    Team: DellTechnologiesNA Team ID 236797
    https://stats.foldingathome.org/team/236797
    Same name, w/ my passkey for more points:
    https://foldingathome.org/support/faq/points/passkey/
    https://TinkerTry.com/fah Updated!

    I have some VMs on the VMware Team ID, and some VMs on the DellTechnologiesNA Team ID. It doesn't really matter, since we only have one human race on earth, if a cure for COVID-19 is eventually developed, we all win.


    Mar 29 2020 Update

    2020-03-28_22-58-31

    I had some datastore errors on my 4 core Xeon D GIGABYTE SERVER system, so I shut down and reseated the NVMe drive. It's an old Micron 22110 M.2 NVMe SSD, which is extra long, so I have to use a PCIe adapter. That adapter may be at fault here. Rebooted, restarted the VM, and no more disk errors, for now. Will monitor.

    2020-03-29_21-52-58

    I noticed that chrome tends to have issues with the Web client, status won't update. Firefox seems to work fine for days. Chrome is just unreliable, even after clearing cookies or forcing a network reload with Ctrl+F5.

    I also noticed that if I look at my Donor: tinkertry.com/fah page:
    https://stats.foldingathome.org/donor/74495481
    for some reason my DellTechnologiesNA Score is going up, but the WUs haven't moved from 1 at all. Not at all sure why, rebooted the VM just in case it makes a difference.


    Video

    TinkerTry YouTube Channel - Mar 23 2020 - VMware Appliance for Folding@home - help researchers fight Coronavirus/COVID-19 disease virtually

    I agree with @sggrc's tweet about the video below:

    A fabulous med school level COVID-19 whiteboard walkthrough:
    https://bit.ly/3dlwjV1
    Sit down and rev-up your propeller beanie. You will learn SO MUCH from this. (The last deep diagnostic portion can be skipped, IMO.)

    Ninja Nerd Science - Mar 16 2020 - COVID-19 | Corona Virus: Epidemiology, Pathophysiology, Diagnostics
    Linus Tech Tips - Mar 22 2020 - Here's How YOU Can Help Find a Cure for COVID-19!

    Screenshots

    2020-03-21_21-37-09
    Deploy OVF Template - 4 Review details
    2020-03-21_21-46-04
    Deploy OVF Template - 8 Ready to complete
    folding-running-Dell-team-work--TinkerTry
    Here's how it's supposed to look. I have a browser open to the hostname on port 7396, which shows the status of the VM at right. You can use the IP instead of the name, if you prefer.
    folding-running-VMware-team-work--TinkerTry
    Here's how it's supposed to look. I have a browser open to the hostname on port 7396, which shows the status of the VM at right. You can use the IP instead of the name, if you prefer.

    Photos

    TinkerTry-workbench-running-3-nodes-2020-03-20
    folding cluster CPU
    Paul-Braren-certification-and-recognition-logos-2017-05-16
    Unfortunately, 3 of my 4 VMs failed to get much work done so far, dropping one-by-one overnight into Saturday March 21 2020.

    Social

    1241076924652425217
    1241161021848420354

    This is crucial information, and I'm looking into this.

    Davide Abrigo @zhaknafein 13 hr
    Ok, looks like I was wrong Smiling face with open mouth and cold sweat...probably Covid projects are assigned only to "bigadv" clients (16+CPU). BTW, joined 32 cores to help

    1241345185331765248

    See also at TinkerTry

    how-to-replace-your-windows-10-certificate-so-browser-security-warnings-go-away-after-replacing-your-vmware-vcsa

    edge-router-lite-update

    See also

    how-to-start-contributing-to-the-foldinghome-project-tips-and-tricks

    fighting-the-covid-19-coronavirus-with-vmware-fusion-and-folding-at-home
    • Fighting the COVID-19 Coronavirus with VMware Fusion and Folding At Home
      Mar 23 2020 by Michael Roy at VMware Blogs | VMware Fusion Blog

      >> Quick Link to the OVA Appliance

      What a time to be alive.

      I’m writing this from my apartment in San Francisco where I’ve been sheltering in place for almost 2 weeks now.

      Personally I had been wondering just how I could help, beyond just applying the rules of today… social distancing, not panic-buying, keeping in touch with friends and family with Zoom and FaceTime, trying to limit time spend on Facebook (okay that last one I’m having a hard time with, but still…).

      All that stuff is good, but surely there has to be more to do without putting anyone at risk, right?

      Well thankfully I’m not the only one thinking that.

      My friends and colleagues William Lam and Amanda Blevins, along with the support of the VMware community, have taken the onus to put together a free virtual appliance that can contribute your spare CPU cycles to the Folding At Home project.
      ...

    how-nvidia-is-using-its-gpu-technolgy-to-fight-against-covid-19-virus

    covid19-update
    • FOLDING@HOME UPDATE ON SARS-COV-2 [10 MAR 2020]
      Mar 10 2020 by John Chodera at Folding@Home

      This is an update on Folding@home’s efforts to assist researchers around the world taking up the global fight against COVID-19.

      After initial quality control and limited testing phases, Folding@home team has released an initial wave of projects simulating potentially druggable protein targets from SARS-CoV-2 (the virus that causes COVID-19) and the related SARS-CoV virus (for which more structural data is available) into full production on Folding@home. Many thanks to the large number of Folding@home donors who have assisted us thus far by running in beta or advanced modes.
      ...

    vmware-appliance-fah
    • AMANDA BLEVINS
      Mar 20, 2020
      A Force for Good: VMware Appliance for Folding@Home

      At VMware, our EPIC2 values are a core component of who we are. We manifest our values in all aspects of our work and through being a Force for Good in our communities. When I initially read an article about Folding@Home and the project’s impact in helping researchers find treatments for diseases such as Coronavirus, Cancer, Alzheimer’s, and more, I knew that I had to contribute to the project! To quote foldingathome.org:

      The Folding@Home project (FAH) is dedicated to understanding protein folding, the diseases that result from protein misfolding and aggregation, and novel computational ways to develop new drugs in general.
      ...

    dig-deeper
    • Folding@Home Dig Deeper

      This section describes how Folding@home simulations work and why our methods benefit from distributed computing. The descriptions found here may be slightly technical, but aims for people who would like to learn more about how this project works and how their computer is involved.
      ...

    Podcast segment about this web site with maps and graphs on Daily Tech News Show at this spot.

    Interactive-Incidents-Map
    IBM Weather Channel's Interactive COVID-19 Map