VMware vSphere Server Appliance 5.5 SSO doesn't like lack of proper DNS in a home lab, here's a workaround

Sep 25 2013 Update:
vSphere 5.5 has been released, including ESXi 5.5 and vCenter Server Appliance 5.5. This problem with Single Sign On not liking lack of reverse DNS lookup is still there, so this post is still relevant. Figured out a work around yesterday, to be properly documented (write-up and video) soon.

Oct 01 2013 Update: Comprehensive, beginning to end build video now available:
Build your own VMware vSphere 5.5 Datacenter with ESXi and VCSA

Original article appears below, with all the updated info appended.

Like most home networks, I've got no fully-configurable DNS server in my current home lab, nor do I want one. This also means no DNS reverse lookup that vCenter really digs. So it just stands to reason that configuring the VMware vCenter Server Appliance (VCSA) would be just slightly more difficult in such a situation. The workaround I came up with is fairly simple, and the results are looking good!

Before we dive in, I should also add that it doesnt help that I have a complete lack of documentation for this greatly enhanced appliance (since this code isn't GA yet), but that just makes the tinkering that much more fun! And this is still a whole lot easier than installing vCenter on Windows Server platforms.

Unlike earlier betas, what I discovered the hard way, was that the vSphere Server Appliance 5.5.0.5100 build 1266838 struggles with most attempts at using the "Configure with default settings" wizard option, usually failing with a nasty error, seen below.

Here's the cut-and-paste of the actual, full error:

Failed to execute '/usr/sbin/vpxd_servicecfg 'sso' 'write' 'embedded' CENSORED CENSORED 'default-pass'':
VC_CFG_RESULT=702(Error: An unexpected error occurred during the installation of the appliance SSO service. Please collect a support bundle and file a service request.)

Like 9 out of 10 times, it fails. Hmm. Not good. Earlier builds didn't have that issue. But then there was that one time this week when by chance, my router granted a DHCP lease that just happened to match the hard coded IP address that I wanted. Then everything worked. So it appeared that changing the IP address and/or hostname of the appliance might have something to do with it. So this breakage wasn't due to the usual culprits, like time/NTP settings, for example.

So for this week, and specifically for tonight, I just wanted to get a procedure together that's reliable, and repeatable. Now that I've tested my method 3x in a row, I'm ready to begin to document the procedure.

Here's the workaround that'll do nicely for now, unless I come up with something more elegant:

• set a Manual MAC address on the vCenter appliance, in my case, it's 00:50:56:00:00:51
• set DHCP reservation on my home router's DHCP Reservations table, so it'll always get the same IP address, in my case, it's 10.10.1.51
• perform the rest of the configuration like Vladen Seget here (but using 'Configure with default settings')

That's it! More details to follow, in my upcoming comprehensive step-by-step install video, and a full guide to go along with it. At least now I have an appliance ready for my broadcasts hangouts tonight, so I can then use the vSphere Web Client that is essential to trying many of the new ESXi 5.5 features.

Sep 01 2013 Update:
I got this repeatable process working well, and have recorded a comprehensive, step by step guide to building your own datacenter (ESXi 5.5 and vCenter Appliance), all demonstrated in under an hour. I'm working on publishing this 1920x1080 video by Tue Sep 3 2013, with written instructions to follow.

Sep 24 2013 Update:
The video was published as promised, Build your own VMware vSphere Datacenter in under an hour with the free ESXi 5.5 hypervisor by Paul Braren on Tue Sep 3rd, 2013. With ESXi 5.5 now out, a new, simpler workaround to this DNS issue is being worked on now, to be documented soon. See also comment below.

Sep 25 2013 Update:
A reliable method to avoid a lack of true DNS and reverse lookup in the home lab has been developed. I will be documenting  the procedure in writing, and on video, soon.

You can see that I got DNS working well, no issues with vCenter appliance, and nice simple hostnames names like 'esxi' and 'vcenter', pictured below. Yes, that's on the actual released ESXi 5.5.0 Build 1331820 and vCenter Server Appliance Build 1312297.

To get notified of the article that details the exact steps, please follow TinkerTry.com using any of the described methods, and I'll be sure to let you know when it's done.