Should I try to upgrade my Intel Core i7 2600 (Sandybridge) past 32GB of RAM for juggling a lot of VMs, or plan to rebuild on Core i7 4770 (Haswell)?

Posted by Paul Braren on Apr 24 2013 (updated on Jun 17 2013) in
  • CPU
  • ESXi
  • Variety-of-OSs-on-my-ESXi

    So, it would seem I'm at a fork in the road. Trusty vZilla has now been alive for about 2 years, a Core i7 2600 CPU on a Z68 (Sandybridge) ASRock motherboard with 4 DIMM slots, humming along with ESXi 5.1 and nested Hyper-V quite nicely. Right now, I'm at 4 x 8GB, totaling 32GB, and I'm running pretty maxed out on memory, 24x7, with about 10 VMs always going. It's a good thing, that my personal family cloud has grown, is getting heavily used, with only memory as a bottleneck. Most days, CPU usage and disk contention just aren't an issue at all.

    So I'm also beginning to wonder what's next, how do I grow affordably. The answer is currently looking pretty murky.

    You see, Intel's Haswell ain't quite out yet, not until roughly June 2013, according to Tom's Hardware rumors, with it's new H87 designation and LGA 1150 socket type, which may turn out to be the last socket to allow customer-installed CPUs. At least consumers won't get the USB 3.0 bug.

    The the preliminary information also seems to indicate just 4 DIMM slots in 2 channels, discussed here for example. Perhaps one of those ~$600 64GB DIMM kits (16GB x 4) will be a possibility, with some Kingston examples seen here:
    www.kingston.com/us/memory/valueram/server
    with 128GB DIMM kits also listed, but not yet affordable. Then again, this rumor chart only shows 32GB DIMM capacity maximum as well. Uh oh.

    The other thing is that the Haswell's at 84 TDP  (Watts).
    en.wikipedia.org/wiki/Haswell_(microarchitecture)

    That don't impress me much. My 2 year old Core i7 2600 TDP is 95 Watts. So it'll be 2 and a quarter years, but only 11 watts less?

    Also unimpressive is the expected fairly modest increase in overall performance, described here:
    www.tomshardware.com/reviews/core-i7-4770k-haswell-performance,3461-8.html

    Things are much worse when looking into ECC memory for server class mobos. While they may have 8 DIMM slots, likely to be found on a new wave of Xeons arriving 2H2013 rumored here, it's likely to be rather spendy, with upwards of $2000 needed just for an ECC 64GB kit, examples seen here. Rumored DDR4 version will not only be more expensive, but at a TDP of 130 Watts, I'm just not interested, especially for 24x7 use.

    What I'm thinking is that a doubling of the memory is all I really need to get at least another year out of my current configuration. Could it be possible that a 64GB kit might actually work on my current Sandybridge motherboard? I don't know, but I'm going to try to find out.*

    Haswell-Microarchitecture
    from hexus.net

    *Apr. 25 2013 Update:
    From the looks of this Intel article, it sure doesn't look likely. Sure feels like we're being pushed to go server class to break this 2 year 32GB barrier. Then comes Skylake then Skymont, but that speculation is ever further out. Or will it be soldered-on Broadwell, using the Ball Grid Array package mentioned here?

    Perhaps it's time to get creative, and set aside an SSD as a way to alleviate potential memory pressure (swap) related slow-downs, using the Host Cache Configuration that I talked about, way back in October of 2011.

    We can't forget licensing barriers as well. We cannot assume ESX 5.5 or 6.0 or whatever they name it will support anything greater than the current 32GB limit in the free hypervisor version either, see also:
    TinkerTry.com/32gb

    Consider also Z77 mobos with 8 DIMM slots, as suggested by fellow Google+ member Ernesto Morin. Good point!  The thing is, those came out late 2011. Let's hope something affordable comes out with 8 DIMM slots in the next year or two.

    Ernesto also asked, why not Ivy Bridge E? It may come out in November 2013, discussed here. But I wouldn't consider a refresh of late 2011 tech, stuck at 130 watts TDP, particularly innovative, although it may be the only near-term option to somewhat affordably go past 32GB. The higher cost of electricity to run it, moving from 95 watts to 130 watts, would come out to about $50 per year for that 35 watt delta, but then there's also the the higher cost of cooling to consider.


    May 01, 2013 Update:
    Well, hearing more about Haswell during their recent IDF presentation here:
    TinkerTry.com/haswell-at-idf2013

    I'm thinking that perhaps the CPU will stay in a lower power state more often in real-world use, burning less watts during periods of inactivity. In other words, I may have overstated the TDP importance, because if it rarely reaches that maximum 84 watts, then it could be that Haswell winds up being much more miserly than Sandybridge and Ivybridge. Will those lower power states would kick in with VMware ESXi or Hyper-V juggling a typical home lab's VMs, that are idle most of the time? That remains to be seen...


    Jun 03 2013 Update:
    Haswell's out, and here's an interesting related story at ServeTheHome:
    Maximum Memory of the Haswell Intel Xeon E3-1200 V3 Series, Posted June 2, 2013 by Patrick Kennedy


    Jun 17 2013 Update:
    Want to read more about 2014's Haswell-E, LGA 2011-3, and the Wellsburg Chipset? Check out vr-zone article.

    Haswell will not be succeeded by Broadwell 14nm Tick in 2014 (I’m sorry to break all your hearts on this). Instead, we’ll be treated to a Haswell refresh, most certainly improved clock speeds at the same (never-changing) price.

    We will finally have an 8-core desktop CPU from Intel!

    With Haswell-E, Intel will do away with 4-core configurations of these GPU-less dies and offer users a choice of 6 and 8 core CPUs, with up to 20 MB of L3 cache. Of course, Hyper Threading isn’t going anywhere giving the CPU a maximum of 16 logical cores. Maximum TDP for the platform will lie in the range of 130W to 14oW and the processors itself will be built on 22nm 2nd generation Hi-k process.

    Read more: http://vr-zone.com/articles/intel-core-i7-ivy-bridge-e-core-i3-haswell-lineup-detailed/37832.html#ixzz2WTsxvo4B

    ...

    The chipset will support low voltage (1.2V) DDR4 RAM kits (must be the 1333 MHz ones).

    Read more: http://vr-zone.com/articles/intel-core-i7-ivy-bridge-e-core-i3-haswell-lineup-detailed/37832.html/2#ixzz2WTuVyDyi