« Managing Block Storage At Scale: Introducing EMC ProSphere | Main | How PEER 1 Built A Commercial Cloud Storage Service With Atmos »

July 14, 2011



In the world I live in my VMs run operating systems that aren't expecting the amount of installed RAM to vary from minute to minute and it's unnecessarily complicated to spin up and shut down load-balanced VMs in response to workload (if the given application could even support such a scenario) so I allocate the VM as much as it needs to handle its peak workload and sleep easy knowing that I'm using a hypervisor that allows other VMs to take advantage of that memory when it's not being actively used. This has come to mean that more memory is allocated than is being used at any given time. I've realized some considerable cost savings from this over the past 4 years which are about to be collected in arrears.

Fortunately we just upgraded from Enterprise to Enterprise Plus for access to the additional features or we would be looking at additional license costs just to maintain compliance for our current environment. As it stands, a capacity expansion planned and budgeted for next year is going to have to go back to the drawing board because we weren't anticipating any net-new licenses being required.

Is there a shipping product today that will give vRAM back to the pool when it's not being used? Should I wait to upgrade to vSphere 5 until such a thing becomes available? If they wanted vRAM to be seen as a 'dynamic' resource maybe it should be licensed based on how much the VMs are using instead of how much is allocated. It would also be clearer if sockets were removed from the equation altogether and it was just licensed per GB. It's been a pleasure using ESX for the past 4 years as a product that essentially stayed out of the way and let me do what I was already doing better, faster, and cheaper. I'm not sure paying for the privilege of redesigning my environment to conform to their vision is going to be quite as sweet.

Stu Fox


Ys, there is a shipping product that gives vRAM back to the pool when it's not being used. It's called Hyper-V Dynamic Memory, and does exactly that. It's designed exactly for the scenario you describe - you set an upper limit for peak load, and Hyper-V will allow that memory to shrink and grow as demand requires.


Disclaimer: I work for Microsoft NZ, but this is my personal post, not that of my employer.


I run a couple of smallish vSphere clusters on ordinary twin-socket hosts with >100GB of RAM each. My VMs are ordinary Windows machines, so they aren't going to be dynamically changing their RAM allocation soon. I suspect I am fairly typical of the non-"ginormous" end of the market. The licensing changes make memory overcommit look very unattractive now. Which is odd, because it used to be one of VMware's better selling points.

48GB per socket is a stingy allowance now - look at the hardware. What is it going to look like in a year's time? It isn't just about the disruption, we need licensing that reflects the reality of modern hardware, is predictable, and fits customer needs.


Party like it's 2007 - The new vRAM licensing scheme makes me think it's time to bring back 4GB dimms and 1GbE.

As a customer/user of VMware for 12 years (I still have my VMware 1.0.2 CD for linux I believe it was their first release on hard media) it makes me sad more than angry.

Unless they make a serious change I'll stick to vSphere 4.1 for a while longer then move to KVM. I suspect the backlash is becoming large enough for them to be getting cold feet now and they'll probably change direction in the coming weeks before vSphere 5 is released. I read someone else mention a similar scenario last year when VMware was pitching to service providers, there was a massive outcry and they ended up slashing the costs in half but it was apparently still really expensive.

If VMware had a more ala carte offering that could make up for a lot of it. Myself I don't care about a lot of these new fancy management bolt ons that are coming out. I care most about a hypervisor. I ran ESX and GSX before it for years without things like vMotion or DRS or HA. The only reason I would go for something like Enterprise plus is because I'm forced into it due to licensing. I like having the features but they aren't required for the stuff I do.

I didn't see one technological thing introduced in vSphere 5 that got me interested in using it -- even if it did not have this new vRAM licensing. The only thing missing from vSphere 4 IMO is to remove the limitation on CPU cores/socket which is a licensing issue, vSphere 4 itself goes to 160 cores/threads. 16-core cpus are just around the corner.

vSphere 4 by contrast I think was a much more significant technological advancement.


Compare that vSphere 4.0 to this vSphere 5.0

I see maybe 5-6 new things in vSphere 5, a lot of "enhancements" on vSphere 4 features like

- Elimination of ESX everything ESXi now (myself I liked the thick ESX so I will miss it)
- vSphere file system - just a new version of VMFS that scales better
- Storage I/O control - already in vSphere 4.1, perhaps they made it better
- vStorage API - already in vSphere 4.1 - looks like they improved it
- Network I/O control - already in vSphere 4.1
- Distributed switch - same..
- HA - same
- vmotion - old news, F5 was showing off WAN vmotion over VERY high latency links a long time ago

- ESXi firewall - it was there in ESXi v3, they removed it in v4 though only to bring it back in a new improved form
- larger VMs - I can't think of anyone in their right mind running a VM with a TB of memory, I mean come on. Run it on real hardware so your not hindered by 32 vCPU limit, since most systems that can run 1TB of ram are going to be capable of having 80+ CPU cores.

To me all the new high end scalability of vSphere is nothing more than a marketing gimmick. I believe I read somewhere vSphere 5 can do 1 million IOPS. How many customers are going to have more storage arrays than servers? Since there is no storage system out there that can come close to a million IOPS by itself.

Think back to 2008...

Where VMware was bragging about doing 100,000 IOPS on one system, they needed 3 storage arrays to pull it off. Even today the thought of one system needing 100,000 IOPS is just crazy.

Same goes for network throughput.

Ok I take that back - I see one good feature in vSphere 5 that is the new vCenter linux-based appliance and web management console. While I haven't tried to check I assume that is backwards compatible with vSphere 4, just like vCenter 4 could manage ESX 3 hosts.

I am not even considering building VMware hosts with less than 256GB of memory these days.

I don't think VMware understands how fragile their position is in the market. I'm sure some of their big core customers won't budge one bit, but a large portion of the market out there will(including me though I'll be kicking and screaming!)

Don't get me wrong I think VMware is a great technology and I have been an extremely loyal user/fan/customer since their inception, but this licensing stuff is worse than anything even Oracle has pulled(I like Oracle DB too btw).

LAI Loong Fong

The new pricing metric is vRAM per socket, not just vRAM alone. What is the point of having VM that support 1TB of vRAM when it is more expensive than a physical machine?! We have been trying to migrate all the large memory machine to VMware and setting up ESX 4 sockets hosts with 512GB RAM but it looks like we do not have enough licenses to support such an environment?! I would need 3 times the licenses to support such an environment!

We have been upgrading through various version of VMware and have been taxed along the way, i.e. ESX2.5 + vMotion -> ESX3 Enterprise Tax -> vSphere4 Enterprise+ Tax -> vSphere5 vRAM tax!! Are we going to have IOPS tax or VMFS capacity tax next??

The comments to this entry are closed.

Chuck Hollis

  • Chuck Hollis
    SVP, Oracle Converged Infrastructure Systems

    Chuck now works for Oracle, and is now deeply embroiled in IT infrastructure.

    Previously, he was with VMware for 2 years, and EMC for 18 years before that, most of them great.

    He enjoys speaking to customer and industry audiences about a variety of technology topics, and -- of course -- enjoys blogging.

    Chuck lives in Vero Beach, FL with his wife and four dogs when he's not traveling. In his spare time, Chuck is working on his second career as an aging rock musician.

    Warning: do not ever buy him a drink when there is a piano nearby.

    Note: these are my personal views, and aren't reviewed or approved by my employer.
Enter your Email:
Preview | Powered by FeedBlitz

General Housekeeping

  • Frequency of Updates
    I try and write something new 1-2 times per week; less if I'm travelling, more if I'm in the office. Hopefully you'll find the frequency about right!
  • Comments and Feedback
    All courteous comments welcome. TypePad occasionally puts comments into the spam folder, but I'll fish them out. Thanks!