I had a bit of fun last week, commenting on Nigel Poulton’s blog post discussing this topic, among others. What I really enjoyed was the polite but vivacious discussion that emerged in the comments.
For the most part, people were disagreeing without being disagreeable. How civil.
That being said, I tend to reflect on things after the fact. I came to the conclusion that it was an incomplete discussion on several levels.
If you’re not into storage stuff, perhaps this would be a good post to skip. A friendly reminder: everything in this blog represents my personal opinion, and is not reviewed nor approved by employer. Or anyone else for that matter.
Why Are We Talking About This?
Storage professionals can make a good living by helping IT teams to tame the storage beast so that’s it a bit less unruly.
The winds of change are upon us, though. Server designs (and storage software) has matured to the point where it can be seriously considered as an alternative to the familiar external storage array for a growing number of use cases. Management models are converging, where it's becoming more common to see servers and storage managed via a single role.
New choices abound. And when there are new choices to consider, the debates start in earnest. All part of the fun!
Most everyone should be familiar with external storage arrays. These are purpose-built boxes, often comprised of commodity components, running specialized software with a wealth of functionality: snaps, replication, tiering, dedupe, encryption.
Next up, VSAs, or virtual storage appliances, if you prefer.
These are pure software stacks that run on familiar servers. Their presentation model is also familiar: they expose iSCSI, NFS, etc. and are basically run like dedicated storage, except using off-the-shelf hardware. The software may either run physically on bare metal, or virtualized in a VM alongside other applications. Many examples of these, with more coming.
And, finally, the new kid in town: VSAN, or VMware Virtual SAN more formally.
Like a VSA, VSAN runs on most anything that supports VMware, alongside other applications, with a few caveats. Unlike a VSA, VSAN is an extension to the ESXi hypervisor: it’s deeply integrated into everything the hypervisor does. This may sound like a minor implementation detail, but it’s not — it fundamentally changes the value proposition of the product.
So, What’s Important Here?
Being a huge fan of oversimplification, I’d like to use three primary criteria for comparing the “goodness” of our contestants.
First, does it do the intended job? There’s a vast universe of workloads out there, each different, and a given technology is either a good fit or not. Because we tend to use imprecise labels to describe workloads, there’s a bit of confusion, e.g. all Oracle workloads are not created alike.
Second, does it give good value? Here we have a broad discussion around price/performance, price/capacity, price/functionality, etc. We also want to consider the operational aspects over the life cycle: implementation, management, upgrading, etc.
Finally, does it prepare me for the future? Storage evolution continues to pick up speed. We’ve got new devices (NVMe, motherboard-based flash), new converged management models, and entirely new application workloads to consider. Nobody wants to end up in a technological cul-de-sac, especially when you consider the 3-5 year average life cycle of a storage investment.
What will the world look like in 2020?
Cross-Compare: VSA and External Arrays
Before we go too far, it’s important to point out that VSAs seem to target two distinct markets. The first are smaller environments where budget and resources are inherently constrained. The second are larger, scale-out environments that look more like a service provider or cloud model.
Both VSAs and external arrays preserve the familiar storage-centric management model. You provision and manage storage separately, and make it available through a network protocol: iSCSI, NFS, object, etc.
If we go to our first criteria (does it do the job?) the first thing you’ll notice is that very few vendors are positioning VSAs for bread-and-butter IT workloads: Microsoft apps, databases and the like. Here, external arrays are the more likely choice. Write-heavy transactional workloads and most VSAs don’t seem to get along well.
VSA vendors, please correct me if I'm wrong here :)
On the second criteria (good value for money?), there are pros and cons on both sides. VSAs have the obvious advantage of running on most anything, which is especially convenient if you have unused (or underused) hardware on hand.
Apples-to-apples on a new acquisition is a much closer compare, financially speaking. Parts is parts, although pricing on server hardware is generally much more transparent than larger array hardware.
Operationally, larger-scale VSAs seem to require more “fiddling” than purpose-built arrays, but that's just my observation. The arrays also have the advantage that their software knows *exactly* what it’s running on, which confers noticeable benefits in supportability and performance optimization.
On the third criteria (does this prepare me for the future?), some may see the VSA to have a small advantage.
Upgrading to new hardware with a VSA is usually the incremental insertion of new technology into your server farm and updated software. While arrays can be upgraded within a range, they periodically go through a major product revision, which generally results in acquiring a completely new array.
Most often, you can take your VSA licenses to new hardware, new hypervisor, etc. Similarly, you can take your array to support new servers, operating systems, hypervisors, etc. In my eyes, that’s sort of a wash.
When cool new storage technology comes out (new flash devices, etc.) the VSA approach makes it easier to adopt vs. waiting for your array vendor to support it, either in the array you own today, or more likely the next array you buy. Depending on how hard you want to push the technology curve, that could be a factor.
Choices, choices …
Cross-Compare: VSAs vs. VSAN
We actually need to make two compares here: one compare for smaller environments, and another for larger, scale-out environments.
The first is the technology model; VSA storage software runs in user space as does any other application; VSAN is an integral part of vSphere and the ESXi kernel.
The second is the management model: VSAs present as familiar, separately managed storage, while VSAN is an integral part of VM management — which can be frustrating for traditional storage types :)
Let’s take on our first criteria — does it do the job?
When we look at the smaller end of the market, VSAN has a minimum of 3 nodes (two for protection plus a witness), while some VSAs can support 2 or even 1 node — of course, with associated compromises in certain kinds of availability handling.
But if you’re driven on price, price, price — that could be a factor.
At the other end of the market (large scale out), VSAN does scale considerably today (32 nodes, 4.4 petabytes using 4TB drives), with rumored expansion to 64 nodes and 6TB drives (that’s 13.4 PB for those of you keeping score). I'm not going to confirm or deny rumours here, but my point is -- not small, by any means.
But, by comparison, some VSAs can get truly big: many hundreds or even thousands of nodes. That could be appealing to cloud-like service provider environments where you’d like to manage the storage farm separately, or huge content farms.
But let’s put the shoe on the other foot, shall we?
VSAN is very comfortable in mixed VM workload and write-intensive environments which you’ll find in many bread-and-butter enterprise settings. Lots of flash to buffer writes, enough spindles to destage efficiently, excellent resource management thanks to kernel level integration, very little overhead, etc. Most VSAs don’t go there. Read performance with low latency? VSAN is no slouch either. Bandwidth intensive? It depends on what you’re doing.
But there are clear differences as to where each fits.
Arguably, many of today’s VSAs offer richer data services than today’s VSAN. But not everyone needs a great deal of those features, and -- besides -- I don’t think that situation will persist for long :)
Let’s move to the second, more controversial question — good value for money?
No, I can’t point you at published documentation on this -- I really wish I could -- but it’s real and easily observable over a wide range of configurations and workloads. Do your own testing (or talk to someone who has), and you’ll see what I mean.
That’s just one of the advantages of being integrated into the kernel vs. being forced to run in user application space.
As a standalone, list price proposition, some have thought the license cost could have been lower for VSAN, but there’s more to it than that. First, VMware customers tend to buy a lot of different VMware products, so list pricing is certainly not the norm. Second, it’s per-socket pricing (vs. capacity pricing) which can be very cost-effective if you want, errr, capacity.
But all of this masks the real win: operational efficiency. Talk to any VSAN user, and the first thing they’ll usually say is “nothing could be easier” in day-to-day operations. It’s really quite remarkable.
If you’ve got (or want!) a single role managing both servers and storage for your VMware farm using a converged, application-centric policy-driven model, you’ll understand the unique appeal.
Now onto an even more controversial subject — prepared for the future?
Both VSAs and VSAN can accommodate new storage technology quickly, simply through software updates. Let’s call that a tie.
One of Nigel’s points in his blog post is that VSAN is tied to vSphere, and many VSAs are hypervisor agnostic. Without sounding too partisan, this is a red herring for most people. Many shops are fully committed to vSphere, and plan to be that way for a while.
It is a rare and painful occurrence for someone to actually attempt to replatform a perfectly good vSphere cluster onto another hypervisor, but — theoretically — that’s a valid point. And if you do decide to migrate, there are nice tools (e.g. storage vMotion) to help you out. Just to be complete, there are a handful of shops that fully intend to run multi-hypervisor, and want a single storage solution that’s uniform across all environments. VSAs would win here, as would external arrays.
More strategically, if you believe the future involves software-defined data centers (and software-defined storage), I would offer the mildly partisan argument that VSAN has a long term advantage here. VMware is pouring substantial resources to make VSAN the poster child for software-defined storage as part of the bigger, fully-automated SDDC picture. You see a lot of that already in today's product.
Cross-compare: VSAN vs. External Arrays?
There's just too much discontinuity that results from trying to directly compare the two. Many people get lost in the discussion, and miss the key points. Instead, I've found the topics are easier to understand when you put them on a continuum.
One set of things happen when you go from external arrays to considering VSAs, and break the vapor lock between external storage hardware and storage software. Pros and cons, to be sure.
And another set of benefits materialize when you consider moving from VSAs to VSAN, and no longer have storage software being forced to run as any other user application.
Stepping Back A Bit
A few years back, VCE was created on the premise that many people would see value in having storage, network and compute integrated and supported as a single product. The category of converged infrastructure was born. Very popular, as it turns out.
A bit later, a few enterprising startups took VSA technology, and wrapped into a single, integrated appliance. They started to use the term hyperconverged infrastructure to describe this approach. Also becoming popular.
VMware came along this year, and went one better -- integrating storage and network as part of the hypervisor.
The most logical term I can think of is hypervisor-converged infrastructure. So far, doing quite well thank you. And now there's EVO:RAIL for those that prefer the simplicity of pre-integrated hardware.
The point here is that more and more of the market is starting to prefer to consume storage functionality as an integrated part of the whole vs. a standalone proposition. Whether that's converged, hyperconverged or hypervisor-converged -- it's an undeniable factor in the storage market and broader IT thinking today, and will continue to be so in the future.
Despite some people banging on about lock-in being the work of Satan, or similar :)
The nice thing about the storage market these days is that there are so many choices out there -- technology, management model, consumption model, etc. -- keeping every storage vendor on their toes.
Including VMware :)
Like this post? Why not subscribe via email?