I've seen disruptions -- small and large -- come and go. If you're into IT infrastructure, this is one worth watching.
A few years ago, I moved from EMC to VMware on the power of that prediction. So far, it’s played out pretty much as I had hoped it would. There’s now clearly a new dynamic in the ~$35B storage industry, and VMware’s Virtual SAN is very emblematic of the changes that are now afoot.
There’s a lot going on here, so it’s worth sharing. In each case, you’ll see a long-held tenent around The Way Things Have Always Been Done clearly up for grabs.
See if you agree?
To be clear, I’m not talking so much about specific technologies, or how this vendor stacks up against that other one.
I’m really far more interested in the big-picture changes around fundamental assumptions as to “how storage is done” in IT shops around the globe: how it's acquired, how it's consumed, how it's managed.
If you’re not familiar with Virtual SAN, here’s what you need to know: it’s storage software built into the vSphere hypervisor. It takes the flash and disk drives inside of servers, and turns them into a shared, resilient enterprise-grade storage service that’s fast as heck. Along the way, it takes just about every assumption we've made about enterprise storage in the last 20 years and basically turns it on its head.
Storage Shouldn’t Have To Be About Big Boxes
If your organization needs a non-trivial amount of storage, you usually start by determining your requirements, evaluating vendors, selecting one, designing a specific configuration, putting your order in, taking delivery some time later, installing it and preparing it for use.
Big fun, right?
The fundamental act of simply making capacity ready to consume — from “I need” to “I have” — is usually a long, complex and often difficult process: best measured in months. I think the most challenging part is that IT shops have to figure out what they need well before actual demand shows up. Of course, this approach causes all manner of friction and inefficiency.
We’ve all just gotten used to it — that’s just the way it is, isn’t it? Sort of like endlessly sitting in morning commute traffic. We forget that there might be a better way.
The VSAN model is completely different. Going from “I need” to “I have” can be measured in days — or sometimes less.
For starters, VSAN is software — you simply license the CPUs where you want to use it. Or use it in evaluation mode for a while. The licensing model is not capacity-based, which is quite refreshing. That makes it as easy to consume as vSphere itself.
The hardware beneath VSAN is entirely up to you, within reason. Build a VSAN environment from hand-selected components if that’s your desire. Grab a ReadyNode if you’re in a hurry. Or go for something that’s packaged the ultimate in a simplified experience: EVO:RAIL. Choice is good.
Depending on your hardware model, getting more storage capacity is about as simple as ordering some new parts for your servers. Faster, easier, smaller chunks, less drama, etc. No more big boxes.
Yes, there is a short learning curve the first time someone goes about putting together a VSAN hardware configuration (sorry!), but — after that — there’s not much to talk about.
There are some obvious and not-so-obvious consequences from this storage model.
Yes, people can save money (sometimes really big $$$) by going this way. Parts is parts. We’ve seen plenty of head-to-head quotes, and sometimes the differences are substantial.
But there’s more that should be considered …
Let’s say a cool new flash drive comes out — and it looks amazing. Now, compare the time elapsed between getting that drive supported with VSAN, and getting it supported in the storage arrays you currently own.
There's a big difference in time-to-usability for any newer storage tech. And that really matters to some people.
One customer told us he likes the “fungibility” of the VSAN approach, given that clusters seem to be coming and going a lot in his world. He has an inventory of parts, and can quickly build a new cluster w/storage from his stash, tear down a cluster that isn’t being used for more parts, mix and match, etc.
Sort of like LEGOs.
Just try that with a traditional storage array.
More Performance (Or Capacity) Shouldn’t Mean A Bigger Box
Add more servers that drive more workload, and you’re often looking at the next-bigger box — and all the fun that entails: acquiring the new array, migrating all your applications, figuring out what to do with the old array, etc.
Yuck. But that’s the way it’s always been, right?
VSAN works differently.
As you add servers to support more virtualized applications, at the same time you’re also adding the potential for more storage performance and capacity. A maxed-out 64 node VSAN cluster can deliver ~7m cached 4K read IOPS.
Want more performance without adding more servers? Just add another disk controller and disk group to your existing servers, or perhaps just bigger flash devices, and you’ll get one heck of a performance bump.
Without having to call your storage vendor :)
Storage Shouldn’t Need To Be Done By Storage Professionals
There certainly are parts of the storage landscape that are difficult and arcane — and that’s where you need storage professionals. There are also an awful lot of places where a simple, easy-to-use solution will suffice quite nicely, and that’s what VSAN brings to the table.
With VSAN, storage just becomes part of what a vSphere administrator does day-to-day. No special skills required. Need a VM? Here you go: compute, network and storage. Policies drive provisioning. Nothing could really be simpler.
No real need to interact with a storage team — unless there’s something special going on.
Can't We All Just Work Together?
Any time you get a team greater than a handful of people, people split up into different roles. The classic pattern in enterprise IT infrastructure has a dedicated server team, a dedicated network team, a storage team, etc.
The vSphere admins are usually dependent on the others to do basic things like provision, troubleshoot, etc. For some reason, I’ve observed particular friction between the virtualization team and the storage team. As in people on both sides pulling their hair out.
That’s what virtualization is supposed to do — makes things far more flexible and liquid.
When that world bumps up against a traditional storage shop that thinks in terms of long planning horizons and careful change management — well, worlds collide.
With VSAN, vSphere admins can be self-sufficient for most of their day-to-day requirements. No storage expertise required. Of course, there will always be applications that can justify an external array, and the team that manages it.
It’s just that there will be less of that.
Storage Software Is Now Not Just Another Application
The idea of doing storage in software is not new. The idea of building a rich storage subsystem into a hypervisor is new. And, when you go looking, there are plenty of software storage products that run as an application, also known as a VSA or virtual storage appliance.
In this VSA world, your precious storage subsystem is now just another application. It competes for memory and CPU like all other applications, but with one exception: when it gets slow, everything that uses it also gets slow.
We’re talking about storage, remember?
And the resource requirements needed to ensure adequate storage performance using a VSA approach can be considerable. Very healthy amounts of RAM, lots of CPU. Nom, nom -- a monster VM? That approach makes your servers bigger, your virtualization consolidation ratios poorer, or both.
Once again, VSAN does things differently.
Because it’s built into the hypervisor, its resource requirements are quite reasonable. It doesn’t have to compete with other applications, because it isn’t a standalone application like a VSA is. Your servers can be smaller, your virtualization consolidation ratios better — or both.
Why do I think this will change things going forward?
Because VSAN now establishes the baseline for what you should expect to get with your hypervisor. Any vendor selling a VSA storage product as an add-on has to make a clear case as to why their storage thingie is better than what already comes built into vSphere.
Not only in justifying the extra price, but also the extra resources as well as the extra management complexity. Clearly, there are cases where this can be done, but there aren’t as many as before.
And that’s going to put a lot of pressure on the vendors who use a VSA-based approach.
The Vendor Pecking Order Changes
The last wave of storage hardware vendors were all array manufacturers — they got all the attention. In this wave, the storage component vendors are finding some new love.
As a good example, the flash vendors such as SanDisk and Micron are starting to do a great job marketing directly to VSAN customers. Why? A decent proportion of a VSAN config goes into flash, and how these devices perform affects the entire proposition.
This new-found stardom is not lost on them — especially as we start with all-flash configurations.
At one time, there was a dogfight between FC HBA vendors who wanted to attach to all the SANs that were being built. In this world, it’s the storage IO controller vendor. Avago (formerly LSI) as well as some of their newer competitors are aware that there’s a new market forming here, and realizing they can reach end users directly vs. being buried in an OEM server configuration.
There’s A Lot Going On In Storage Right Now …
We’ve seen one shift already from disk to flash — that much is clear. Interesting, but — at the end of the day, all we were really doing was replacing one kind of storage media with another.
An attractive alternative to the familiar big box arrays of yesterday.
Storage being specified, acquired, consumed, delivered and managed by the virtualization team, with far less dependence on the traditional storage team.
Storage being consumed far more conveniently than before.
Storage software embedded in the hypervisor having strong architectural advantages over other approaches.
Component vendors becoming far more important than before.
And probably a few things I forgot as well :)
Yes, I work for VMware. And VSAN is my baby.
But there’s a reason I chose this gig — I thought VMware and VSAN were going to be responsible for a lot of healthy disruptive changes in the storage business. Customers would win as a result.
And, so far, that’s been exactly the case.
Like this post? Why not subscribe via email?