Chuck's Blog

The Split In Software Defined Storage

You just have to love enterprise IT tech.  

While much of it can be safely identified as “settled science”, interesting portions predictably morph and evolve quickly at the edges.

In my cozy little world, it’s clear that one category is showing obvious signs of splitting cleanly into two completely different beasts that share the same name — software-defined storage.

In many ways, this split has very little to do with vendors and technology — and everything to do with how customers are putting the technology to work.

Why I Think VSAN Is So Disruptive

Looking for a great disruption story in enterprise IT tech?  I think what VSAN is doing to the established storage industry deserves to be a strong candidate.  

I've seen disruptions -- small and large -- come and go.  If you're into IT infrastructure, this is one worth watching.

A few years ago, I moved from EMC to VMware on the power of that prediction.  So far, it’s played out pretty much as I had hoped it would.  There’s now clearly a new dynamic in the ~$35B storage industry, and VMware’s Virtual SAN is very emblematic of the changes that are now afoot.

There’s a lot going on here, so it’s worth sharing.  In each case, you’ll see a long-held tenent around The Way Things Have Always Been Done clearly up for grabs.

See if you agree?

VSAN, VSAs and External Arrays

I had a bit of fun last week, commenting on Nigel Poulton’s blog post discussing this topic, among others.  What I really enjoyed was the polite but vivacious discussion that emerged in the comments.

For the most part, people were disagreeing without being disagreeable.  How civil.

That being said, I tend to reflect on things after the fact.  I came to the conclusion that it was an incomplete discussion on several levels.  

If you’re not into storage stuff, perhaps this would be a good post to skip.  A friendly reminder: everything in this blog represents my personal opinion, and is not reviewed nor approved by employer.  Or anyone else for that matter.

Are Servers The New Storage?

At one time, almost all storage was directly attached to servers.  Whether that was through disk drives in the server enclosure itself, or external racks directly connected, there was a clear 1:1 mapping between server, application and dedicated storage resources.

Over the last twenty years, the pendulum has swung in the opposite direction: intelligent external storage arrays that share resources across multiple applications and servers.  I can still remember back in 1995 trying to convince people  that shared storage was better than the dedicated kind, and it certainly wasn't easy :) 

But now, there are clear signs that the pendulum has started to move back: intelligent shared storage, but without the familiar external storage array. 

And I've now been asked more than once -- are servers the new storage?

The idea of using familiar, commodity-based servers to provide shared storage services has repeatedly become popular --  and then faded -- but it's not the same this time -- the motivations are very different.

Managing Availability In A Cloud World

The public cloud proponents often wonder -- why is that the enterprise IT world is not beating a path to their door?  After all, each of these services provide great functionality coupled with an attractive consumption model.

I patiently explain to anyone who will listen: enterprise IT is different.

And one of the ways enterprise IT is very different is the critical need to manage availability.

Availability is one of those things that everyone takes for granted -- until there's a problem.  When there is a problem, a speed recovery is expected: service resumed, data restored.  It doesn't happen by magic -- there's an enormous amount of hard work that goes into creating highly available environments that protect predictably.  And never enough resources to do this right.

Most people not deeply entrenched in the topic think this must be a simple matter: a public cloud vendor simply establishes an SLA around their services, and incurs penalties if this is not met.   Case closed.

The reality is far more complex: effectively managing availability at scale is a difficult and nuanced discipline.  It's incredibly hard to get right in a traditional enterprise setting; much more so when external services are part of the landscape.

What stands out for me is that one of the core tenents of virtualization and cloud -- abstraction -- is in direct conflict with what many availabilty professionals demand: a precise knowledge of configurations and associated failure domains.

And, until we solve that one, I don't think we're going to see a lot of critical enterprise workloads using cloud resources.

Software-Defined Storage -- Where Are We?

If you're involved with virtualization or any form of IT infrastructure, you're probably paying attention to SDDC (software-defined data center) concepts, as well you should. 

It's a powerful set of ideas: by abstracting intelligence into software, the IT world becomes a better place: more agile, more controllable, and more efficient.  Infrastructure behavior will be ideally driven by the application workload at hand, and not pre-ordained.

As the three core infrastructure categories pass through the SDDC wormhole, they are inevitably altered. 

For example, how we thought about servers five years ago in the physical world is nothing like we think about them today: servers are now abstracted, dynamic, resizable, and relocatable virtual entities -- all thanks to virtualization.

Now both networking and storage are approaching the same event horizon as server and compute did years before.   And both will look very different before long.

This inevitability is not lost on industry storage vendors, either.  To date, we've heard much about software-defined storage from EMC, NetApp and HP.  We will eventually hear from HDS and IBM, and maybe even Dell.  And, of course, no shortage of small, nimble startups that smell an opportunity.

Where are the flags being planted?  Is there any consistency in the perspectives?  How do various vendor views stack up?  And what might we see in the future?

In this post, I'll vainly attempt to be an impartial industry observer.  I do admit I have my favorite horses in this race, though …

When Public Cloud Isn't Cheaper

The economics of various cloud models has been hotly debated back-and-forth for over four years.

Are public clouds cheaper? Are private clouds cheaper? Under what circumstances is one form of cloud more cost-effective than another?

I don't think we'll ever have the definitive answer, but there's some interesting new data to contemplate, courtesy of the recent EMC analyst day.

For most people doing IT at any reasonable scale, the number shows that private clouds can be significantly less expensive than public alternatives in many situations.

And that's *before* any concerns about availability, security, regulatory compliance, control, etc. are factored in.

Software-Defined Storage And The Potential For Disruption

Last weekend, two weather systems merged off the coast of New England, creating a blizzard of "historic proportions".  Each weather system wasn't particularly important by itself; but somewhat disruptive when combined. 

And it took a while to dig out :)

I believe a similar pattern is brewing in our IT world -- two trends are on a collision course; with the likely result being a significant disruption in the storage sector.

The first trend is, of course, the software-defined data center, and all its software-defined subcategories: server, network, storage, etc. 

Anything that's virtualized has demonstrated it's far easier to consume: easier to acquire, provision, and manage -- and that includes storage.

The second trend is the insatiable demand for ever more storage capacity and associated functionality.  Historically a market with healthy growth, it shows clear signs of ratcheting into an entirely new gear, fueled by demand for all sorts of big data applications.

Although this particular IT mega-storm isn't here yet, you can see what's likely to happen.  And all of us are advised to think ahead.

What I Mean When I Say "Enterprise Storage"

The recent announcement of the VMAX 10K brought up an interesting discussion for us storage folks: what -- exactly -- might "enterprise storage" be?

I find that the definitions offered by the industry analysts are incomplete and unsatisfying at best. 

And there's certainly strong motivation for any storage vendor to slap an "enterprise" label on their product in an effort to attract more potential customers.

My credentials to offer an informed perspective are substantial, but not impeccable.  I've been in the storage business now for almost twenty years.  I meet over a hundred customers and partners every year, and I've been doing that for many years.  My biases are to be expected: I've been at EMC now for 18 years, and have worked closely with the Symmetrix and VMAX product lines for virtually all of that time.

Like another infamous quote, I know an enterprise storage requirement when I see it.

Of Puppet And Razor

Cool-hunting at EMC is much more fun than it should be.  Over time, you keep finding the same people doing really cool stuff.  And there are a lot of them here.

One of our most visible cool-masters is Nick Weaver (@lynxbat).  You might know him as the guy that did all those great "uber" Celerra emulator releases. 

Or the guy who built the ultimate VMware partition alignment tool and handed it out to everyone.  Or perhaps the mad scientist behind many of the face-melting demos you've seen at EMC World.  Or a whole bunch of other cool stuff.

Well, Nick is now part of the CTO Office at EMC, which means he now has the opportunity to do even more amazing things.  That's part of the fun of working at EMC.  And, with this announcement, I think he's surpassed himself.

And, in the process, he's made EMC an essential part of the burgeoning open stack cloud and devops discussion.