« Considering The Next Wave Of Storage Automation | Main | Ten Reasons Why VMware Is Leading The Hyperconverged Industry »

March 11, 2015

Comments

Vaughn Stewart

== Disclaimer: Pure Storage Employee ==

Chuck,

Hyper-converged infrastructures like VSAN (and the prepackaged version EVO:RAIL) seem ideal in addressing market needs where even the smallest storage array is a finanical burden - ala branch offices, retail locations, militarized vehicles, etc.

As hyper-converged systems are comprised of a shared-nothing storage architecture they inherently laden by low storage utilization. How can you position VSAN or EVO:RAIL for enterprise-scale deployments? Doesn't their ~20% storage utilization disqualify them from AFAs that deliver greater than 300% storage utilization?

I graphed out the utilization of the EMC VSPEX BLUE solution here: https://virtualstorageguy.com/2015/03/12/hyper-converged-infrastructures-are-not-storage-arrays/

To reiterate, I think HCI addresses significant market needs - I just disagree with the notion that HCI can address enterprise storage needs. Customers wrestle with data center resource scarcity as much as they do with storage challenges.

I look forward to your reply.

-- cheer,
v

Chuck Hollis

Hi Vaughn

We disagree on fundamental points here.

First, you're trying to position hyperconverged infrastructure as limited to small scale: remote offices and the like. While it obviously is a good fit there, there's also real customer evidence that it's a great fit in the data center as well.

Second, I think you're trying to make it all about storage efficiency. For some shops, that will be one concern among many. For most, "people efficiency" is far more important.

These IT leaders look at things differently: how efficient are my people in delivering the services that the business requires? That's where products like VSAN and EVO:RAIL shine.

In addition to "price per usable", there are strong advantages over dedicated external storage arrays that have to be procured separately, managed separately, supported separately, etc. Just like many of us prefer using a single mobile device over multiples.

I love the marketing claim of "300%". Go big, or go home!!! Even though we both know that actual results might vary significantly :)

Can HCI storage solutions go toe-to-toe feature-for-feature with the best arrays out there? No, not in some aspects -- but uniquely strong in others. And I think you'll find the gap closing fast indeed.

Let's not forget that the hardware in a VSAN implementation is the same low-cost commodity hardware they're using in their servers. Even with an array's moderate abilities to dedupe, the VSAN proposals often come out less expensive thanks to how parts are priced in the open market.

Stepping back, if you think back to how hypervisors found their way into data centers, we're seeing the same process today, only vastly accelerated. Back then, there were valid cases where physical made sense. Far fewer of those today, no?

I know where all those VSAN implementations are. And if you're trying to convince yourself that a good portion aren't in data centers supporting demanding applications, you'd be mistaken.

Best regards

-- Chuck

Vaughn Stewart

Chuck,

I agree we view the world with different lens... which is a good thing. Diverse perspectives are required to address diverse needs.

Thanks for posting my comments and sharing your perspective.

-- cheers,
v

Mark Burgess

Hi Chuck,

I personally feel that VMware needs to be careful to not over hype the technology.

In the right use case I am sure it makes a lot of sense, but at the end of the day it is not a mature solution yet as it has only been available for a year.

It is important that customers are aware of some of the limitations:

1. Usable to raw capacity is poor
2. Double disk protection, a requirement for many, is not really viable
3. You need to licence every node in the cluster even if it does not use VSAN
4. No redundancy for the cache drive - if it fails the entire disk group goes down - not ideal
5. ...

I think the biggest issue the storage industry has is over inflated list prices - this is what causes many of the problems listed above (i.e. slow procurement process).

If the storage array vendors can sort out their list prices, let's hope VSAN forces them to do this, and addresses the ease of use issues by making converged stacks available that are built on simple commodity servers rather than UCS then I think it will be an interesting battle.

I am sure there is a significant place for VSAN moving forward but I still think a modernised storage array industry has a lot going for it - on the other hand if it continues to make things complex and insists on outrageous list prices then it is in trouble.

As always your comments would be appreciated.

Best regards
Mark

Chuck Hollis

Hi Mark

You've got your facts wildly incorrect, so let me see if I can help you?

1. If you use a policy to protect against a single failure, roughly 50% of raw space is available for consumption. That's essentially RAID 1. The good news is that VSAN does this with low-cost server disk drives. Go price out a usable config, compare $/usable to any array, and you'll see what I mean. Using RAID 1 also has advantages on read performance and rebuilds.

2. Protection level is set by policy on a per-VMDK basis if needed, using FTT, or failures to tolerate. FTT levels from 0 to 3 are supported, so that would protect against double (and even triple!) disk failures. Again, users can pick and choose which VMDKs get which protection level, or none if that's indicated.

3. Yes, you license every node in a cluster, because ideally every node is both producing and consuming storage services. Other licensing approaches can be debated, but this approach got the nod from our customers as being simple and consistent with other things they do.

4. The level of cache protection (hence redundancy) is determined through on a per-object basis using the FTT (failures to tolerate) setting: none, one, two, three etc. If a cache drive fails, it affects all the capacity devices behind it, usually 1-4. If FTT>0, production continues and a rebuild of the failed components begins.

Having come from the array business, this is an entirely reasonable failure domain.

I will agree with you, though, that the lack of transparent pricing in the array business doesn't do customers any favors. In the array vendor's defense, though, component prices change downward frequently, and there are dozens of variable that go into pricing a given configuration.


Mark Burgess

Hi Chuck,

I do not think it is fair to say I have got my facts wildly incorrect - I have spent a considerable amount of time studying VSAN 5.5 and 6.0 over the last year.

1. If you use a policy to protect against a single failure, roughly 50% of raw space is available for consumption. That's essentially RAID 1. The good news is that VSAN does this with low-cost server disk drives. Go price out a usable config, compare $/usable to any array, and you'll see what I mean. Using RAID 1 also has advantages on read performance and rebuilds.
>>>> 50% usable capacity is not competitive today, take a look at products like XtremIO they use a 23+2 RAID Group which will eventually also enable double disk protection - NetApp FAS would also be very similar, and it has double disk protection today

I appreciate what you are saying about low-cost drives, but in order to compete with products like EMC VNXe or NetApp E-Series in the low/mid market the drives would need to be at a negative cost:

4 nodes each with 2 CPUs for VSAN = 8x$2,495 = $19,960
Annual maintenance (20%) = $3,992
3 year cost = $31,936

You could get a very nice E-Series or VNXe with a considerable amount of storage for this amount of money whereas with VSAN this is software only - I appreciate that as you progress beyond year 5 VSAN will become more competitive as at year 5 you will probably replace the entire array whereas you will just pay maintenance on VSAN and replace the hardware

2. Protection level is set by policy on a per-VMDK basis if needed, using FTT, or failures to tolerate. FTT levels from 0 to 3 are supported, so that would protect against double (and even triple!) disk failures. Again, users can pick and choose which VMDKs get which protection level, or none if that's indicated.
>>>> I am aware you can have double disk or triple disk protection but it does not look like something you would want to use in the real world as it would use up more CPU resources and reduce usable capacity to around one third/quarter - server disks are cheap, but not that cheap

3. Yes, you license every node in a cluster, because ideally every node is both producing and consuming storage services. Other licensing approaches can be debated, but this approach got the nod from our customers as being simple and consistent with other things they do.
>>>> Why not just license the nodes that hold the storage? I think if you ask most customers that is what they would ask for - with this model you would address the problem that storage and compute do not scale linearly, the idea of a 20 node cluster with 4 dedicated VSAN nodes looks very attractive to me

4. The level of cache protection (hence redundancy) is determined through on a per-object basis using the FTT (failures to tolerate) setting: none, one, two, three etc. If a cache drive fails, it affects all the capacity devices behind it, usually 1-4. If FTT>0, production continues and a rebuild of the failed components begins.

Having come from the array business, this is an entirely reasonable failure domain.
>>>> I really do not see how you can defend an architecture whereby the failure of one drive (in a 1+7 disk group) takes all 8 drives offline - surely it must be on the the roadmap to resolve this so that you can have multiple cache drives to both scale performance (another limitation today) and local redundancy

I may come across as being anti VSAN and pro storage arrays, but this is not true I am just trying to objectively determine the strengths and weaknesses of each and at this stage I would conclude:

1. VSAN is too expensive in a lot of scenarios - it will make the most sense when you have a small number of nodes and either a lot of capacity or a lot of IOPS, everything in between it will struggle with
2. VSAN is immature and unproven compared to storage arrays (but we will not be able to say that in a few years)
3. It has some architectural limitations compared to storage arrays (no parity RAID or Erasure coding provides poor capacity utilisation and a caching architecture that on failure of a single drive fails up to 7 other drives)
4. It's simplicity and linear scaling of compute and storage undoubtedly will fit some use cases today
5. The traditional storage array vendors need to move to more realistic street prices otherwise they will continue to "shoot themselves in the foot" and drive customers to the start-ups and hyper-converged products like VSAN
8. Storage arrays need to move to erasure coding as RAID is now looking "a bit long in the tooth"

Just my opinion and as always the market will decide so it will be interesting to observe what happens over the next few years.

More thoughts at https://blog.snsltd.co.uk/an-introduction-to-vmware-virtual-san-software-defined-storage-technology/

Out of interest why do you think the storage industry cannot let go of very high list prices relative to street price - are there other industries that work this way in your experience?

Best regards
Mark

Chuck Hollis

Mark, you still have your facts a bit messed up. It sounds like you're trying to skew your subjective observations. I'll try once more with real facts, but I'm not hopeful here.

1. The prices you are quoting for software are list prices. Many customers already have volume licensing arrangements. Second, I know what E-Series and VNXe arrays price at. Go ahead and do a head-to-head usable GB comparison.

I should also point out that protection is defined on a per-object basis. Depending on your needs, some object classes could have zero protection, be able to tolerate a single failure, be able to tolerate two failures, etc.

Also, keep in mind when you sign up for one of those entry-level arrays, your future options are limited around expansion, hardware upgrades, etc. Not the case with VSAN.

2. VSAN is designed to use no more than 10% of host CPU, frequently much less. That's true regardless of protection mode selected. A Seagate 4TB SAS drive goes for $269 these days online. Yes, Mark, server drives *are* that inexpensive! How much do you think that exact same drive would cost from an enterprise array vendor?

3. Regarding your "ideal" configuration, you're still thinking like a traditional storage array dude. An evenly balanced cluster is just that -- evenly balanced. Great performance, minimal CPU and memory impact. While the product certainly supports creating unbalanced configurations, it would be a more complex environment to manage, and the main motivation here is simplicity.

Licensing is licensing. We went with a licensing scheme that was simple and consistent with what vSphere customers are doing today. To date, neither pricing nor licensing has been a barrier to customer adoption.

4. I think you're mistaken about how VSAN failure domains work. Yes, the failure of a cache device will render all the capacity devices behind it unreachable. That is the same behavior as a cache controller failing in a storage array. A given server can support multiple storage controllers and/or multiple cache devices if desired. So if you wanted to have 4+ small cache devices in a server, each with a modest amount of capacity behind it, that's an option.

Yes, that costs more -- but the flexibility is there.

As you your other subjective personal opinions: VSAN is too expensive, immature, architecturally lacking key features, etc. -- you are more than welcome to your perspectives, but none of these are objective facts.

I think your biggest misperception is that VSAN is directly competing with traditional storage arrays. That's not really the case. I could make a long list of things that storage arrays do that VSAN does not. And, conversely, I could make a long list of things that VSAN does that storage arrays do not.

VSAN is designed for vSphere administrators who want a storage product that works the way they do. Which is why it's not a storage array.

-- Chuck

Mark Burgess

Hi Chuck,

Thanks for all the feedback - much appreciated.

As I said I am not in any way anti VSAN,I am just not convinced that we will see a large percentage of vSphere customers adopting the technology for a significant proportion of their storage estate - only time will tell and I might well be wrong!!!

I also think VSAN has come a long way considering it has only been on the market for a year and I am sure there will be major improvements over the next few years (i.e. de-dupe, erasure coding, multiple cache drives).

At the end of the day it is much easier for me to have an objective opinion as I do not work for a vendor, but I do agree I am most comfortable with a storage array so this is always going to be my frame of reference at least for the next few years.

Best regards
Mark

Mario

Oh, Chuck, you can save my life. Tell me about what VSAN offers that traditional storage provider don't do. Regards !!!

The comments to this entry are closed.

Chuck Hollis


  • Chuck Hollis
    SVP, Oracle Converged Infrastructure Systems
    @chuckhollis

    Chuck now works for Oracle, and is now deeply embroiled in IT infrastructure.

    Previously, he was with VMware for 2 years, and EMC for 18 years before that, most of them great.

    He enjoys speaking to customer and industry audiences about a variety of technology topics, and -- of course -- enjoys blogging.

    Chuck lives in Vero Beach, FL with his wife and four dogs when he's not traveling. In his spare time, Chuck is working on his second career as an aging rock musician.

    Warning: do not ever buy him a drink when there is a piano nearby.

    Note: these are my personal views, and aren't reviewed or approved by my employer.
Enter your Email:
Preview | Powered by FeedBlitz

General Housekeeping

  • Frequency of Updates
    I try and write something new 1-2 times per week; less if I'm travelling, more if I'm in the office. Hopefully you'll find the frequency about right!
  • Comments and Feedback
    All courteous comments welcome. TypePad occasionally puts comments into the spam folder, but I'll fish them out. Thanks!