Well, it's time to step back a bit from the fray, and look at what we've been able to learn since I posted my comparison last Thursday.
While not a perfect process, I think we've been able to uncover several useful nuggets that will help shape the discussion going forward.
Thanks to all who contributed their knowledge and expertise.
This Is A Hot Topic
I think this particular topic (how much of my storage array can I actually use?) surprised me in terms of the passion that it brought out in people.
The larger the environment, the more this seems to be an issue. There seemed to be a strong difference between people who had a few modest arrays, and those who were responsible for multiple data centers.
Should Vendors Be Making These Comparisons?
One school of thought was that vendors shouldn't be in the business of comparing their products to other vendors' products.
While I too would wish for an impartial comparison between storage products, no such mechanism exists today -- not regarding performance, and certainly not regarding capacity efficiency.
Even though we picked a relatively standard environment (Exchange) for which many vendors publish best practices, there's enough variability in Exchange implementations to warrant a more precise comparison mechanism.
Points Made By HP Regarding EVA
The folks at HP thought we had done them wrong by configuring too many disk groups in our sample EVA.
Upon further analysis, I believe they had a valid point.
HP published best practices for an EVA running Exchange point to two disk groups, one for the main data store, and one for the log file. We had configured 7, assuming other work would be done on the array.
As a result, we will be revising our estimates for HP EVA efficiency somewhat upwards going forward.
Points Made By EMC Regarding EVA
If an array is only doing Exchange, HP's recommendations appear reasonable. However, we at EMC were of the mind that arrays often do more than one thing.
On an EVA, disk groups are the mechanism to provide both performance isolation (apps don't interfere with each other) and availability isolation (an unrecoverable disk failure won't impact multiple applications).
More disk groups mean more performance isolation, and more availability isolation. More disk groups also mean more overhead as hot sparing is associated with a disk group, and not globally across the array.
We continue to be of the opinion that EVA customers wishing to support multiple critical applications will be strongly encouraged to either use multiple disk groups, or multiple EVAs as a result.
Points Made By NetApp Regarding FAS
NetApp claimed that the amount of space reserve for snaps was entirely arbitrary, and could be set as low, or as high, as needed depending on customer requirements.
NetApp also claimed that their customers were in the habit of keeping snaps around for a very long time, hence the need for a large snap reserve.
Left unanswered was the question of why NetApp's documentation for Exchange was very clear in strongly recommending a 100% snap reserve, and the severe consequences of running out.
Or why this was the apparent default on customer systems.
Points Made By EMC Regarding FAS
EMC claimed the reason for such high reserves on block-oriented applications stemmed from the NetApp snap implementation: unlike other implementations, running out of snap reserve means your application crashes, rather than just having your snap job fail.
When you're considering an important application like Exchange, SAP, Oracle et. al. we would think this would be an important consideration, hence the conservative recommendation from NetApp in their documentation, and the corresponding default on their systems.
It was also clear that any changes made to the NetApp-recommended defaults were largely at the customers' own risk.
As a result -- for now -- we're sticking by our original storage capacity efficiency comparison for NetApp FAS in block-mode environments such as Exchange -- until such time as NetApp publishes different recommendations.
Some questioned the "fairness" of comparing RAID 5 (used by CX and EVA) with FAS's RAID 6.
I would remind people that (a) we're talking 146GB drives here, not terabyte monsters, and (b) we simply went with what each vendor recommended.
A detailed debate as to whether RAID 5 with proactive hot sparing compares to dual-parity RAID is beyond the scope of this blog post. Let's just say that there are valid points on both sides, shall we?
Some felt that the resulting configurations should have been benchmarked to provide a standard measure of equivalence across the configurations.
I would agree, but -- alas -- we don't have the resources to do that, and -- even if we did -- I"m sure even more vociferous debate would break out if we attempted to do so.
Finally, we neglected to externally post EMC's paper on Exchange recommendations. Sorry about that -- here it is. There are some other supporting documents that complete the package as well -- I'm rounding those up and will share in the near future.
I'd invite you to take a thorough look at the EMC best practices for Exchange -- and compare this document to others supplied by other vendors. I think it'll be pretty clear to all that EMC takes this whole topic to a fairly advanced level.
One interesting thread that was uncovered in all of this was the proper use of snaps: are they short-term conveniences, or longer-term backups?
At EMC, we've never really thought of them as true backups. Sure, you can use a snap or clone to expedite the making of a backup, but it wasn't the backup itself.
Why? Simply because storage arrays -- like all computer equipment -- can have bad days. And the idea of storing your primary production data -- and all your "backups" -- on the same physical device -- well, that just doesn't seem too prudent.
But I learned in this process that -- yes -- there are snap-happy vendors who seem to be proffering this as a substitute for a safe copy of your data sitting on a separate physical device.
Best of luck to you if this is how you're doing your backups ...
What Lies Ahead?
This is an industry that enjoys a good debate. We debate performance. We debate new features. Hopefully, in the future, we'll debate storage capacity efficiency as well.
With all modern arrays, vendors tend to quote one number regarding raw capacity, and a very different number regarding usable capacity.
And, from what we've seen here, there are significant differences between what different vendors provide.
As such, hopefully we've added another dimension to the storage discussion for customers going forward: how efficient is your array?
And, if nothing else, that's a good thing.
Thanks to all who participated -- it was an eye-opening experience.