« CMIS -- It's Not JAS | Main | Big Visions From VMware »

September 12, 2008

Comments

Lee Razo

Hi Chuck,

NetApp guy here - courteous comment to follow:

I think at the end of the day, the whole NetApp discussion is going to amount to one about documentation. While I agree that that is an important topic, if that and a couple of specifically selected emails from anonymous senders is all you got against us, well shoot - I don't see what all the fuss is about either.

--Lee

iStorage

I've been using NetApp filers in both iSCSI and FCP (SAN) modes for a few years now. Initially we had to use 100% space reservations, but with the recent releases of DataONTAP, SnapDrive & SnapManager we've easily been able to scale that down to 0%.

However, I see NetApp space efficiency in an entirely different way. It's not how much water I can fill in the bucket, it's how many usable buckets I can get by buying just one.

In that regard, Curtis Preston is bang on. With NetApp, I can use 10's of buckets and still only buy one. With other arrays like CX or EVA, I'm still trying to make the most out of just the one bucket.

Regarding "conduct", I say you reap what you sow. If you call someone out in your blog (and in your comments you did just that Chuck), then expect a response and be man enough to live with it.

I still think filtering comments is weak. I would let them fly and have the public measure for themselves whether the commenter is enhancing or diminishing their own credibility by their own words.

Dave McDonald

Chuck,

I know you want to vindicate your position about NetApp's space efficiency against the opinions of an independent expert and their CTO, but this whole exercise is futile.

Reagrdless what you can produce via private emails and chosen passages from their texts, who are you trying to convince?

If I want to understand NetApp, I'll go ask them about their functionality, not EMC. If I need to double-check, I'll go to a 3rd party, not EMC.

There's no way for you to settle this, so why do you persist?

Chuck Hollis

You're absolutely right -- there is no way to settle this, and -- to be sure -- that is not my goal.

However, I found many of the responses interesting in context, and wanted to share.

Just for the record, the individual in question is not NetApp's CTO, he seems to use a variety of titles in different situations. And the "independent expert" in this case was the first to admit he had limited expertise in this area.

I think someone else here said something to the effect of: if you had a choice of listening to vendors, experts or users -- choose the users -- which I did.

Chuck Hollis

Thanks for commenting, iStorage

Please forgive my cynicism, but your comment sounds like it might have come from a NetApp employee.

Not that this would be the first time; those of us who blog at EMC routinely field comments from NetApp IP addresses pretending to be "users".

If this is not the case with you, my apologies in advance.

I'm curious though -- what types of application might you be running on a mix of both iSCSI and FCP? Usually it's one or the other in a given shop, and very rarely both.

Thanks!

Alex McDonald

Chuck

Martin G's requirement of 400TB (base 2) is fulfilled with usable capacity of *greater than* 50% of the drives -- not less than 50% as you state.

1. 400TB in base 2 is 429.5TB in base 10
2. 1TB drives (which are always quoted in base 10) have less than 1TB addressable bytes; around 977GB (base 10)

That's approximately 440 drives worth of data, *not* 400 drives. A 10% difference is significant.


I will be blogging this coming week on NetApp figures collected from user data. Much more accurate and verifiable, and much more preferable over anecdotal and erroneously calculated numbers.

Chuck Hollis

Hi Alex

Yes, that will be interesting to see how NetApp positions this with "official" user data, and not run the risk of being accused of gaming the results by picking selected data points, rather than representative averages.

My take?

I think you guys know you've got a problem with this issue. Part of it is attributable to your design, part of it is attributable to the recommendations you give your customers.

If nothing else, I bet this is an active discussion within your engineering community now ...

Cheers!

-- Chuck

Alex McDonald

Chuck

The data will be drawn from the same set of data used in our research papers; for example http://www.ssrc.ucsc.edu/Papers/leung-usenix08.pdf higlighted by Robin Harris over at StorageMojo last week (http://storagemojo.com/2008/09/09/our-changing-file-workloads/)

Unlike the authors of that paper, I am not a researcher and I am not a statistician, so my work based on the same data (and it will be *my* work) won't have the professional polish or peer review of an academic paper.

But neither am I a liar.

I resent the suggestion that I would "game the results". You have been quick to point out the unacceptable nature of what your perceive to be attacks on you personally. Given that, I would suggest that you need to consider what you have said here very carefully.

I look forward to your comments on my work in the coming weeks. Courteous ones.

Chuck Hollis

Alex

You need to go carefully re-read what I wrote.

I said that you might be accused of gaming the results by others.

This is the same problem that any vendor faces when presenting "independent" research.

It is not limited to NetApp, and it is not a personal attack.

Good luck!

Martin G

Alex,
sorry you are right and it was me who misled Chuck; in my defence, I would say that the calculation that I got from *NetApp* was not especially clear. I've gone back and checked it.

The calculation actually left out the 10% Snap reserve, so yes I would get 440 spindles. So apologies all round! I had asked for a quote for 400 Terabytes useable, unfortunately I got a quote for 400 Terabytes+10% snap-reserve.

Now, just to upset the apple-cart and in fair disclosure; our Exchange 2003 environment currently sits on DMX-3 146 Gig drives in a RAID-1 format. Why, because Microsoft and EMC told us so. Throw the protected BCVs into the mix and our utilisation in this environment is terrible!!! It drives me to distraction that Microsoft could have written an application which handles i/o so terribly!!! The sooner our Mail guys go to Exchange 2007, the better!!

Steve Klinkner

Chuck said

"If nothing else, I bet this is an active discussion within your engineering community now ..."

There's some truth to that at the surface level.

However the discussions are focused on correcting mis-representations in public forums rather than on product deficiencies.

From a NetApp engineer's perspective: the follow-ups have been enlightening, but with all due courtesy the root cause (marketing on a soap box) is *technically* non-interesting.

W. Curtis Preston

Yes, I followed all that, although the arguments are starting to lose my interest.

First, let me say that I think you have a point, although I don't think it's the point you're making. NetApp's docs seem to be pretty solid on 100% snap reserve. NetApp customers that I've worked with seem happy with something significantly less than that. (Which is why I don't agree with the point you are trying to make, that you HAVE to have 100%.) If NetApp is OK with less than 100%, why is it not documented? We'll see what changes come in the docs.

I still disagree that this means that EMC arrays are more space-efficient than NetApp arrays. I believe that how people USE their NetApp arrays takes up more disk than how people USE their EMC arrays. That's not the same thing.

A few issues...

The user you're quoting says they're a NetApp customer, but they don't mention anything from their own experience. All they do is quote NetApp manuals, which is the same thing you're doing. His/her comment would hold a lot more weight for me if they spoke from their experience, not from the docs.

You say you've never launched a personal attack, then you put "industry expert" in quotes and imply that Val is making up his title. How are those not personal attacks?

As it says on snia.org and communities.netapp.com, Val is in the Office of the CTO. He is in charge of competitive positioning. If that translates into CTO-at-large, so be it. But to imply that he's making up his title is a personal attack and is unfair.

As to putting "industry expert" in quotes, the only reason for doing that is to imply that I'm not an expert. Since my entire livelihood comes from BEING an expert, I consider that a personal attack. See? It's all about the vantage point.

If 15 years of doing nothing but backup and recovery using just about every product imaginable, three books, and a website with thousands of registered users doesn't qualify me as an expert, I'm not sure what does.

As to Val's blog entry that you considered a personal attack, EMC bloggers bring it on themselves by censoring comments. Uncensored blog comments allows the reader to determine who is full of it and who is not. Censored blog comments make the blogger look like they're not fighting fair. Google "censoring blog comments" to see what the blogosphere thinks about it. It's always associated with a lack of ethics. If you don't want to be accused of it, then don't do it.

iStorage

Hi Chuck,

I'll forgive your unfortunate "un-courteous" accusation that a former EMC customer must somehow be a NetApp employee impersonating someone else ... and I'll leave a you "courteous" comment instead so perhaps you can follow what you preach.

The use-case for a truly unified storage system (such as NetApp's) is an app development and VM certification environment. Our developers or VM infrastructure qualifiers are connected over iSCSI for low cost and flexibility.

When their apps and/or VM's are finished a testing or certification cycle (as the case may be) we promote them over to FCP (NetApp's name for FC-SAN). For small apps we do this in place with no LUN migration to get to production, and for larger apps we (snap)mirror and (flex)clone to a production filer over FCP for fully isolated and predictable performance.

We tried this with both Celerra NS systems as well as CLARiiON CX3's and it was far too cumbersome to work. I haven't seen anything in FLARE28 on CX4's to indicate this kind of dual block protocol integration has improved, but perhaps you can enlighten me?

Chuck Hollis

Curtis, I appreciate your comments.

However, I don't believe in crowd-sourcing personal ethics and values.

I was raised to expect certain standards of behavior and conduct from civilized people, and to express my disapproval when their interpersonal conduct doesn't meet my standards.

Call me an elitist (or any other name you choose), unless people draw lines, all of our values tend to degrade to zero.

I'll allow that I've made a snide comment or two on our NetApp friend's excessive posturing on titles and roles, but I think that the multiple blog posts from him and others that essentially attempt to collectively trash us as individuals -- and our company as whole -- well, I don't think I have to endorse that sort of conduct.

And, frankly, you shouldn't either.

I'm not the only one who's experiencing this -- if you're interested, go watch the interaction on virtualgeek.typepad.com

Different players, same pattern.

It's ugly, and none of us should condone it.

I don't know if you've ever had someone cross the line on your blog, but -- at some point -- you may feel the need to take a stand and say "enough".

And, if you ever encounter this sutations as I have, I for one will understand your decision to do so.

Chuck Hollis

My apologies, iStorage -- we've got a ton of NetApp employees trying to spoof us six ways to Sunday these days, so please accept my very humble apologies.

Now that you describe your use case, yeah, I get it, it makes much more sense why you use NAS, iSCSI and FCP all at the same time -- you're developing software and need to qualify in a variety of environments, and -- frankly -- the NetApp device would make a great test bed for your use case, perhaps better than what EMC could offer in this case.

But, you'll have to admit, your case is a bit different than most production environments, yes?

And, not to offend, but I would tend to guess that you probably don't do a lot of performance envelope testing in your environment?

Thanks for taking the time to set me straight -- cheers!

W. Curtis Preston

(Had to look up crowd-sourcing...) So if I understand you correctly, you're saying you don't care if many of the readers of your blog think you're being a (insert expletive here). You're going to do it anyway. Interesting. ;)

BTW, I haven't experienced someone going to far in my comments because I'm usually not making blog entries that make then want to say things like that. (You know, like saying that their product is crap.) ;)

Hmmm.... Not wanting to take a side here...

I don't believe in competitors bashing each other. I think it's unprofessional. Tell me what's great about your product, not what sucks about theirs. I (and other customers I've worked with) have taken points off in the past when a vendor starts slamming the other guy.

One of the biggest reasons that this is the case is that (the proverbial) YOU DON'T KNOW THEIR PRODUCT; YOU KNOW YOURS. Competitive info like that is almost always WRONG. Until you've actually used their product, don't tell me how it works.

So I don't condone what either of you are doing.

Chuck Hollis

Curtis --

Your first observation is correct. I'm not trying to win the November election here.

For me, a key blogging concept is individuality, authenticity and transparency -- warts and all. I'm not pretending to be someone I'm not.

Put differently, if someone doesn't agree with my views on certain issues, they're free to go on to the next item in their RSS reader.

As far as vendor behavior, points noted -- vendors should (ideally) talk about what they do best, and not slam the competition. No problem there.

If we only lived in an ideal world ...

But what happens when one vendor believes another vendor is intentionally misrepresenting their capabilities? And doing so in what appears to be a consistent and intentional pattern?

Who has the responsibility of calling them out for misleading and/or incorrect claims?

Or, if you personally thought that some vendor was being slightly unethical in their representations, would you call their product cr*p for a particular use case?

If you've got an answer to that one, I'm game ...

-- Chuck

Craig Simpson

Chuck, about HP’s SPC-2 result (http://www.communities.hp.com/online/blogs/datastorage/archive/2008/09/08/a-new-world-record-for-xp.aspx) : No, most won’t use an XP24000 just for video streaming. But lot’s of folks use them for storage consolidation. Might that include some data mining? Yep. Might that include some video clips in this day and age? Yep. Might it all need to be backed up? Yep. Might it include a whole bunch of transaction processing, for which SPC-1 is the standard benchmark? Yep. Does XP have top notch performance and commensurate benchmark results for all cases? Yep. Does EMC’s DMX have ANY benchmark results? NO!!!!

We challenge you, whether it’s SPC-1, SPC-2, or even capacity utilization, to submit to an independent third party test. Until you do I think everybody can see that if DMX could put up a good SPC-1 or -2 benchmark number you’d do it. No amount of smoke screen about relevance can hide that.

Chuck Hollis

Hi Craig

You seem like a nice guy, so let me explain.

All of us here are very familiar with the UPS-V aka 24000.

Go take a hard look at the configuration you benchmarked. Can you honestly say that anyone would buy one that way? 128 146GB drive pairs in your 1152 drive box? A pure video streaming workload?

Help me here -- what's the point of all of this? What are we proving here?

I'm pretty good at positioning things, but I would have a tough time explaining to a customer what this particular test proved or disproved, especially in the context of a specific problem or concern.

I was discussing this fascination that some vendors have with the SPC the other day with a customer, and he had something very insightful to say, "market leaders point to market share, market laggards point to benchmarks".

I don't know if I entirely agree with that sentiment, but it made me think.

-- Chuck

Alex McDonald

Time at ShadeOfBlue Towers is in short supply, but I eventually managed to start the riposte to your capacity assertions;

http://blogs.netapp.com/shadeofblue/2008/09/space-is-mind-b.html

More to come, if I can get out from underneath a pile of stuff on my desk...

Chuck Hollis

Actually, I've kind of lost interest in this one -- there seems to be so much variability in use cases, vendor recommendations, tradeoffs, etc.

I think that we both can agree that -- in the long term -- customers may be taking a harder look at effective utilization of storage assets, much as they have with server assets.

Perhaps we can pick a topic of mutual interest that's worth debating to our mutual audiences?

I've got a few ideas, if you're game ...

Craig Simpson

Chuck – First the point. Pretty much everybody who buys an XP (or a DMX) has to back up the data on it with time constraints on getting those backups done. That’s a sequential workload represented by SPC-2. Lot’s of people use XP’s for data mining operations. I can’t speak for whether or not people use DMX’s that way. But SPC-2 represents data mining as well. A good number of folks use XP’s for some serious number crunching, also represented by SPC-2. Lots of folks use XP’s for transaction processing represented by SPC-1. The mix varies a lot from customer to customer. With both SPC-1 and SPC-2 numbers available people can get a good relative understanding of how an XP will perform on these portions of their workload based on standard, independently audited tests. No shenanigans, just useful, objective data that’s applicable to every use of a high end array.

Second, you seemed to think the SPC-2 config was pricey. The XP is a high end array. Those are pricey whether they’re ours or yours. In a lot of cases the bullet proof availability, blazing fast single array performance, and ability to consolidate many workloads in one place are a must, justifying the price. That’s why there’s a market for high end arrays, ours or yours.

Now to your question about buying an XP like that for video streaming. There are folks out there with huge amounts of video content who need blazing fast, bullet proof access to some of that content. Based on their business results they’re pretty sharp people. Can I honestly say anybody would buy an XP for that? They have!

Chuck Hollis

Hi Craig -- thanks for the response.

Now, seriously, how many people would take an array designed for 1152 drives, lightly populate it with 128 pairs of 146 GB 15k drives, and run a single sequential workload against it?

And how many vendors would even suggest that's a "representative configuration" to someone who knows better?

My dad always told me, when you find yourself in a deep hole, stop digging!

-- Chuck

Craig Simpson

Certainly the workload and configuration are in use for our products. I can’t speak for yours.

You seem to be hung up on some “representative” config or workload. What I observe out there is a pretty good diversity in customers’ specific needs. We choose to provide objective information that can span a wide range of needs. You don’t.

Chuck Hollis

Hi Craig -- thanks for the response, I think.

Maybe we look at this benchmarking thing fundamentally differently. I can't explain it any other way.

When we at EMC say "benchmark", we mean "representative workload, representative configuration" which we usually summarize as "use case".

In this case, I'd agree that "representative workload" for the SPC-2 is pretty clear -- it's sequential streaming (no writes, just multiple data streams). No argument here.

Where I have a major beef is in the "representative configuration" department.

I think most of the readers here would recognize that a full-blown USP-V aka 24000 with a full complement of controllers and cache, capable of supporting 1152, but very lightly populated with 128 pairs of mirrored 146GB 15K drives doesn't really satisfy the "representative configuration" for this workload.

So, step back a minute.

I'm imagining a customer that we're both calling on, and the HP guy comes in with their $1.6m proposal for 18TB usable, and the rest of the competition comes in for anywhere between $250k and $600k -- and offers roughly equivalent performance and availability.

That's where this approach fails, Craig.

Your benchmark testing doesn't reflect what happens in the real world, does it?

Sure, you provide data. It's just not useful data, is it? Do you expect customers to make informed buying decisions based on these results? Seriously consider a USP-V / 24000 for this specific workload?

How does that work?

I mean, HP published the benchmark, so it implies that HP believes it's useful, valuable information to help customers make informed buying decisions, since HP doesn't believe in cheap marketing stunts, right?

And that's where I think the two companies philisophically disagree.

I'm just waiting for you to see the light here ...

The comments to this entry are closed.

Chuck Hollis


  • Chuck Hollis
    SVP, Oracle Converged Infrastructure Systems
    @chuckhollis

    Chuck now works for Oracle, and is now deeply embroiled in IT infrastructure.

    Previously, he was with VMware for 2 years, and EMC for 18 years before that, most of them great.

    He enjoys speaking to customer and industry audiences about a variety of technology topics, and -- of course -- enjoys blogging.

    Chuck lives in Vero Beach, FL with his wife and four dogs when he's not traveling. In his spare time, Chuck is working on his second career as an aging rock musician.

    Warning: do not ever buy him a drink when there is a piano nearby.

    Note: these are my personal views, and aren't reviewed or approved by my employer.
Enter your Email:
Preview | Powered by FeedBlitz

General Housekeeping

  • Frequency of Updates
    I try and write something new 1-2 times per week; less if I'm travelling, more if I'm in the office. Hopefully you'll find the frequency about right!
  • Comments and Feedback
    All courteous comments welcome. TypePad occasionally puts comments into the spam folder, but I'll fish them out. Thanks!