Anyone in the IT infrastructure business is fully aware that flash memory is redefining how we think about performance, and not just storage itself.
With most applications, you're usually performance limited by the physical challenge of getting data into (and out of) the processor; shrinking I/O latency from milliseconds to microseconds means a huge boon for anyone concerned about cost-efficient performance.
As a result, flash is redrawing the storage landscape. Flash has found its way into traditional storage arrays (e.g. EMC's FAST), server-side caching (e.g. EMC's VFcache), and will soon be available as caching appliance, e.g. EMC's Project Thunder.
But all of these variations have a buried assumption: they assume that there's traditional disk-based storage (aka "rotating rust") somewhere in the mix.
What if you built a storage array entirely out of flash? And, in doing so, were able to free yourself from twenty years of legacy storage approaches as a result?
That bright, shiny thought is what makes XtremIO's approach to block storage so compelling.
As a reminder, EMC announced the acquisition of XtremIO in May of this year. We went sort of quiet about the technology at EMC World a few weeks later, only mentioning "Project X" in passing. But as we get closer to announcing an offical EMC product based on XtremIO technology, people are justifiably curious as to what might make it so special.
Especially as there seems to be a lot of interest in the topic lately :)
Standard disclaimer: the statements here are based on XtremIO's technology prior to EMC's acquisition. Don't take this a commitment that all of these capabilities may find their way into an eventual EMC product. No, I don't have a date for an official announcement. If you'd like more details, or perhaps participate in a beta of some sort, please contact your EMC representative.
Flash Is Disruptive
Much has been written in the technology industry about big disruptions that come along, and how various established vendors react to them. Here at EMC, we've learned to be almost paranoid about every new development in the industry.
If there's going to be a disruption, we'd much rather be the disruptor rather than the disruptee.
We tend to think along two lines. First, how can we use the new technology to benefit people who are currently invested in the predecessor technology? Second, how can we create entirely new products that maximize the new technology's benefits, and aren't so concerned with bridging old and new?
Products like EMC FAST fall into the first category; products like XtremIO clearly fall into the second.
The (Hardware) Basics
At first glance, the core hardware doesn't seem all that unique: dual Intel-based controllers in each X-brick scaling unit along with a shelf of flash drives, 2 host adaptors with 2 ports each (supporting FC and iSCSI), and Infiniband connecting the modules together in a scale-out manner.
One thing you'll note is that there's a decidedly heavy processor complement: currently two multi-core Intel Westmeres (6 cores each) per controller, yielding a robust 24 cores across four processors per X-Brick.
If you're not familiar with storage controller designs, that's much more than the norm for this class of device -- and you'll see why that is in a moment.
But once you get past the CPU-heavy design, it's basically all commodity stuff -- not much to get overly excited about yet. As modules get added, things scale very linearly as you'd expect: both capacity and performance, as we'll see.
The real magic, as we'll see, is in the software.
XIOS: The XtremIO Storage Operating System
There's a lot to like in how XIOS uses the hardware resources.
At the core, everything is based on 4K blocks. Groups of blocks are composed to present the familiar volumes, LUNs, and snap devices, although a given physical storage block might be presented in multiple logical volumes.
Every new block written is analyzed, and hashed in real-time.
The resulting content-based signature is very useful in two important ways.
First, it helps determine a randomized static I/O location across available physical devices. This ensures that all storage devices are read from (and written to) equally. As you might surmise, the randomization helps boost performance, increases utilization of available media, as well as provides even wear characteristics across the media pool.
And -- more importantly -- the hashed content address enables the X-Brick to do global inline data deduplication.
Yes, the XtremIO array does data deduplication, and does so at ferocious speeds -- thanks to that heavy processor configuration.
The array delivers a predictable sub-millisecond I/O response time for every 4K block no matter what you happen to be doing: read, write, sequential, random, snaps, etc.
All fully deduped across the entire capacity -- only unique blocks are ever stored on the XtremIO array. Here's the point: global dedupe is built in, not bolted on. It's not something you turn on and off, it's integral to the design.
And that wonderful underlying architecture turns out to have some pretty interesting implications.
Rethinking Performance
First and foremost, this box is mind-bendingly fast. More importantly, it's predictably fast in every situation. You're not depending on the efficacy of a caching or tiering algorithm or predicting what your IO profile might look like.
You won't hear "well, it depends" ...
All data lives on flash. There's no need for data movement to adjust performance. All reads and writes are evenly distributed across available hardware resources. Each and every time. Snaps and primary volumes run equally fast.
For many use cases, that's exactly what the doctor ordered.
This also means that arbitrarily large I/O numbers can be achieved by simply adding more modules, much like Isilon does for file access.
The formerly big number of a meelyun IOPS can result from a very modest configuration of XtremIO modules. And that's with today's components.
It also achieves this predictable performance without delaying IOs or buffering the heavy lifting. Data is always written to its final resting place; there's no need to come back and reshuffle or reorganize written data. Dedupe is done inline, there's no asynchronous compression process to think about.
Run it red-hot 24x7 -- no worries.
Rethinking Capacity Efficiency
With many current storage approaches, you can have efficiency or performance, but not both.
Not the case here, as the XtremIO array only stores unique 4k data blocks.
Run it nearly full, it performs the same. Thin provisioning and space-efficient snaps? Sure, of course -- only unique blocks are stored.
And here's a potential head-twister: the more compressible your data is; the more performance you'll see -- not to mention being more space efficient.
VDI, anyone?
Putting The Pieces Together
A legacy-free storage design. A contemporary scale-out block architecture based on commodity components. Fast and predictable flash performance while delivering unmatched storage efficiency. A simple and intuitive array management interface.
That's what you can see today.
Now, take that technological gem and embed it within EMC's vast capabilities. Global manufacturing, distribution and world-class enterprise support. Rich and powerful replication capabilities. All manner of management and application integration. Tight integration with VMware and other hypervisors. Solutioneering around popular use cases. Security and compliance. Etc. etc. etc.
Obviously, there's a lot EMC can potentially add to what's already a pretty good story. Just as we have with other storage acquisitions such as Data Domain and Isilon.
How Does This Fit With Everything Else?
So, now we've got an all-flash array from EMC to consider before too long. How does this affect how we look at other approaches?
When evaluating alongside various external caching approaches (e.g. server-side VFcache, or perhaps newer caching appliances as the to-be-announced Project Thunder), there are some interesting decisions to be made.
For example, server-side (or appliance-based) cache is just that -- cache.
When we compare it against a persistent storage array that routinely delivers sub-millisecond latency and global inline dedupe for both reads and writes, the available choices for an optimized approach certainly change.
While EMC's FAST (as implemented in the VNX for example) is an extremely useful mix of storage-side nonvolatile cache, flash drives and more traditional disk drives, how will it eventually stack up when compared to an all-flash array that does native dedupe? I know the answer, but it's not time yet to have that discussion.
Clearly, we're going to have to arm our customers and partners with tools they'll need to help them understand the new choices available. Yes, we think that more choices is a good thing.
My Obligatory Industry Commentary
Separately, you've probably noticed all the recent industry buzz around flash-this and flash-that. M&A is starting to heat up, vendor PR agencies are revving up, new partnerships are being formed -- just as you'd expect with any new technology starts to gain widespread interest.
That's all good.
What annoys me is that the focus seems to be almost exclusively on the hardware side. Frankly, that's not where the differentiated value is going to be - chips are chips. The real value is going to quickly end up being in software, and how various vendors can make it easy for their customers to exploit the power of flash storage in all its forms.
Going To VMworld?
If you're interested (and you should be) the XtremIO folks will be at the show, scheduling 1x1 meetings. You can reserve your time right off their homepage at www.xtremio.com.
Personally, I think they've got some pretty compelling storage technology to show you.
And it's only going to get more compelling over time.
Looks great. What I will be looking forward to in future would be:
1) Cost
2) Any actual performance figures
3) Comparison with say a simple VNX with all SSDs
4) Reliability???
5) Additional features - remote replication, application integration capabilities
6) Would it make sense to put all data on SSDs or maybe just a fraction of required capacity on SSDs/PCI based cache, sacrifice maybe a 0.5 to 1ms of response time but save on cost!!!!
Posted by: DumaDum | August 17, 2012 at 01:04 AM
The product isn't formally announced yet or widely available, which is when you'll get all those useful details. This is just a preview.
Posted by: Chuck Hollis | August 17, 2012 at 07:16 AM
Fantastic content Chuck. One comment in regards to 1x1's with XtremIO - the team has been receiving such attention they let me gather high value influencers by facilitating two exclusive meetups.
Internal employees get a session on Tuesday you can RSVP to here: http://twvt.us/xtremIO
Customers and partners can show interest to Wednesday's offering here: http://twvt.us/xtremIOvm12
Looking forward Project X seeing the light of day!
@mjbrender
Posted by: Mjbrender | August 17, 2012 at 11:00 AM
Hello Chuck,
It is true that XtremIO seems to be a good solution to solve many business critical situation. Of course under EMC arms, I can't imagine that XtremIO will not be part of intelligent data placement feature that will allow automatically write the data in the right storage support.This is the key success of any sensitive customer datacenter.
Regards
Karl
Posted by: K Herbst | August 21, 2012 at 08:51 AM
Hello Chuck enjoyed your good post and coverage of project X.
However you bring up a good point about how the industry in general likes to focus on the hardware vs. the software. Keep in mind that you cant have software without hardware (even in software defined hardware - SDH environments ;) ) or hardware without software, they rely on each other. Granted ability to leverage more common components is a good thing that get their value added via software, firmware, etc which as you point out deserves more coverage.
That focus on software vs. hardware however can start with the vendors as well as others who blog, write, speak, present opine or whine about related themes. In that spirit, why then did you spend so much coverage (granted its cool and I really enjoyed it) on the number and type of processors/cores which is hardware? ;)...
Perhaps see you next week at vmworld, hope all is well, cheers - gs
Posted by: greg schulz | August 22, 2012 at 03:01 PM
Hi Greg
Good point you make, they work together. The cool scale-out dedupe storage stack couldn't do what it does without a beefy computing complex, of course. But all that cool compute wouldn't be interesting unless it could harnessed to do something useful and interesting, etc. etc.
Not going to be at VMworld this year -- duties elsewhere -- but it should be a great time for all. Hope all is well with you.
-- Chuck
Posted by: Chuck Hollis | August 22, 2012 at 05:34 PM
Hello Chuck,
Now are interesting times for being able to witness a disruption caused by 'all-flash' products. Ofcourse price point and easy transition path will be the deciding factors here. Thanks for the nice article.
-- Manu
Posted by: Manu V Batura | August 26, 2012 at 04:09 AM
EMC has parches 70 companies since 200." Shouldn't this be ... "According to an article published on the Boston.com website, EMC has purchased 70 companies since 2000." ?
If you would like to be credible, your articles should be properly proof read.
Posted by: CTO Chief | March 13, 2013 at 01:00 AM