No one was more surprised than me when my recent post on pNFS and the associated EMC-sponsored Bake-A-Thon sparked a flurry of comments and behind-the-scenes debate.
I had naively assumed that many people knew what it was, had been tracking its progress, understood its impact, etc. I guess I was mistaken.
So, given the level of interest, I thought I'd offer up some follow-on thoughts for you to consider.
The Basics
pNFS 4.1 might best be described as a "converged storage access protocol".
The application gets to access storage using a convenient protocol (e.g. NFS 4.1), but the underlying storage device might be a file system, a block device, or even an object storage device.
Coordination and mapping is done using a "layout" request/response protocol to a metadata server which used to function more as a NAS head. The metadata server is now out-of-band: once storage objects are found, mapped and locked, the underlying storage protocol between application server and storage device is abstracted by pNFS.
A simple example?
You're running a large, production Oracle database on what looks like NFS. When the database is opened, file-oriented requests are sent to the NAS (now metadata) server. A layout is supplied back to the server running Oracle and pNFS. That layout can now be used to have a SAN-style conversation with a block-oriented storage device. The Oracle application now gets the performance, response time, resiliency, etc. of SAN -- but with the convenience of NFS.
A more complex example?
You're approaching billions of content objects to store and manage, so you've made the leap to an object-oriented storage device -- maybe EMC's Centera or Atmos. However, your applications are stuck somewhere in the 1990s, so you need to present a file system to them.
Rather than use some gateway in between the two, consider using pNFS. Legacy applications see a familiar NFS model. The pNFS metadata server supplies a layout mapping that permits a direct conversation between the application using pNFS, and the underlying object server.
Disclaimer: although EMC is now shipping a version of pNFS 4.1 on Celerra, it's going to take a while before the rest of the portfolio, industry and ecosystem catches up. Maybe a year or so.
Convergence At The Server
Most of the discussion this year has been around converged storage networking -- how today's world of Ethernet and FC is quickly becoming 10Gb ethernet running whatever: NFS, CIFS, iSCSI, FCoE, etc.
One wire to rule them all ...
Most astute observers would realize that this does nothing whatsoever to address the diversity of storage access models that servers and applications see: two major variants of file (NFS and CIFS), one primary flavor of block (e.g. LUNs seen over a SAN) and multiple flavors of object (EMC's Centera and Atmos as de-facto market standards, with a whole bunch of "other" waiting in the wings").
A converged storage access protocol would take something that's well-established and popular, and simply extend it into areas it hadn't been that good at. And NFS is the obvious choice.
Endowing NFS with SAN-like characteristics (and preserving its topology, infrastructure and management model) is an interesting outcome of pNFS. Simply put: you get the performance and resilience of SAN with a standard NFS access model.
Endowing NFS with object-like characteristics (scale, ability to embed metadata, enforcing policy at the infrastructure level, etc.) is also an interesting outcome of pNFS. Simply put: you get the architectural and strategic advantages of object with a standard NFS access model to use where needed.
The Inevitable Debate Arises
There is a school of thought in the storage industry that simply hoisting one access model on top of another at the array (e.g. iSCSI and object stores on top of a file system) is the preferred way to go.
NetApp has done this, as well as EMC's Celerra. The underlying storage file system is used to create emulated containers of LUNs and/or objects that use the underlying native file system. The storage array presents whatever the customer needs for the application.
While this is certainly convenient for many storage use cases, it's hardly optimized. SANs behave differently than file systems -- that's why they exist. The same can be said for object storage schemes -- they behave fundamentally different than file systems. That's precisely why they're so attractive.
This "convenience vs. optimization" scenario will likely be argued vociferously by all the usual suspects. At smaller scales, optimization will likely be outweighed by convenience. And as scale and requirements grow exponentially, the balance will shift towards an optimized approach.
We see it all the time in our own business. A customer starts off with perhaps an emulated iSCSI SAN running on top of a filer. Their requirements grow. Pretty soon, they start using their SAN as a real-deal SAN. And, as a result, they frequently go looking for a different approach.
I think that's been one of the advantages of EMC's Celerra over the years -- the FC SAN connects to an underlying EMC CLARiiON (and is not emulated by the file system). When you need a real SAN, you've got a real SAN.
Now, compare and contrast this approach with pNFS. Everyone gets to use NFS to access their data. As applications grow and expand, the underlying access protocol can transparently change (thanks to the magic of the metadata server) while still using a converged ethernet-based storage network.
If a filesystem approach isn't keeping up with scale and metadata, an underlying object protocol can be used -- applications and servers are unaware. Or if the use case starts looking more SAN-like, same deal. Transparent and non-disruptive data mobility? Being done today, although not specifically for a pNFS use case -- that's just a matter of time and roadmap.
How Will It All End?
I think we'll see both approaches in the market. No winners or losers, just more choices for customers to consider.
Smaller and more modest use cases will be drawn to the all-in-one approach of a converged storage array that can support multiple access models.
But I think that larger and more demanding use cases will be drawn to storage protocol abstraction using pNFS, and choosing differentiated storage arrays that are really good at what they do: file or block or object.
Let the debate begin ...
Comments