EMC Information Infrastructure for SAP

A few months ago, I wrote an extended post regarding all the things we’ve been able to do in a Microsoft environment; applying information infrastructure concepts to the entire Microsoft landscape.

Turned out it was a pretty popular post.  Few realized the breadth and the impact of the portfolio of capabilities we had put together in this important space.

So today, I’d like to try and do that again for SAP. 

From my perspective, SAP is at the cutting edge of enterprise functionality.  They’ve pretty much defined the landscape for the industry.  But, at the same time, customers face significant challenges exploiting all that great stuff, which is where EMC fits in.

It too is a pretty broad, impactful story, and worth telling.

I also think it’s a great lens on a broader discussion – customers run their core business processes on SAP, so it’s at the heart of any information infrastructure discussion.

My First Exposure To SAP

When I first came to EMC in the mid 1990s, my job was to get EMC into the open systems market with Symmetrix. 

I met a customer who had just done a big SAP implementation, had gone live, and was getting killed on performance. 

Absolutely slaughtered.   An unmitigated disaster.

Not only were users upset, there was distinct possibility that company revenues would suffer severely.

Faster servers didn’t help, the software architects didn’t know where to start.  Things were looking very dark, indeed.

I knew they were getting killed on transactional performance.  Disks under UNIX mandated a synchronous write; otherwise you’d lose transactional data if you crash or reboot.  Big performance hit.

We had a very fast array (Symmetrix) with large amounts of nonvolatile cache (think very fast write speeds) that was basically a mainframe storage architecture adapted for open systems.  All writes went to cache at near-memory speed, and were destaged later.

At the time, nobody else had anything like this.

I took a gamble, and bet the customer that we’d double his SAP performance – without any modifcations – or he could send the array back. 

Key tasks would run in half the wall-clock time, or we’d eat it.  No questions asked.

Now, at the time there was no hard data to support my view.  And, if I was wrong, it’d be an expensive mistake, and there’d be a good chance I’d be fired.  I’ve always had courage in my convictions.  And, mostly, it’s worked out OK.

We ended up being twice as fast on some tasks, three times as fast on others, and something like 20x on batch updates.  He kept the array, and bought two more.

I remember that I knew we had stumbled onto something – infrastructure matters in the application world, especially when you’re talking about a big, honkin’ enterprise application like SAP.

My Second Exposure to SAP

Later on at EMC, I was a marketing guy, and we had these two products: SRDF and TimeFinder. 

SRDF did remote replication, TimeFinder made very high-speed internal copies of disks that you could use for other things.

Needless to say, I remembered my SAP experience, and decided that we’d need to talk to these big SAP sites about the need for a remote copy in case the unthinkable happened. 

Nice market there, to be sure.

The real surprise was TimeFinder.  We found that this simple ability to make a high-speed internal copy was useful for all sorts of things in the SAP environment. 

Backing up a large production SAP instance while on-line. 

Instant restore of a database in case of corruption. 

Spinning off test beds at warp speed to help the developers and testers. 

A safety net for migrations.   Helping with consolidations, or data center moves.

Loading a data warehouse or reporting instance from production quickly. 

The list just kept going on and on and on.

So we went back to those same SAP customers, did more and learned even more about their environment.  But, at the end of the day, it was just storage technology cleverly applied.

The EMC Big Bang

You probably know that EMC has morphed into a very different company over the last few years – 22 acquisitions will do that to you – so what we’re able to do these days in SAP environments is pretty stunning when you lay it all out.

At least, it boggles my mind.

And the few times I’ve been fortunate enough to lay this out in front of serious SAP users, they’ve been pretty impressed as well.  More to do here, to be sure.

So, fasten your seatbelts, return your seats and trays to the upright position, and let’s take a tour of what’s out there in the new EMC landscape for the advanced SAP user.

SAP Landscape Optimization

There’s a ton of hardware that grows up around an SAP environment – servers, storage – and you need stuff for production, development, test, BI, remote recovery, etc. 

Lots and lots of stuff.

So we’ve found that we can take our basic storage and server consolidation skills and usually re-rationalize the SAP hardware landscape.   

Tier storage, manage it better, and so on.  Usually this is all about cost-savings and operational efficiency.  Savings can be significant – depending – but it’s tactical, and not strategic.

A special case of this applies to the larger development environments we’re encountering.

We’re using VMware’s Lab Manager as the basis for building very cost-efficient (and flexible) development environments that do far more with less investment than the old models people used to use.

Yes, it’s saving money, but what it’s really saving is time – developers can pop virtual machines on demand, save them, archive them – with no time being spent to set up and tear down hardware and storage. 

Very cool.  And productive developers deliver on their projects sooner and with better quality.  That’s strategic.

But what about optimizing service delivery?

This seems to have gotten out of hand in larger SAP shops in a couple of regards. 

First, the interaction between logical and physical components has gotten so complex, no one can figure out what the hell is going on.  By the way, NetWeaver seems to make this particular issue worse, not better.

Second, usually there’s SAP and non-SAP applications interacting to deliver a user-visible business process.  There’s a lot of plumbing around that interacts to deliver what users see.

SAP does a decent job about telling you what’s going on in its own world as far as service levels delivered against key business processes that it implements, but it has two fundamental limitations: it doesn’t know about very many non-SAP applications, and it doesn’t know much about infrastructure.

The usual enterprise management frameworks (Tivoli, CA, HP/Openview, et. al.) do a decent enough job of showing you all the piece parts, but not how they interact.

How do you wrap the entire landscape together so you can manage service level delivery?

We’re finding that the one-two combination of nLayers Application Discovery Manager (now Smarts Application Discovery Manager) and Smarts completes the monitoring picture, and does it in an immediately useful way.

ADM builds a real-time, hi-def dependency map of the entire SAP landscape and supporting infrastructure, showing all the interactions and dependencies at a logical and physical level (something you should have anyway, right?) and doesn’t require any agents.

The integrated service delivery view that it creates is immediately useful to Smarts, which can then correlate between SAP’s view of the world, HP/OpenView’s view of the world, MicroMuse’s view of the world, and so on, to deliver real-time correlation and root-cause analysis of service delivery problems in large, complex SAP environments.

Nothing upsets a business user more than having a service delivery problem (polite IT speak for the system is down or intolerably slow) and watching IT  flap around trying to find and fix the problem is not pretty to watch. 

And, of course, storage management that knows about SAP, even with Vmware

You probably are aware that EMC’s enterprise SRM offering (ControlCenter) is the de-facto standard in large shops.  What you might not know is that ControlCenter is reasonably SAP-aware, and can map from SAP instances all the way down to physical drives, which is a useful trick. 

All the usual goodies: storage utilization reporting, configuration management, performance stats, chargeback support, replication management, etc. – all the stuff you want. 

And the next version will be probably the only SRM offering that knows how to deal intelligently with VMware.

I won’t bore you, but we’ve also got a whole boatload of accumulated knowledge about the physics of really big databases.  The usual Oracle and DB2/UDB, but more recently, a whole spate of jumbo SQL Server implementations.   Transactional and decision support variants.  Large-scale BI and DW.   

If it’s a database issue, we’ve probably seen it before, and know what your options are.  Especially if it’s a very big, very important one.

And, of course, we can virtualize file and block environments, and deliver some operational efficiency in those areas as well.

SAP Data Protection

We’re talking about making sure that applications, servers and information is available to run the business when needed. 

Lots to talk about here.

A good starting point is the traditional backup and recovery model.  Of course, we’ve got Networker which understands SAP landscapes very well, but we’ve also got disk libraries (also known as virtual tape) for very-high-speed restores.

If you need to go even faster, there’s a whole suite of local replication products (TimeFinder, MirrorView, et. al.) that make complete copies of SAP instances, and can bring them back in an instant.  All pre-integrated with SAP to address consistency issues.

Speaking of consistency, there’s a nasty problem that we address when people have multiple product instances.  It’s called the consistency problem – and it’s worth a moment to discuss.

Imagine you’ve got several instances, and a transaction that has to update each of them.  When you’re backing up (or making a local copy) you want to make absolutely sure you’re getting a clean copy at the exact same instant.

Otherwise, when you recover, the different instances will be recovered at different points in time.  Some may have the transaction, others won’t. 

Chaos will ensue.  Data salad.

One of the more obscure (yet important) areas of expertise is using unique consistency technology to ensure – even if you’ve got multiple servers across multiple arrays – we can get a clean point-in-time copy. 

On top of the local replication stack, we’ve got Replication Manager.  It lets you organize and automate replication-oriented workflow, which there seems to be a surprising amount of in larger SAP environments. 

Maybe you want to take a pre-scheduled copy, run a consistency check on the instance, then initiate a traditional backup off the copy, then leave it around until the next one gets made for an emergency restore, if needed.  Or some other interesting combination of replication and other tasks.

There’s a strong interplay between archiving and backup in the SAP environment.  SAP has good archiving tools (which we support, naturally), but we’ve found a tiering / archiving exercise can shrink SAP instances to the size where they’re much more easily backed up, replicated, etc.

What about next-gen backup using data de-duplication, e.g. Avamar?

Tantalizing, but not really ready for direct use against SAP instances these days. 

Works great for file systems, parts of the development environment, front-end app servers and the like, but needs a few enhancements before it can take on the terabyte monsters we see in back-end SAP instances.

Back-end data-dedupe solutions seem to exact a performance penalty in doing their magic that most SAP environments can’t tolerate.  We’ll need to wait a bit for the technology to evolve here, for most people.

And then there’s remote replication for business continuity or disaster recovery.

Long discussion here, to be sure. 

SRDF is the de-facto standard for serious SAP replication.  I don’t have market stats, but I would bet we’ve got most large instances these days.

Why?

Synchronous replication. 
Asynchronous replication. 
Semi-synchronous. 
Dynamic mode changes.
Point-in-time remote copies. 
Consistency groups. 
Integration with local replication. 
Multi-site failover capabilities. 
Proven management tools. 
Many thousands and thousands of successful implementations over the last decade.

Sorry if I’ve bored you.

I don’t think anything comes even close when you’re serious about mission-critical remote replication at scale.  No, I’m not biased here, just stating facts.

That being said, we’re seeing strong interest in some of the newer forms of replication, e.g. CDP (continuous data protection) – I call it Tivo for your data center. 

The ability to roll back a complex landscape to an arbitrary point in time is tantalizing to many.  And the fact that it runs on an intelligent SAN switch (e.g. Cisco MDS) rather than buried in the array makes sense to many.

The nice thing about all the replica-oriented recovery technologies is that the data is stored in a native format, which means that – aha! – it can be used for other purposes: feed a test bed, run reports, make a skinny snap if you need to write to it, etc.

We’re also comfortable with log shipping, remote tape vaulting, using service providers, and a few other related approaches.

Finally, a note on automated server failover.  We’ve done more than our fair share to qualify and test all the usual server clustering technologies, including some of the newer VMware flavors – both locally and remotely.

We’ll work with most of the popular ones, but we do have a few favorites.

Does your head hurt yet?  Wait, there’s more …

SAP Application Lifecycle Management

SAP environments seem to always be in one of three phases:

getting ready for a big change,
making a big change,
or getting over a big change.

Whether it’s new functionality, new versions of existing functionality, or the inevitable tech refresh, change is constant. 

Turns out we can do a whole lot here as well.  Many of the same technologies we’ve already discussed can be directly applied to this important area.

Server virtualization to create cost-effective (and flexible) test beds.

Local replication snaps and clones to make fast copies of data.  Replication management that automates all the scripting required.

Storage virtualization that lets you move running workloads between arrays without disruptions.

We’ve discovered that nLayer’s ability to do real-time discovery of the landscape is turning out to be invaluable to understand exactly what you’ve got, how it all interrelates – and, more importantly, has anyone made any changes that they’ve neglected to tell you about?

The big challenge here is that most people don’t realize what the technology can do – or, more importantly – how it can shave weeks or months off a large transition *and* reduce risk along the way.

As people move to SOA and NetWeaver, they’re finding that their traditional approach to dev and test has serious limitations. 

Gone are the days when you’d set up a developer with a server and some data and say “have at it”. 

You’ve now got to have multiple entities all communicating at the same time (think VMware test beds).  You have to understand how the logical and physical interact in an entirely new way (think nLayers). 

The real value here is our global services people who know how to design SAP Application Lifecycle Management environments, and help people to use them effectively.

If you always find yourself in one of the three states above, we should talk.

SAP Archiving

Now we’re really going to have some fun. 

I’ve mentioned before that most companies inevitably end up doing three archiving projects: one to save money, one to avoid risk, and one to create new value. 

I keep saying that they should just do one that does all three.

The first driver is “gee, the production instance looks like it’s getting ready to swallow the data center whole, do we ever take anything out?”.  The first goal is to save money on server consolidation and storage tiering. 

And, yes, there are usually serious savings to be had.  Tactical, but not strategic.

The second driver is “gee, the finance and the legal guys are talking about this compliance stuff, we need to have this around in case … well, we don’t want to think about that”.

Usually, this takes the form of compliant storage.  EMC’s Centera is very popular here, since you can prove that the data hasn’t changed once it’s loaded.  Yes, that matters.

And, then, finally, someone wakes up and says “gee, there are users out there who really want to see this archived stuff, and put it to good use”.

Turns out that SAP’s archiving tools don’t really help you *use* the information.  Two useful tools here, one tactical, one strategic.

EMC’s ViewPoint can best be thought of as an “archive viewer” that lets SAP archived data be used from the same tools that SAP users already use.  Or present it differently, if that’s what you want.

More strategically, SAP information needs to be combined with other forms of information, and built into workflows, collaboration environments, enterprise search, and so on. 

Think Documentum.

On a more strategic note, the information locked inside of SAP is some of the most useful information in the company.  But SAP’s tools focus more on the transactional process side, rather than the knowledge worker and collaboration side. 

We’re seeing some pretty big companies realize this and step up to hefty Documentum environments for just this purpose.  Hope this trend continues.

SAP Information Security

There’s some pretty sensitive data in SAP environments that needs to be protected against unauthorized use.  And SAP has some decent tools in their environment to help you do just that.

We’re just starting to explore the interaction between our new security portfolio (RSA) and SAP.  There’s a lot we could potentially do, but we’ve uncovered a few preliminary nuggets to consider.

First, you’d be surprised how many SAP environments use a simple name/password scheme to gain access.  No two-factor authentication.  And it seems to be OK to use your dog’s name as a password. 

Fixing this is simple hygiene.  And all the RSA authentication integration is done.

Next, we found that a whole lot of SAP-protected information ends up in reports, spreadsheets, powerpoints, downloads, memos, etc. – which in turn land in poorly protected file and email systems – which, of course, can get into the wild despite best intentions.

EMC’s new Infoscape plugs this hole. 

It can discover all the file systems you own (including the ones you may not know about), run a powerful filter search to identify sensitive situation, and then gives you choices on what to do with what you find – archive it, move it to a more secure environment, plus the ability to apply DRM on-the-spot at some point in the future.

Next, we found that SAP generates security logs, and the network generates security logs, and the firewall generates security logs, and … but there’s no “catcher” collecting them all, and analyzing what might be going on – either in real-time, or after the fact.

That’s the rationale behind our Network Intelligence offering.  Having all that good logging doesn’t do much good unless you put it to work.  And it’s a good way to catch the logs you already have and really figure out what's going on from a security perspective.

There’s a more esoteric discussion around DRM embedded within SAP.  I’ll spare you. 

And a more mundane discussion around encrypting data-at-rest, especially tapes.  And recently, there’s been strong interest in data masking for development environments, and we’ve found a few good solutions out there.

And, sad to say, we’ve just scratched the surface. 

Sorry.

Content-Enabled SAP

When SAP was designed, all useful information was events and transactions.  Nice, compact text and numeric fields.

The world has changed a bit. 

There’s scanned documents, pictures, video, voice, powerpoints, graphics – well, you get the idea.  And more and more organizations find themselves dealing with a host of interesting challenges in getting all this useful content to work seamlessly with their SAP environment.

All sorts of things in the Documentum portfolio are useful here. 

Products like Captiva that can scan and parse document images, and feed both SAP and Documentum environments.  Using Documentum as the “repository of record” and just putting pointers into SAP for the content bits.  The ability to create custom presentations that show SAP and non-SAP information in an integrated fashion. 

It seems that Documentum does just about everything you’d wish that SAP would do in this regard.

Whether it’s putting content in SAP, putting pointers to content in SAP, or blending presentations of information in new and useful way, we’ve done an enormous amount of work in this area, and it’s becoming more useful every day.

So, how much of your information isn’t neat transactions, and ought to be integrated with SAP applications?

Collaborative BPM

BPM (business process management) is a hot topic, but I think many people are missing something important.

Every time I get into a BPM discussion, I see them discussing transactional processes like order processing, or customer service processing, or maybe procuring a box of paper clips.

Nothing wrong with these transactional processes – they’re important, to be sure.

But I am very cognizant and vociferous that the action has moved on to what we are becoming more of these days: knowledge workers vs. process workers.

And, if we’re going to support these kinds of core business processes, we’re going to need a new style of collaborative BPM that is fundamentally different (and requires very different tools) than classical transactional BPM. 

Call it “enterprise collaboration management”, if you will.

I wrote a rather detailed piece about this subject that’s worthy of pondering, if you have time.

Bottom line – you’re only going to get so far in improving productivity with classical transactional BPM.  Think Documentum.

Putting It All Together

If you’ve made it this far, you’ve done better than most. 

We’ve talked about a lot:

SAP Landscape Optimization
SAP Data Protection
SAP Application Lifecycle Management
SAP Archiving
SAP Information Security
Content-Enabled SAP
Collaborative BPM with SAP

And, oh yes, we have some nice storage as well, if you need some ;-)

On one final note, if you’re a regular reader of this blog, you probably know what I’ve been saying about the new mandate of IT, becoming an informationist, and the role that information infrastructure will play.

As I look across the IT landscape, I think the first people to make the transition will be the SAP crowd. 

They understand the importance of information from a corporate perspective. 

They understand information ownership issues. 

They understand how information plays out differently for different business units at different times. 

They know how information can cost you money, make you money, or – put you at risk.

Will they be the first informationists?