Learning To Escape The ROI Trap

Of all the intellectual tools foisted on business professionals, I have come to loathe the familiar ROI (return on investment) discussion.

On one hand, there is nothing wrong with expecting some sort of return on any investment.  And it's always a good intellectual exercise to figure out exactly what you expect to achieve from your efforts.

My rant isn't so much about the tool; it's about the widespread misuse of the tool.  I think it's been responsible for shutting down more promising ideas in corporate environments than anything in its class.

I find myself in meetings occasionally getting very agitated when I see the egregious misuse of the concepts.  I think word has gotten around here at EMC -- don't bring up ROI in front of Chuck unless you're very sure about what you're doing.

Originally a simple concept around financial investments (money in, expected money out), it's now morphed and mutated in many large organizations so that it's become almost unrecognizable. 

More concerning, it's often become a weapon to extinguish investment in the big ideas that every organization could benefit from.

Here's The Picture

You're in a meeting. 

There's an interesting idea on the table.  It's going to take some money, people and time.  Doing so means that fewer resources are available for other potential investments. 

How do you decide whether or not it's a good thing to do, or not?

Someone says "We need a business plan".  To be shortly followed by the "yeah, and we better show a great ROI for what we're proposing".  Which is inevitably followed by a long silence on who gets to run with that particular thankless task.

Thwooop!  In the space of 10 seconds, a promising idea has been instantly sucked into an intellectual black hole, perhaps never to be seen again.

What happened?  And how can it be avoided?

Understanding Motivations

Imagine a well-understand, well-measured business process: something that's familiar to everyone.  An investment is proposed to incrementally improve said process.  The investment consists of well-understood inputs with predictable outputs. 

Doing some flavor of ROI makes sense to me.  But these cases tend to be the exception, rather than the rule.

What if the process in question isn't well understood or well measured?  What if it's very unfamiliar to the organization?  What if you're looking for something other than mere incremental improvements?  What if the inputs and outputs aren't all that predictable? 

Attempting to use ROI tools in these cases seems a very bad idea indeed -- the answers that result will give you no insight whatsoever as to whether or not it's a good idea or not.  And the preceding rock-fetching exercise is likely to crush all the life out of the idea, and everyone involved.

Here's the problem: all the cool, important, really-change-the-game ideas fall into the latter category, and not the former.  Because we don't teach people how to evaluate imprecise opportunities, they fall back on tools that expect precision.  And we end up with poor outcomes because we chose the wrong tools.

Personal Example #1

If you follow me, you'll probably remember that I led a social media proficiency strategy here at EMC many, many years ago.  Early on, I found myself being sucked into demands for ROI justifications internally.  And as I started to go to external events, there were plenty of other people preoccupied with the same concerns. 

Where's the ROI? -- almost like it was a holy mantra.

After a bit of introspection, I realized the true goal of my project was to create entirely new capabilities for the organization with literally unbounded potential.  We could easily imagine dozens and dozens of important business processes that could be potentially transformed when recast via social media: marketing, customer service, hiring, product development, etc. etc.

But I couldn't document or prove any of it. 

The processes weren't well understood, well measured, etc.  The inputs and likely outcomes were similarly vague.  And forcing the proposal into a classic ROI discussion would mean crushing the very life out of the initiative. 

Early on, I got a quick ballpark estimate for what it would cost to have some consultants come in and study the problem.  The proposed expenditure was greater than the total cost of the initial pilot.

The more I resisted, the more some people pressed their demands.  I started to suspect that "ROI" was code for "I'm really uncomfortable with what you're proposing, so let's run you through the wringer".

I went looking for the proverbial judo throw where I could use a bit of leverage.  Polly Pearson (who was working in the office next to me at the time) came up with clever and useful twist: ROI == Risk Of Ignoring. 

Put differently, if we didn't get busy on investing in these new capabilities, we'd likely suffer at the hands of our competitors who did. 

Given the extremely competitive and aggressive nature of EMC's business, that did the trick.  I got very good at painting a future scenario where we didn't make the investment -- and what could happen when our competitors did.

Investment approved.

But there was a clear point in time where I had to intentionally commit to navigating away from the intellectual black hole of "Return On Investment", otherwise the initiative would have disappeared into the event horizon, never to be seen again.

Personal Example #2

A few years ago when I got really serious about proposing cloud strategies and IT-as-a-service business models to IT leadership teams, I started to see the ROI discussion again. 

What was the ROI of a cloud investment?  How could we financially justify it to our taskmasters in finance?  Where was the well-documented, completely air-tight and unarguable business case for doing so? 

Where was the ROI?

Once again, I felt a strong gravitational pull into yet another black hole.  Once I gave in, there would be no escape once we crossed the proverbial event horizon.

I ended up structuring my rebuttal into two components.

First, I argued -- if you're concerned about financial models, you're probably measuring IT spend incorrectly.  You're focused on aggregate spend on IT.  That's a finance discussion, not an IT discussion.  Your specific focus, friends, should be around lowering the unit cost of IT service delivery --- as consumed by the business.

Do you know your unit cost to serve?  Can you compare it favorably against external alternatives available to business users?  You're focusing on the cost of the inputs, where you probably want to shift to the cost of the consumed outputs. 

Long pause inevitably ensues.

Second -- and more importantly -- you're going to have to learn to compete for internal IT spend.  Business users are starting to think of you as just another potential source of IT services, and they're increasingly willing to shop around.  If they always find better alternatives than what you can deliver -- whether built or brokered -- the future world is not good.

In this light, ROI clearly means Risk Of Ignoring. 

Longer pause ensues.

Personal Example #3

I now am frequently engaging around the amazing transformational potential of big data and predictive analytics.  I can usually make a few heads in the room explode, but not all of them.  Since we're talking business processes that are not well-defined, well-measured and a great degree of uncertainty around inputs and outputs, guess what inevitably pops up?

"You're proposing a substantial investment here of people and resources.  We'll need a business plan, and have to demonstrate the ROI".  Sound familiar?

Once again, there's that strong pull into a oh-so-familiar intellectual black hole ...

I structure my argument in a familiar way. 

You're creating important capabilities in the organization you don't have today.  We can construct interesting scenarios where this stuff could potentially make an amazing difference, but there's no guarantee whatsover as to specific outcomes.  We can run very cost-effective pilots to see if there's value there.  But, more broadly, it's sort of like corporate R&D -- you're not exactly sure what you'll find, and that's the point.

And, wait for it ...

More strategically, you'll need to develop these capabilities in some form, or potentially suffer at the hands of your competitors who do.  They'll have better insight into key business processes, consumer behavior, risk management, predictive outcomes, etc. that you don't have. 

And your ROI really should be thought of as Risk Of Ignoring.

Square Peg, Round Hole?

When life presents you a square peg and a round hole, you have a choice to make.  Do you round off the peg, or enlarge the hole to accomodate the new and interesting shape? 

Or do you simply discard the peg and go looking for one that fits?

As our careers progress, we'll likely end up more frequently in situations where we have to make our own personal choices on how we'll choose to react to the established consensus.

I've made my decision -- have you made yours?

Comments

[Dave Locke]

I totally agree with the sentiment and know we have great products that fall into this category (disclosure: I work for VCE). However, the real ask here is 'trust'. We are in a position to deliver; but only if we can have the customer believe and trust that our organisation can deliver on all the promises. We know we can, and once seen, the 'ROI' reveals itself there and then.

The challenge, is people doing what they've always done, and being cynical about the new ways which leads to an easy retreat to an ROI shelter.

Certainly one to think about over the long weekend...

[Jon Nelson]

What about projects like your EMC|ONE? It must be difficult to place an ROI on an environment that doesnt generate any revenue. But the productivity gained from such an endeavor has value.

How can an individual hope to define the value of a project that could have such benefit to a company when they are always confronted with an ROI argument to spend money?

[IAmOnDemand]

excellent piece .. though I suspect that biz will still want to see the ROI. IMO Smart leaders will relate to it as one of the factors that impact the decision and not THE factor.

WDYT ?

[Chuck Hollis]

I think a smart leader would want some sort of handle on the numbers involved up front, and likely scenarios at the back end, but that's a far cry from a formal ROI study.

My belief? Once you invoke the ROI spell, everyone gets hypnotized. For anything I'm involved in, I do whatever I can do avoid the term.

-- Chuck

[Steve Kaplan (@ROIdude)]

Chuck,

Good post and I agree with the overarching theme that ROI, used as a rigid, narrowly defined process as a means to avoid truly understanding and evaluating complex technologies – is bad.

ROI is just a financial analysis tool, and like most tools, can be beneficial or not – depending upon how it is wielded. In my experience, the problem isn’t typically a misuse of ROI, it’s that most IT organizations either don't utilize financial analysis at all when evaluating alternative technology approaches, or they do so at only a very basic level – primarily considering just investment CapEx costs.

The benefit of going through an ROI analysis as opposed to a more basic financial evaluation such as TCO, is that it demands a much broader perspective. Calculating the relative aggregate savings each alternative is expected to generate in relation to the outlays significantly helps both make and justify a decision. Factoring in both outgoing and incoming cash flows on a discounted basis by year further improves the comparison. A discounted cash flow approach additionally allows the finance department to better evaluate proposed IT investments versus other opportunities for the firm’s capital.

ROI doesn’t necessarily have to equate to hard dollars. As you point out, the benefits of staying ahead of the competition may far outweigh the cost of doing nothing. That being said, there is no reason why an attempt shouldn’t be made to quantify the cost of falling behind the competition. Why perhaps not particularly meaningful to an accountant, going through the exercise can potentially provide insights and identify both costs/risks and opportunities not otherwise contemplated.

A final note is that Cloud is driving more interest in extensive financial analysis including ROI. This is partly because organizations struggle about what components of their IT should be in a public vs. a private cloud. But it is also because many IT departments don’t have a good grasp on what their true costs are. In order for them to effectively provide IT-as-a-Service, they need to be able to accurately cost and price IT resources in order to encourage optimal utilization.

[Tal Borenshtein]

So once we unable to give a hard evidence on the return on investment and we know the risk of ignoring will cost us either in revenue or reputation.
How can we quantify the risk of ignoring?
surely, numbers should be thrown, a reason for the resources to be given.
What happens after the idea of being second best and by that loosing revenue is pitched and you have the boards/finance attention?

[Chuck Hollis]

Tal, Steve

Please don't get me wrong, I do believe that a thorough examination of potential investments and likely outcomes is a healthy part of any business discussion.

The problem usually arises when people go looking for precise predictions denominated in dollars. The tools at hand are cumbersome at best, or -- at least -- require a very advanced practitioner to wield effectively.

What is the ROI of a good marriage? What is the ROI of a college education or advanced degree? Having children?

It seems to me that all the really important decisions in life aren't amenable to a straightforward ROI discussion. I suspect the same thing might be true in the business world as well.

-- Chuck

[Thesocialswede]

Amen to that.

Wonderful to read this inspirational piece about to fight the ROI-talibans. I needed some more politically correct answers than my three standard ones:
1 - How much time did Steve Jobs spend on calculating ROI for the iPad, do you think?
2 - What's the ROI of you providing email/mobile phones to your employees?
3 - (aligned with your argument about known/unknown): If you had asked consumers some 15 to 20 years ago about how interested they were in being able to create 160 character text messages on their phones, what would the answer have been?

Many thanks for added variation and intellectual elevation in argumentation.

[Jay Norris]

Interesting post. I am often surprised by how much ROI is a buzz word these days. You are right, showing an ROI is certainly no guarantee that the numbers will match reality. But with the right tools and in the right situation, ROI can help in the decision process.

[Chuck Hollis]

Thesocialswede:

I've learned to have a great deal of empathy with folks who are in the ROI trap.

1 -- it's the only business justification framework they're familiar with.

2 -- it may be the only business justification framework their management is familiar with.

3 -- the notion that there might exist better and more useful frameworks to arrive at a decision is an uncomfortable thought at best.

Many, many years ago I worked for a small tech shop that coded everything in COBOL when I was working my way through college.

I wrote a very large piece of modular code that should have been done in Pascal, C or similar, but was forced to use COBOL. I simulated encapsulation, parameter passing, etc. all within the constructs of bog-standard COBOL.

It worked as advertised (and was probably horribly inefficient) but ended up being some of the strangest COBOL anyone had seen at the time. The old-school types were collectively scratching their heads at what I had done with their precious language.

Which goes to show what happens when you force someone to use a tool that isn't suited to the task at hand :)

-- Chuck

[Gwen Zierdt]

Great post and great ideas on how to creatively respond to arguments intended to foster good business decisions but are innovation killers.

[jimmyPx]

Great post and you are so correct.

In my experience, someone demanding an ROI is really saying "I need to CYA and/or I don't like this project but don't want to be the bad guy and kill it and make enemies or look bad. So, the ROI will kill it and/or I can nitpick the ROI and not be seen as a bad guy".

In my experience an ROI is a total waste of time, money, and resources for most IT projects. A simple TCO should be able to suffice for the financial end. In my opinion, if someone is always demanding an ROI then he needs to be moved out of IT management.

[Dmitri Kalintsev]

Chuck,

An excellent post.

One note: we often go and do something because we've picked up on cues that are difficult to quantify, sometimes calling them "common sense". However, in many cases because they are difficult to quantify doesn't mean it's impossible.

I believe there's at least one methodology out there that appears to works in many cases, enabling a structured opportunity discovery that yields results that can be reliably quantified - read: "it allows to figure out how much one can expect to make by addressing a particular discovered opportunity".

It has been developed by Tony Ulwick, an HBS colleague and friend of the famous prof. C. Christensen, and been used to help companies across a wide variety of industries to discover (and justify, with hard, solid data) opportunities for growth.

The theory behind the method is based on the notion of "jobs to be done", and the associated "desired outcomes". A person doing a job may not be able to provide you with a good idea on how to help him do it more efficiently, but they sure can tell you what steps they go through when doing the job, how important each step is, and how they know if they done well at each step.

This information, when collected and analysed correctly, highlights the opportunities for improvement (important but underserved desired outcomes). Typically, the more important a particular underserved desired outcome is, the better chance that there's willingness to pay if you can help them do better. And because you can quite reliably judge if one or the other product improvement or a business initiative is likely to improve on satisfaction of an underserved important desired outcome, you can make business decisions with much more confidence.

This isn't something that magically produces you an ROI, but it is something that can get you a long way to making a solid call on what the benefit is likely to be. Not only that, but it can help you find opportunities where you may have thought there were none.

Don't want to appear soliciting or something, so no links. There is a book called "What customers want" that describes the methodology in detail, and there is a very good white paper on Tony's company web site, strategyn.com, titled "Outcome Driven Innovation", that summarises it.

Cheers,

-- Dmitri

[Cobiacomm]

Excellent post. especially the point about IT groups not usually knowing their internal delivery cost. I have created a PaaS TCO analysis, and I welcome your feedback:
http://blog.cobia.net/cobiacomm/2012/05/13/paas-tco-and-paas-roi-multi-tenant-shared-container-paas/

[DMR]

Chuck, this was an intersting read. I think part of the problem with the ROI blackhole is that so few people are either willing or able to do fair ROI analysis. An ROI can certainly be based on opportunity costs, as your Risk of Ignoring suggests. In the 3 examples you cite, I feel that each of those could have held up in the light of an ROI analysis that was savvy enough to consider the opportunity costs of inaction. Very interesting read. Finance can be as creative as any of the sciences in our business, yet is one of the least understood. I think that is mostly to blame for the "ROI abyss" that is so often the seen as a tool to kill projects that make folks uneasy.

[Chuck Hollis]

Hi DMR

I agree with your sentiments. The tools are only as good as the people who wield them. And I have seen wonderfully nuanced analysis from professionals that lay out different scenarios that include opportunity costs as well as more traditional inputs.

But -- unfortunately -- those are the exception vs. the rule.

Thanks for the comment!

[Chris Markham - Data Driven Marketing Strategist @ Bizfix]

Chuck, thanks for raising the ROI elephant. I'm a numbers guy, a statistics guy, I am definitely an ROI guy but I'm right with you that ROI does not mean total risk elimination. Using the minimum viable product approach is a way for enterprises to 'tolerate' higher levels of risk while routing out real ROI as in these examples http://bit.ly/blogROI
Chris