Log in to watch

Log in or create a free account to watch this video.

Log in
San Francisco 2014
Share
Download slides

DevOps and the Bottom Line

For the first time in recent history, researchers have found a link between IT investments and organizational performance — if these IT investments occur with the right mix of IT, culture, and practice called DevOps.


For the last two years, Dr. Nicole Forsgren has worked in collaboration with Gene Kim, Jez Humble and Puppet Labs to determine the health and habits of DevOps organizations, examining over 14,000 survey responses to identify the top predictors of IT performance and organizational performance. Dr. Forsgren will give a brief summary of the shocking findings of the 2013 survey, as well as an outline of the desired outcomes for the 2014-2015 survey.


She will close with a call to action, inviting the DevOps Enterprise Summit attendees for their help in shaping the IT Revolution research agenda.

Chapters

Full transcript

The complete talk — auto-generated from the talk's captions.

I want to thank everyone for letting me visit with you today about some of the exciting things that we've been finding. So I'm not a horse, I'm not a unicorn. I'm the one that sits around the campfire at night and tells stories. Usually about some of the things that we've seen as we go out and we explore and we see what happens when we're among the horses and the unicorns, but also what we see in the data.

What we see as we go in and we start exploring. And I'm excited to talk about the things that we see among companies that are using DevOps, especially among the things when we talk about impacts it can have at your company. And not just in the IT function, but the impacts it can have on the bottom line. So for the first time in recent history, researchers have found a link between IT investments and organizational performance.

But only if these IT investments occur with the right mix of IT, culture, and practice that we know as DevOps. Now, for the last two years, I've worked with Gene, Gene Kim, Jez Humble, Alana Brown, the fantastic team at Puppet Labs. We've studied organizations that use DevOps practices to really understand what contributes to the success of DevOps in their organizations. And for the first time, test and measure and see what the outcomes of those are.

So for the last two years, we were able to test and see what the outcomes on the IT function and organization of organizations were. And last year, we actually tied that to organizational impacts and really see what the impacts were on. Now, so we say organizational impacts. What does that mean to the bottom line?

This is revolutionary. We like to think it's common sense, but it isn't. In fact, it flies in the face of decades of research. Research, experience, this is exactly opposite of what everything says.

Investments in IT don't impact the bottom line. Time and time again, we see no link. Studies fail to show a link. Nothing ever shows a link.

No investments in an IT show any impact on the bottom line, any kind at all. It actually has a name. It's called the productivity paradox. Any company can buy a server, throw it in a closet, give it really pretty uplighting.

Stick enough lipstick on a pig, it's still a pig. Any other company can buy the same server or a competitor's server, throw it in a closet, give it the same uplighting, maybe red. It's still a server. It doesn't give it competitive advantage.

Or if it does, it just doesn't last. So you can't have sustainable competitive advantage. And the path from IT investment all the way down to the 10K, the income statement, or the balance sheet is long and winding. ROI, that return on investment, rarely pans out.

And if it does, you're looking at three, four, five years. It just doesn't work. So I knew this. I knew this.

But I just didn't have the heart to actually tell it to everybody else on the team. So I'm like, "Guys, let's throw this in the survey this year. Let's see what happens." And they were so excited. And I did it because I had a hunch.

I had a hunch that DevOps was different. Here's why. So what makes DevOps different? Why are we seeing the impact now?

We're seeing the impact because DevOps is not just kind of different, DevOps is fundamentally different. I alluded to it earlier. It's because we're seeing impacts when the IT investments occur with the mix of culture and people and processes. It's a major re-engineering shift.

We aren't just buying a server and throwing it in a closet because anyone else can buy a server and throw it in a closet. We aren't just investing in IT. We have to invest in IT and the culture and the process. Very much like the stories that we've been hearing here at DevOps Enterprise Summit.

In fact, this is a lot like the stories that we heard coming out of the lean and the Toyota way of manufacturing in the '80s and the '90s. This is very, very similar. And so much so that I believe, and Gene and Jez and I as we sit and talk about this, we believe that just like lean and Toyota revolutionized the way manufacturing was done, the DevOps movement will be the force that revolutionizes the way technology is done across industries and all organizations. All industries.

This is the hunch that I had when we were planning the survey last year. This is why I went ahead and I threw it in to the study that we were doing. Even though it flew in the face of decades of researchEven though it flew in the face of all the experience that we had, all of the work that has been done. But how did we get here?

So I'm sure several of us are familiar with this talk. Five years ago, Velocity 2009, John Allspaw, John Hammond. They gave this fantastic talk, "10 Deploys per Day: Dev and Ops Cooperation at Flickr." 10 deploys per day? It is interesting because you talk about people who were there, and they remember being at this talk.

Or you talk about people who weren't there, and they remember the ripples that went through the community. And it was just insane, right? This is crazy. This might be completely irresponsible.

It's definitely historic. It could be visionary. And that was 10 deploys a day. Well, that was then.

Look where we are now. So Etsy Code Deployment. Mike Benton gave a talk and said what once required six to 14 hours and what they called their deployment army now takes-- well, now, this was 2013, 15 minutes and a single person. And they were doing 30-plus deploys a day.

Well, March 2014 at QCon in London, they were reporting 50 deploys a day. In just April 2014 at ChefConf, they were reporting 80 and 90 deploys a day. And Amazon deployment stats just in production and host environments, we're seeing over 1,000 max deploys in a single hour. That's a deploy every 11 and a half seconds.

This is a five-year difference. This is a fundamental shift in how we're doing things. On average, we're seeing 10,000 hosts receiving deploys simultaneously. That's on average.

And a maximum, by the way, is 30,000. This is at Amazon. Now look at what Scott Cook said, Intuit's founder. Now, these call-outs are mine.

Emphasis is mine. "By installing a rampant innovation culture, we performed 165 experiments in the peak three months of tax season. Our business result? Conversion rate of the website is up 50%.

Employee result? Everyone loves it because their new ideas can make it to market." 165 experiments during busy season. We wouldn't have considered this five to 10 years ago. This is insane.

You don't do experiments during busy season. But when else is the right time to deliver and test handing off functionality to customers except when customers are actually using it? Okay, then look at that second call-out. See that second emphasis?

Conversion rate is up 50%. This is where the storyteller comes out, right? These are organizational impacts. This is our bottom line.

This isn't just for the unicorns. This is for the horses, too. All the stories that you heard yesterday, the stories we've been hearing today, these are examples of how organizations are creating business value, competitive advantage by adopting DevOps principles and patterns. These stories paint a rich, nuanced picture of what DevOps looks like in these organizations, in our teams, and it suggests that it can and does help these organizations achieve our goals.

And these qualitative stories that add rich detail and texture to what we're doing are fascinating. But I also wanted to step back and see what the data says. Does this hold out? Right?

Because all these studies say it just doesn't. So I want to see if it does. So all this data that we collected over the last two years, covering 14,000 respondents, hundreds of organizations, it does, backs it up. DevOps is good for IT, good for your IT function.

Not just that, it's good for your organization, okay? So let's start by looking at the impacts of DevOps on IT. High-performing DevOps teams are more agile. We're seeing 30 times more frequent deployments, 8,000 times faster lead times than their peers.

This backs up the last two or three years of data. We're seeing consistent results here, okay? High-performing DevOps teams are also more reliable. Twice the change success rate, 12 times faster MTTR.

Again, we're seeing consistent results across years. DevOps is good for your IT function. That's fantastic. Okay, but I want to get to the bottom line, right?

Does DevOps help your company? Now we know it's good for your organization. It has impacts that we can see in the bottom line. It's because DevOps isn't just IT.

DevOps is the practice of IT. Okay? High-performing IT organizations, high-performing DevOps organizationsAre twice as likely to exceed profitability, market share, and productivity goals. High-performing DevOps organizations also have 50% higher market cap growth over three years.

Note there's a little asterisk on that one. It's because for that set of the study data, we really had to rely on organizations or survey respondents that volunteered a stock ticker because not everyone's publicly traded. Companies that are publicly traded are very large companies. DevOps has impacts on the bottom line for the horses.

These are the big companies. This is us. DevOps has big impacts for unicorns and for horses, and we're seeing it throughout the data, and it's really exciting. And I've got to say that when I start sharing these results with people who are in academia, really stodgy, who don't get out and who don't see the really exciting, detailed, nuanced stories, they think I'm crazy.

Because no one has seen this. The last big study that investigated IT impacts and the bottom line found zero correlation. And when I show them some of this data, everyone gets excited. This is huge.

This is revolutionary. This is DevOps. And again, it's because DevOps isn't just IT. It's the practice of IT.

This is why we see the impacts to the organization and the bottom line. Yes, it requires investment in IT, but it also requires culture, practices, re-engineering of processes. The investments are revolutionary in the change they bring about, and not just to the IT function, but to the organization as a whole. So let's take a step back.

What if these revolutionary changes could also have impacts that go beyond the organization? Let's think about DevOps in a larger context. Let's think about our environment and the things that influence our IT practices. How does this influence our own IT practices and patterns such as DevOps?

I look at things in our environment, how they influence our IT practices. I also investigate the ways that IT practices and patterns like DevOps influence people, influence teams, influence organizations. Let's all use this lens together to revolutionize the way we do business in the 21st century, and specifically the role of DevOps in driving organization growth and value. I'll be working with the IT Revolution team this year to continue our investigation into DevOps and organizational impacts.

And I want to hear from you. What should we study? What are your pain points? What are your big questions?

This is your chance to shape our research agenda. Thank you for your time. Thank you