Hi, my name's Stephen Elliot, vice president of IDC. How many folks are familiar with IDC? That's about what I expected. So, we're essentially, in terms of revenues, we're number two behind Gartner Group.

We're an end-user advisory and research firm, do a lot of work with data. And what we like to do with end users is quantify a lot of the things that you're seeing, whether it's in DevOps, hybrid cloud deployments, private cloud, enterprise architecture. Pick your favorite topic. We've got 1,100 analysts.

And we do a lot of work with data. That's where the company has been built up from in looking at market share and forecasts and really solidifying a platform to help people understand where are things going and what are my peers doing. So the research that you're going to see here in these slides, I don't have any fancy videos. I don't have any fancy pictures.

This is all about facts. This is all about what are your peer groups doing around DevOps. And it's about metrics, right? You've seen presentations about business metrics, technology metrics, trying to define the value of DevOps.

And so we're going to talk a little bit about that with some numbers. And we're also going to talk about a lot of the inquiry we get increasingly is around the role of compliance, security. We're seeing some of that, so we have some research around that. So you're the first group of people in the world to see what we're going to go through today, and this is based, again, on quantified survey results of your peer group as it relates to DevOps.

So first off, how many folks here know what your CIO is judged on? How many folks have a sense of what do they get paid on? Not many. So one of the things we did is we surveyed C-level, senior level executives to really think about and ask them, how does your CEO define you, right?

Because if you're talking about metrics, and you're talking about changing and transformation, you really have to understand what motivates people. And I spent a number of years at CA as a vice president of strategy in one of the largest distributed units there. And let me tell you, it makes a big difference when you understand what your manager's motivated by and why they're asking you something, and what actions you have to take, and where you spend your time, and with whom you spend your time. And so we asked this of 233 senior level executives, and as you can see here, probably not that surprising.

We've got a decent percentage looking at operational. They're viewed as an operational CIO, managing risks, cost savings. You've got business service brokers, about 44% or so. And then you have folks that are viewed as a chief innovation officer.

We asked the same question, how do you think your CEO's going to view you in three years? And look at the jump in innovation. It's a massive jump. The other two go down.

But the view in terms of how your senior technology leadership team is viewed in terms of more innovation. Then we said, well, what are the metrics you're judged on as a CIO? And I think to some surprise, you've got high levels of service availability as a key metric. You've got a number of folks being judged around certain levels of compliance.

And I think the thing that stands out here is you've got business process transformation, but you also have, increasingly, senior level executives being viewed on revenue and profit margin. So when you think about DevOps and what you're doing today, ask yourself how could you take that and communicate it as it relates to a revenue or a margin improvement, or really a specific business metric that drives the discussion with the senior level managers. Now, what are some of those metrics? We've seen in some of the presentations a lot of technology metrics.

And this is not all inclusive, but these are some of the hot button ones that seem to pop out during most conversations. Deployment frequency, lead time for changes, change error rates, lines of code availability, recovery time, mean time to repair, mean time to failure. Nothing too surprising, but it's a great conversation to have when you're talking about your teams and the technology metrics. Then you move to the business metrics.

And one of the things that really strikes me is so many conversations that we're having with clients around translation. There was one particularly large gaming development company that was going full in on DevOps, and we were talking to them, and one of the things that occurred was they had great metrics when it came to the technology. They were doing everything right, but they couldn't get the thing started because the business manager didn't understand what they were saying. So the senior technology leadership team took the presentation, translated it, put it in terms that the business executives could understand.

The project was funded. And I think that this is something that when you think about metrics, number one, think about who you're talking to, right? What are they driven by? And number two, how do you communicate that?

What value do they see? Because as was mentioned earlier, DevOps is a pretty broad umbrella term. It means a lot of different things to different people. But when you can start parsing out how you translate deployment frequencies to time to market, to increasing productivity, increasing quality, then all of a sudden you've got multiple types of conversations you can have, which ultimately all starts with what you're doing.

So I think one of the things we're finding are really how do you define value with regards to some of these metrics.Now, again, going back to the notion of translation. We're seeing productivity. You could talk about speed. You could talk about optimizing what you have more and more.

I always found it interesting over the past couple of years when I hear senior executives talk about cost savings. Isn't that expected? And then on top of that, why would you even talk about that when you're going to be asked to give that money back in the next budget cycle? So think through this notion of productivity as a metric, how you communicate that around time to market, config automation, et cetera.

Quality. Improved availability, looking at deeper requirement analysis. These are all things that you know, but I think there's also a wonderful opportunity to, again, translate that into, hey, you're optimizing what you have, and at the same time, you're improving the quality of the service itself. Operating expense.

This is old but still relevant with regards to the common business metric. What you're doing is cost avoidance. You're doing more with what you have. It's never about cost savings.

You're bringing to the business, maximizing every resource you have, every tool that you've decided to purchase, every process you decide to implement. That's wonderful. So don't talk about cost savings, talk about cost avoidance. Fail fast and fail cheap.

Another common theme in the DevOps scenarios. It's a wonderful thing to go to an executive and say, "Look, we're going to change the culture, and we're going to fail. But we're going to fail fast, and we're probably going to fail cheap." It's not a bad idea. Capital expense, again, improving utilization, cloud-based systems, convergence.

So, the net theme here, these first couple slides is, number one, understand who you're talking to, at what level. What are they driven by? What are the metrics they're paid on? Number two, translate.

Understand what are the key metrics, then sort of think through what... Ultimately, we're doing this for business. We're doing this for the business. Doing it as a potential growth platform.

And then think about what is the business impact, and how does that translate? Now, just for fun, I put together this slide, S&P 500 stock-based index of 500 large companies. It's publicly held on New York Stock Exchange or NASDAQ, and it's based on market capitalization of those 500 companies. And what you see here is FedEx, Visa, Aetna, Disney, Facebook, Netflix, Amazon, and Bank of America.

And this is a pretty random group other than some of the unicorns that are on there. And I know some of the other companies are actually doing some DevOps in various instantiations and various levels of maturity. And what I always find fascinating here is, number one, S&P 500's pretty easy. It's a common business metric.

It's a common financial metric. Every executive management team understands it. But what you see here, and I think I would urge you to think about this, if you're a publicly held company, go back and put your ticker in there and compare yourself to the S&P 500, and then say to yourself, you're doing DevOps. Have you did it for the past year?

Have you been doing it for two years, three years, four years? And take a look at how you stack up with the S&P 500, just as a basic financial performance metric. Because it's very intriguing to me when you look at this, and I didn't expect these results, particularly when you look at Bank of America, you look at the ones that dropped under the black line. And I ask myself, when did they start thinking about DevOps?

When did they start deploying some of those projects? How strategic was it in those strategies of those companies, and how high did it get boiled up? Because my takeaway from here is when you look at the unicorns that clearly, for the most part over this three-year trend line, outperform the S&P 500, a lot of those folks built their business on DevOps. They built their application and business service strategy on quality and speed.

So, I would say time is the new currency, and I think this slide is one of those points that makes you think maybe there's something there that you need to rethink. Now, we talked about the surveys. One of the things we did in this 2015 CIO Sentiment Survey was really ask this question: What percentage of your current projects have failed to meet your success criteria? And out of 84 folks that answered this question, they said 19%.

So you're talking almost two out of 10 projects failed to meet the IT executive success criteria. What's interesting is when you dig into why, you get this number of 35%, which is all related to the business. It's all related to the business. They basically said, "They changed business priorities on us," or, "They lacked business stakeholders, ownership." So imagine, when you think about what DevOps is proposing as a methodology and getting the business involved earlier in the cycle, think if that goes down 5, 10, 15%, how much lower that 19% number's going to be.

And so I think there's two things to consider. One of which is DevOps helps solve some of these underlying themes, challenges. But number two, that business managers, executives should be held accountable. They should be held more accountable for understanding a little bit about why they're being pulled in early, understanding the role of technology in their ability to execute their strategy, and also recognize that this can really drive down the 35% number.

We also looked at DevOps expectations within two years. So these are averages based on the surveys. Average number of application deployments will double.More than 20% faster time to market is expected from those services, and 50% of the Global 1000 is going to have some type of dedicated center of enablement or DevOps team. Again, we're looking at more numbers.

We've asked if you have a COE, a center of enablement, center of excellence in DevOps, where's the impact been? Where's the biggest impact been for your organizational change? And there's five key points have come out of the interviews, one which is operations. We've got new tools, we've got accelerated automation deployments, and we're really looking at driving further configuration across teams and asset management and domain expertise collaboration.

Pretty expected. The speed of success and acceleration of IT project success. So Disney had a great point a few hours ago around the better your IT management buy-in, executive management buy-in, not only does it accelerate success, but it greases the skids for further success and acceleration of that. The decrease in shadow or stealth IT, and then the fifth one, which hasn't really been talked about yet, which is allocation-based pricing.

So a higher recognition of how much is that service going to cost you to deliver, and who should get involved as you think about this new delivery model. The role of security and compliance. One of the key things we're finding is right now, most is ad hoc. Compliance came up earlier in one of the earlier sessions.

Security, not so much. But I would submit that over the next two or three years, security is going to exponentially increase in the number of conversations you have in your DevOps practices and DevOps-led projects. We're finding that earlier involvement with auditors, compliance teams, and security teams are critical. Get that trust with the auditors.

Recognize you're going to have to build that bridge. Understand the preexisting security requirements. Recognize that they already know what they need, so approach them and talk their language. You're going to have to learn some of their language.

You're going to have to understand where you're coming from. Language is a key, again, the translation theme. Cost model, cost per service. Recognize what those costs are as you're building out this broader end-to-end theme.

And then the other key thing that's come up, API wrappers. There's this big explosion in APIs and really thinking about how security and compliance fit in the overall integration story of APIs. This is going to be one of the key things that continues to draw a lot of interest over the next couple of years. Help.

I always need help. We're doing a survey. If you're interested, we can get you that link. Feel free.

You'll get a copy of the research. Again, we're into quantifying trends within peers. We've had a lot of people say, "Keep the unicorns out." How to move legacy infrastructure into DevOps. I would submit, is this the next TQM iteration?

Total quality management. If you haven't seen that, I think there's something here with regards to DevOps quality management. Thanks a lot. Appreciate your time.