Steve Brodie

All right. Good afternoon, everyone. My name is Steve Brodie. I'm the CEO of Electric Cloud. I hope everyone enjoyed the morning keynotes this morning. I thought there was some excellent content there.

My talk today is on the yin and yang of DevOps transformation. I'm going to talk about a lot of the dualities that you see in DevOps as you undergo the journey.

So just to get started, this is my picture off my website. I have a suit coat on. I don't know that it's really fitting for the subject, so I thought this might be a little bit better with the saffron robes. My marketing team wanted me to wear this up here, but since it's being live-streamed, I don't think I'm going to do that.

So let's go ahead and jump in.

To start, who's familiar with this symbol? All right. Most people are familiar with the symbol. This is the Taijitu. I'm probably not pronouncing it correctly, but it is the Chinese symbol for yin and yang, and it's the universal symbol for the Taoist philosophy. It first appeared in the I Ching, written in 1000 BC, and the I Ching is the Book of Changes.

What this represents is seemingly opposite forces that are actually complementary, and it means that the whole is greater than the sum of the parts. There are a lot of dualities that are common in dynamic systems, where forces must interact to be in harmony.

So this is what Dev and Ops has traditionally looked like. If you turn the symbols around, is this what it's been like in many organizations you've worked in? Dev and Ops has really been seen as opposing, where Dev is focused on innovation, on change, and Ops is focused on stability and longevity.

So Dev sees Ops as slow, resistant to change, and Ops perceives Dev as dangerous. It creates a lot of conflict and tension.

Excuse me for a second.

Okay. If you look over time, what's happened is walls have built up. This is the Great Wall of China, and this is what's happened over time. The Great Wall got built up over thousands of years, 6,000 kilometers long, and this is really what the environment has been like between Dev and Ops historically.

But the reality is, if you look at Dev and Ops, they're not opposing forces. They're actually complementary forces. They're connected and interdependent, so they have to work in balance and harmony for the whole system to actually be effective.

If you look at Gene Kim's book, who's read some of Gene Kim's books? Right. So there's the Three Ways of DevOps, and I think we can map the Three Ways of DevOps onto this symbol here.

DevOps is really all about systems thinking. If you look at the silos and you optimize for the silos, the whole system does not get optimized. It's really looking holistically at everything. If you look at this symbol, the circle represents the entire system. It's an organic fit. It works together very effectively.

The second way of DevOps, if you have read Gene Kim's book, is all about feedback loops. It's all about getting feedback one way and the other in a bidirectional area. These are very interdependent. Dev provides applications that run on the infrastructure. When it's running, you measure, you provide feedback back into Dev.

The other aspect, the third way of DevOps, is continuous learning. Continuous learning is all about iterating. It's doing small experiments. It's measuring. It's then making changes. If you look at the symbol, it doesn't look static. It conveys rotation. It looks like it's in motion, and that really is all about iterating, about learning continuously.

And there's a fourth one. This is one that I'll add here. This is not one that's specifically in the Three Ways of DevOps, but one that I think is important. I think it's shared mindset. I think that came up a lot in some of the sessions earlier today.

If you look in the yin and yang symbol, you see these dots of opposing colors in the other side of the symbol. For me, what that represents is that you have common objectives. You look at things with a different mindset. If you're a developer, you want to look at things from an Ops mindset. If you're in Ops, you want to understand some of the goals of Dev.

So I think this symbol does a great job of representing high-level DevOps.

Okay, so let's drill down a little bit. I want to look at some of the common dualities that you see in DevOps. In this particular case, I've rotated the symbol a little bit, and you've got culture and people on the top, and you've got process and tools and technology on the bottom.

One of the things you're going to hear a lot this week, I think we heard about it even early this morning, is that DevOps is not exclusively about tools and technology, and it's not even primarily about tools and technology. There's plenty of tools and technology out there to achieve some of these things. Culture is equally, if not more, important.

You've got to get the culture right before you can start focusing in on some of the process and tools. It really is culture first. It's people first. If you don't address this, what you're going to find is that the investments you make in tools and technology are not going to be as effective, and the whole system is not going to be optimized.

John Willis and Damon Edwards coined the acronym CAMS back at the first DevOpsDays in 2010. CAMS stands for culture, automation, measurement, and sharing. If you look at it, it's noteworthy that culture is first, kind of what we talked about before. Culture comes before automation.

Again, if you don't focus on the culture, if you don't break down some of the silos, if you don't address things like shared objectives, what you're going to find is, again, siloed automations and local optimizations across the system.

Once the cultural change and some of the people changes are in place, you can then start focusing in on automation, and then you can start focusing in on the aspects of continuous improvement, which is really around measurement and sharing along the way.

So let's look at some common success patterns around culture and people. One is cross-functional teams. This morning a lot of people may have heard people talk about the importance of having cross-functional teams. There are many different organizational structures that you can have, but this is critically important.

Now, this is from the movie Kung Fu Panda, produced by DreamWorks. DreamWorks happens to be an Electric Cloud customer. It illustrates the power of diverse teams, and it also has a representation of yin and yang.

If you weren't aware of it, the panda is actually revered in China. It's considered the physical manifestation of the yin and yang. It has the black and white pelt, and it has the dots on the eyes. That was one of the reasons it was chosen as the main character.

In this particular case, Po is the main character. He's a big fan of kung fu, but he's pretty clumsy and so forth. He starts training with this other group, and working with this other group, the Tigress, the Crane, the Mantis, et cetera, is able to defeat the evil snow leopard and become a dragon warrior.

Research has proven that diverse teams are significantly more effective than siloed teams, so this becomes very important. Again, as you heard this morning, you may have different structures. You may have a hard-line QA functional team. You may have an Ops team. You may have a shared service team. The important thing is that you put together a group that acts as a cross-functional team across that.

That's proven to work much more effectively. You avoid groupthink when you do it. It fosters empathy, and it can overcome some of those siloed mindsets that you tend to see.

The other thing that's really important is shared mission, shared values, shared objectives. If you have a divergent mindset, if you have divergent objectives, you can actually cause many different risks.

This is one of the ones that we talked about. When you have the objective where Dev is measured on just delivering functionality, but not worried about Ops and uptime and so forth, that can cause problems. If you've got a situation in Ops where your whole focus is on uptime, on service-level agreement, you may be resistant to some change.

So it's important across the overall system to have shared mission, shared objectives. What behavior do we value? What are the shared goals for the business or the overall application team? What is important? What are some of the shared objectives? How are incentives structured across the team? Are there shared incentives across the various teams?

I just wanted to give an example here from Electric Cloud. The company was founded in 2001. I joined in 2012. When we did that, we set out on a new strategy to really focus on DevOps, and we felt it was important to revisit our mission and to revisit our values.

Our mission is that we help the world deliver better software faster. This is really where we wanted to be. This is where we had been.

Then we started focusing on our values. This is interesting. It also represents the importance of cross-functional teams. The executive staff got together, and we started drafting some values, and it's the traditional values you see on a value statement. It was a bunch of nouns. It was kind of cliche.

We put together a cross-functional team with representatives from each of the various functional areas on the team, and what we came up with was a lot better. Instead of nouns, the team came up with a series of verbs. It was behaviors that we wanted people to represent.

It was also reflective of DevOps. If you look at the values that we have here, listen, act, deliver, measure, share, this is very representative of the DevOps life cycle. For us, everything has a duality as well in terms of it having an internal meaning and an external meaning.

Listen externally means listen to the market, listen to customers. Internally, it means to have empathy and to understand someone from another organization, to understand some of their point of view. Act means move quickly. Don't wait. Don't get caught up in analysis paralysis. Deliver is very important. Then you see the measurement and sharing. That's the continuous process improvement.

Measure means internally measure how we're doing, but also externally measure what's going on in the marketplace. Share internally so we can improve internally, but also share back with the community as we go forward.

This is what we came up with. The cross-functional team came up with something that was much better, much more effective, and this is now something that all of us at Electric Cloud can rally behind.

So we've talked about culture and people. I want to talk a little bit about process and tools next.

Again, culture and people needs to come first, but then you can start focusing in on automation. There's a duality in automation as well. If you look at automation for DevOps, you have two major components. You've got your infrastructure layer, and you also have your application layer. These are, again, very interdependent.

Applications can't run without infrastructure, and infrastructure doesn't really provide much business value without the application that runs on top of it. So again, these are interdependent, connected, and they have to work together seamlessly.

Now, when you look at the DevOps tool landscape, it can be very confusing if you're just getting started in DevOps, because it's a very hot market. It's like cloud circa 2007. Everyone is saying they have a DevOps solution, and it can be very confusing for customers in terms of where various tools fit.

The other thing is that there is no silver bullet. There's not one tool or product that you can buy to achieve DevOps. It really is something where you need to have a best-of-breed solution.

We look at the tool landscape in a matrix where you've got Dev on the left, you've got Ops on the right, you've got your traditional activities: develop, build, test, deploy, provision, monitor, operate. But again, we put that horizontal segmentation where we have infrastructure on the bottom and apps on the top.

Then you've got your common tool areas that you see here in this environment. First at the bottom, on the infrastructure layer, is software-defined infrastructure.

One of the great enablers of DevOps has been software-defined infrastructure, cloud, software-defined networking. This actually helps remove a significant bottleneck to moving faster. In many organizations, I used to hear that it would take 40 days to procure a server, get it set up, get it configured. So you can't move very fast if you have that kind of infrastructure environment.

Things like virtualization, VMware, and OpenStack, things like public cloud, like Amazon, and containers, allow you to programmatically provision infrastructure on demand and tear it down.

Once you have the infrastructure in place, you need to configure it. You need to make sure that all the infrastructure is in the correct state. So configuring the servers, configuring all the middleware software that you have out there. There's a new generation of configuration management tools that you see out there.

You've got Chef and Puppet. It's model-based, it's declarative. You say, "This is the state that I want my infrastructure and middleware to be in," and then it makes sure that it remains in that state.

But again, these tools, although they're very important for DevOps and very key enablers for DevOps, don't really address the application layer. So there's another set of tools. Wide variety of Dev tools, things like Atlassian, things like GitHub, who's a sponsor here, Rally, many Dev tools that are out there.

Once you deploy an application in production, there's a number of tools that you see there, too. You've got tools to manage the application, to monitor the application from companies like AppDynamics. You've got to service that application, so IT service management solutions like ServiceNow.

Once the developer checks in code, there are CI solutions like Jenkins that can help you manage the build, automate the build, automate some of the unit testing. But there's still been a big gap after that.

After you check in the code and after you have the build, there's a huge part of the software delivery process, the path to production. Automating everything after the developers check in the code and create the build, that is where Electric Cloud fits in, and you really want something that orchestrates that path to production.

What's important is to tie all these silos of automation together. What you want to be able to do is to take all the various testing tools and make it work as a seamless whole. Ideally, what you want to have is a model-based approach to defining that software delivery pipeline so it can be versioned, so it can be managed, so it can be consistent.

You need something that's going to plug into all the various tools, be able to automatically provision your environments in the cloud, on-premise, whether physical or virtual, kick off all the configuration in Chef and Puppet, and then make sure that everything gets deployed in the right environment, kick off all the tests along the way.

One area that has never really had great automation has been the step when you're actually deploying the application. That has traditionally been very much of a manual process. It's been homegrown scripts. It's been some manual effort.

So there's a new category of tools, and now Gartner and Forrester and Ovum are starting to talk about application release automation, where you can model the applications, you can model the components that make up that application, you can model the various environments, and then you have a repeatable way, you have a model-based way, you have a model that is versionable to automate the deployment into the various environments: dev, test, pre-prod, staging, production along the way.

Again, I just showed a subset of tools. There's many more tools here in the category, but hopefully this helps clarify where things fit in the landscape. Again, the important thing: there is no one tool that you go buy to do DevOps. It really is a combination of best-of-breed tools to orchestrate everything together.

Okay, so we've talked about the landscape. What we hear a lot from our customers is, "Where do I start?" If you look at it, there's so many areas for automation. You saw the landscape that I laid out there. There's some paths that look a little bit easier. There's some paths that look a little bit harder and a little bit steeper, and it can look pretty daunting. So where do you start?

There's a quote from Lao Tzu, sort of the founder of Taoism, sort of the writer of the I Ching, and he says, "New beginnings are often disguised as painful endings." I think this is very instructive about where you should start in your DevOps journey.

Some people may know where the biggest pain points are in the organization. They may know where it needs to be automated testing. There could be some other areas. For others, you may not know, or you may have many different areas, and you're not quite sure where to start.

Puppet Labs, annually, they do their DevOps survey. This is from the 2015 State of DevOps Report, and I think this quote is very important. What it says is, "Where code deployments are most painful, you find the poorest IT performance, you find the poorest organizational performance, and the poorest culture."

So code deployments become one of the key indicators of performance of an IT organization and a business. If you look at it, application deployment challenges are just far too common and far too painful.

This is a survey that Electric Cloud did. We had 101 respondents from a variety of sized organizations, starting under 500 million to some very large organizations greater than $1.5 billion in revenue. We surveyed IT leaders in those organizations, so managers, directors, C-level folks.

We said, "How often do you experience some of these challenges?" Seventy-five percent said product release problems happen frequently or occasionally. Inconsistent deploy process across dev, test, and prod, that was almost 50% of people cited that as a problem. Missed release deadlines, almost 40%. Stabilizing a release after it's deployed greater than one day, a third of people say this is a problem.

So code deployments are a very common problem. Gartner just also issued a survey recently. They were talking about where are the DevOps priorities, and number one on the list was application release automation, application deployment, because this is an area that traditionally hasn't been automated, and it's an area where there's a lot of benefits.

Let's look at it. Why is this such a great area to start? Again, it's a commonly cited pain area that people have. It often relies on scripts and manual orchestration. Sometimes there's manual runbooks. I worked with a customer that had a 156-page runbook for doing their deployments. They had million-dollar conference calls on the weekend where everyone would sit on the call: "DBA, you've done your task. Okay, next person do their task." Very expensive, very error-prone, very time-consuming.

Sometimes people are using suboptimal tools. I laid out the tool landscape. Sometimes what you'll see is people might be trying to use a tool like a CI tool to do deployment, or they might be trying to use configuration management, infrastructure configuration management to do deployment. Again, it's not optimized for the problem area.

The other unique thing about this is that if you do deployment, code deployment, you have to orchestrate across that infrastructure layer and the application layer. So you start breaking down some of those barriers and making that much more of a seamless operation.

It's also a great way to start iteratively. Again, we talk about iteration. We talk about small experiments and moving on. With application deployment automation, you can actually start with one application and then over time extend it out to other applications. Or you can choose to automate deployment into one environment and then over time, start automating into other environments. You may start with a testing environment, then over time, you might move to a production environment, or you may come the other way. Then over time, you can extend this.

Okay, so we talked about culture. We talked about automation. The other thing I want to talk about is this duality of looking at it from your own organization and also looking at things from the broader community.

The focus before on my organization, we talked about the things you can do around culture and people, the tools that you can get to automate. But we're part of a larger whole. If you look at the DevOps community, it's traditionally been very open, very generous, very willing to share.

If you look at DevOpsDays, which started in 2010 and really kicked off some of this movement, it's really where a lot of practitioners that were doing the early types of DevOps came together. They shared learnings, they shared some of the challenges they had, they shared some of the ideas that they had, and that really allowed the DevOps movement to thrive.

By doing this, these organizations got better by learning from others. So I think sharing again is very important, sharing inside the organization, but community is very important as well.

I think that's one of the key reasons why we're here at DevOps Enterprise 2016 in London. Gene Kim and I actually co-founded the DevOps Enterprise Summit back in 2014. We were chatting, and what we said is that we just didn't see a very good industry forum for people that wanted to do DevOps in the enterprise.

If you looked at a lot of the people that were doing it then, it was Netflix, it was Etsy, it was Facebook, it was Amazon, but there was no real forum for enterprises coming together. That's what's great about this particular event, because I think there are some unique challenges for enterprises trying to do this, where you may have some legacy architectures, you've built up silos over time, you have many different business units. There's a lot of unique things, auditability, audit challenges, and so forth.

So while you're here, this is a great forum. The sessions are great, but probably some of the best value that you have here is talking to some of the peers here, learning from others that are out there in terms of what worked for them, what didn't work for them, what are some of their challenges, what are the things that you can help them with.

Outside of this session, there are many other resources that are out there. Again, we talked about DevOpsDays. There are other conferences out there.

Also, Electric Cloud, we curate an online webcast series every two weeks. It's called Continuous Discussions, or C9D9 for short. What it is, is we do a number of topics that are relevant to DevOps and continuous delivery, and the panelists are experts in the area, or it's practitioners from organizations that are addressing that topic. They're working on it. So it's a great resource.

You can go see some of the recordings. Again, it's every two weeks. It's very, very well-received. So I encourage you to take a look at that if you're interested in learning a little bit more about DevOps and what some other folks are doing outside of the DevOps Enterprise Conference.

Coming back to the yin and the yang, one of the things we talked about is one of the key things is learning, continuous learning. It's the third way of DevOps according to Gene. Again, we talked about it. If you look at the Taijitu symbol, it does reflect rotation. It has that sort of impression. So it's about iterating, it's about learning, it's about improving as you go along.

Before we can improve, though, the important thing here is to practice the basics. So here's kung fu training that you see, and over and over again with the master, you practice things. You practice some of the basics. Once you get the basics down and you get your yellow belt, you can move up to the next level along the way.

It's very important to get the basics down and, where possible, to automate everything. One of the things that you see in Jez Humble's book is automate everything that you possibly can. That makes it really repetitive, makes it second nature, makes it boring along the way. Then once you do that, you can start advancing and trying new techniques, addressing new problem areas.

Again, when you're training here, if you're doing kung fu training, it's important to have a master, someone that's been there before, and you can learn from people that have been there before you. Again, that's one of the benefits of being here.

Now, Electric Cloud, we're honored to have worked with some of the largest and most important companies in the world on their DevOps journey, and this is just a subset of our customers. These are a subset of the customers of ours that were at the DevOps 2015 conference in San Francisco.

Again, working with these organizations, we've helped them. We've learned some things, what works, what doesn't work, and particularly, how do you do DevOps at scale? Again, because there are some very unique challenges to do it at scale.

That's one of the things we've heard over time in the DevOps conference. You see people getting started, and they talked about their journey in 2014, and they talk about 2015, and it's like, how do I scale this thing up? That's one of the key challenges that you see there.

Many of these organizations have made great progress. I wanted to share just a couple examples of the benefits that you can see when you start doing DevOps at scale.

The first example that I wanted to highlight here, it's a Fortune 25 bank. For confidentiality reasons, I can't cite who they are, but they're one of the largest banks in the world. They had a goal. They wanted to drive a billion dollars in cost savings out of IT. They were spending $5 billion a year in IT. They wanted to drive that down to $4 billion.

The way they wanted to do that was through leveraging the cloud, software-defined infrastructure, leveraging automation, and selecting tool standards across the organization. There were a number of elements to it, but one of the things they wanted to do was automate continuous delivery and application release automation. They wanted to select a global tool standard across the entire bank.

Because it was across the bank, it was across many different platforms, so Java, .NET, mobile, and many different deployment targets: physical, virtual, and cloud. The scale is pretty astounding here: 6,000 applications made up of 50,000 components running on 150,000 endpoints.

So doing this at scale is pretty impressive. It requires some unique capabilities. They're in the midst of this journey, but again, this has been established as the global application release automation standard.

In the early stages, they're already achieving 10 times improvements in code deployment time, so a tenth of the time that it used to take to do it. To onboard a new application, to spin up a new pipeline for a new business, for a new release, it used to take them days to get that spun up in the prior environment. Now they can spin up a new release pipeline in minutes. So it's very quick to set up a new environment.

Much more predictability, much higher quality because you have a repeatable process, and because you have one system that's orchestrating the whole end-to-end thing, you have comprehensive visibility of what the status is of a release, what's deployed in every environment. Is it consistent? You also get a comprehensive audit trail because it ties everything together. Again, auditors love this capability because you can see everything that's been done.

So again, this is one example. Now, a lot of the talks that we've seen here have been people that have websites, financial services. We've talked about the tax revenue service here. But DevOps is also relevant in the embedded sector.

One of our customers is Huawei. They're a $45 billion provider of network equipment and mobile devices, and they were looking for a solution to automate their whole software delivery process. Because they're creating Android phones, it's a very competitive market. You have to deliver your phones much faster. So the only way to achieve that was to implement DevOps and to automate the whole process.

What they achieved, and this is still an ongoing process because it's continual process improvement, is already a 76% reduction in cycle time to deliver a feature with this automation. Again, it's astounding scale. What these guys do on this platform is 10,000 releases per year, 100,000 builds a day, 100 million test case runs that happen every day on this, 30 million lines of code. This is pretty amazing. It's supporting 10,000 developers at Huawei doing this.

So it is possible to do this at scale. It is possible to get some pretty astounding returns. I think you saw some of that in some of the talks earlier today, and we're seeing a number of customers that are achieving the same types of things.

We're really honored to work with these customers, really happy with the results. For those of you in the room that aren't customers yet, we'd really love to work with you if you have a need around this area and help you become DevOps dragon warriors.

Thank you very much. Enjoy the rest of the conference.