Ross Clanton

First, I just want to start off by saying how excited I am to be at this conference. This is literally my favorite conference of the year, and it has been every year for four years. The reason is, I leave inspired every time I come here. I have new ideas. I have new connections. I have new friends. I have new people to collaborate with. It's just super powerful. It engages you, and it gets you excited to go and continue to drive transformation inside your companies, because that's really what most of us here are doing, if not all of us.

First, let me start. Our talk here is "Fear Does Not Exist in the Dojo." That's a riff on The Karate Kid. We're going to talk about a couple key aspects of our DevOps transformation at Verizon here today.

Gene already did a great intro. Thank you, Gene. But I will just add, I lead our engineering practices and platforms group at Verizon, and we're responsible for driving a lot of the technology transformation across the business. Nanda, I had the pleasure to meet him shortly after I joined Verizon about a year ago, a little over a year ago. What was exciting about Nanda is he actually worked in one of our lines of business at that time, and he was doing some really innovative stuff that Gene just referred to, and he's going to share that today. We were able to convince Nanda to move to more of a central role and work with us as part of this group that was driving transformation across. So very excited to have him onstage with me this year and have him really tell that story.

Who is Verizon? I think many of you probably know Verizon, but let's start with a quick video. If we can cue video.

Every year, companies take the stage and unveil the future. Phones that read our minds. Watches that make us healthier. Apps that help us find true love. The future they've promised the world sounds amazing.

But who's delivering on all these promises?

We are.

Phones, apps, tablets, GPS, cameras, watches, dog collars, that guy, those things. We give these things the power to do what they're supposed to, every single time, without fail.

When phones find new ways to keep people connected, we deliver it. When cars are designed to drive themselves, we help make it happen. And when something promises to do things we haven't even imagined yet, bring it on.

Whatever the future looks like, whatever the digital world promises, we will deliver.

All right. How cool is that? What a fun company to work for. We are solving some amazing technology problems in the industry, and that's probably a good intro on Verizon. I'll just add a little bit.

We're a very, very large enterprise. We're a multi-line-of-business company, Fortune 14 in the Fortune 500. Hundreds of thousands of employees, about 20,000 of them in technology and IT. We've got core network and connectivity businesses, which is where Nanda and I focus, working across those different businesses, but we also have media content and technology businesses like AOL and Yahoo as well.

Let me just share a little bit on the journey we've been on. A lot of this talk is going to be more about where we're going and what we're doing this year. But the path we've been on the last few years: one, we've had this move over about the last three years to DevOps, and the goal this year, we'll have over 1,000 applications that are on common DevOps pipelines, tooling, et cetera.

Now, we're still again doubling down and going back to really work out the culture and practices to further scale that in the organization, but very broad adoption from a technology and tooling perspective.

In the last year or two, we've also had a very aggressive move starting into the public cloud, and we're just starting to get production applications in the public cloud. I'll share a little bit on that, but if you want to really hear about it, come talk to us at re:Invent.

I'm going to talk about the four tracks or four pillars to our transformation program here at Verizon. We'll go into each track a little bit, but we're going to deep dive into a couple of them.

First, enabling technology excellence. That's things like our platforms, patterns. How do we get the right tooling and platforms from a cloud and DevOps and technology perspective in place across the organization?

How do we scale our engineering practices? We have thousands of engineers, and we have to modernize skills across thousands of engineers. So what's a good strategy for that?

Fostering culture, tech culture, DevOps culture. Critical. It's the underpinning of any of these transformations. It's critical to have focus inside your organization, so we'll talk about that.

Then what we're starting to do to transform our operating model. There's a lot of work left for us there, but I'll share a little bit about the plan forward.

Real quick on technology excellence. This talk isn't going to be about the tech and the tooling, but I'm just going to set a little bit of context on it. We have a pretty familiar toolchain. Most of you probably use most of these same tools.

What's interesting for me, especially having come from another large enterprise prior to this, but Verizon is even a much larger scale. We have millions of test runs and code builds already this year, and we're averaging about 60,000 code check-ins a month.

If anything, we're figuring out how to operate these tools at a very, very, very large scale, pushing the boundaries in some cases of how these tools have been used. We have over 20,000 users on some of these tools within the enterprise, and this is our standard shared service tooling for the organization. There's also cases where teams are building their own pipelines, but we have most of the company across multi-line of business using this common pipeline.

Real quick on cloud. Our cloud strategy is largely about enabling business agility and reducing cost of infrastructure. Those are really the two main purposes on why we're doing that. We are going pretty aggressive on that. We have lots of data center workloads that we're working to move into the cloud over the next few years.

We are a sponsor at AWS re:Invent in two weeks. We have multiple people talking about that story and where we're going. We have a booth there. If you want to come talk to us and hear more about that, please talk to us there. I'd need a whole other talk to talk about our cloud strategy today, so it's really not going to be about that.

Let's talk about things that I am really passionate about and really why I wanted to get up onstage and talk to you all today. First, I want to talk about how you scale skills and practices across thousands of engineers, and what's a strategy for scaling.

Let's talk about dojos.

I've had the pleasure of now implementing this model in two companies and collaborating with a bunch of other people that are starting to do this, which is awesome. That's one thing that's been really exciting for me at this conference, is I'm connecting with more people that are already starting to build these models, and I'm going to explain what this model is in a minute, or they really, really want to.

As we get into this, if anyone wants to connect with us after and learn more about this, please reach out to me and we can talk dojo.

I'm going to do one more video because I think it does a really, really good job of explaining what the dojo is, and then I'm going to deep dive a little bit into what this model is all about. If we can cue the next video.

The dojo is an immersive learning environment. It's modeled on a Japanese concept of the place you go to practice and master your skills. It's a service that we're providing to the enterprise that will allow them to rapidly upskill their teams.

We're bringing teams in for extended periods of time, working with coaches to learn new tech skills while they're actually building the products that they own for the organization. The skills that we're largely focused on teaching are around things like DevOps, which is changing how people are delivering and automating technology, agile, cloud skills. Building that well-rounded individual is really what we want to deliver to the teams.

They come in and do a chartering session where all of their core stakeholders figure out the goals and objectives for the engagement, work through a skills assessment, and after we get through chartering, they do these rapid, iterative sprints to focus on building value. They're getting a lot of practice and repetition and learning, which leads to mastery.

Teams have to learn how to break their work down so that they can build a little bit, test it, demo it with the customer, quickly get feedback, and know if they're on the right path.

We ran into some of the challenges in the first two sprints that we had, which gives us the flexibility of trying something. If you fail, you fail fast, and then you move forward.

The speed to market has been our continuous challenge, and dojos and the DevOps model definitely helps us accomplish that. This would be one of the biggest benefits to business.

The dojo really puts them in an environment where they actually work far more cross-functionally. The main benefit from my perspective in dojo, I actually got so closer to my team members. They understand the product better, understand what needs to be delivered better, so it improves the overall quality of what goes into production.

Our culture here is very different. The open workspace and the open mindset, you'll feel the difference as soon as you step in. Come on, a little bit louder. A little bit faster, a little faster. This is very exciting. I love new technologies, and I love learning all the different things that can optimize our process. In six weeks, you can actually accomplish realistic goals.

All right. That's also an internal marketing video, so you guys won't be able to search VZWeb to learn more about that.

I hope that gave people really good context on what dojo's all about. I spoke here a couple of years ago about this model at Target, and so you can definitely hear some of the pattern and playbook if you want to go back to that. But I'm going to zero in on how we're doing it at Verizon.

One of the things that really excited me about coming to Verizon is the scale, the global scale of the company, how many lines of business there are, the complexity that that means in terms of trying to drive transformation and change in the organization. One of the key things I came in to do was establish this model.

This year, we have stood up five dojos globally, two in India, three in the United States. A couple of them stood up in Q2 of this year, and a few of them just stood up here in the last few months. It's still pretty early on, but we're now at a point where we can start really scaling teams going through.

Even though it's early on, we've already put about 30 teams through these dojo challenges, and we've got a backlog of teams forming. We're seeing great results with teams already, and I'll share a little bit of that in the coming slide as well.

That quote at the bottom is actually from one of the first teams that went through. It was from an engineer on that team, and I think it's a pretty powerful quote: "The dojo has transformed the way I work by creating a process where we work as a team rather than the silos we typically work in. Our quality has increased because we are openly sharing our thoughts."

A lot of the dojo is about building better technology practices, but a lot of it is honestly getting teams to learn how to collaborate and work differently together as well. In many cases, that's a very powerful result we get just from teams coming through.

A little bit more on the process. The dojo, the way it works, and you heard a little bit on the video, but the normal engagement model we do is teams come in with a real backlog, the real work that they're trying to deliver for their product, for their application. They bring the work in for six weeks, and we pair with them, we coach them, and they are dedicated and co-located there for six weeks.

Yes, they are delivering something, but the focus of the dojo is about learning and about using that context of them doing their real work to drive learning new skills and practices into the way that they're delivering. We put them through an accelerator. They do two-day sprints, so there's two sprints every week, 12 sprints over a six-week challenge.

Part of the reason is you want to teach teams how to break their work down, how to deliver that end-to-end. We want to get as much repetition in doing that while we have them, because again, you build skills, you build mastery when you do that.

You can see a list of some of the practices we focus on there, but that's an ever-changing list. It's very much agile and product-focused in everything we're doing, but then there's these key technology practices around DevOps, cloud, API, et cetera, that we also focus on.

Outcomes: we're seeing teams improving their skills, as I was talking about. We've actually helped teams move to full stack models. Not every team has done that as an outcome, but some teams do. They're accelerating. They're getting faster time to value. Often, even by the time they leave the dojo, it's pretty measurable.

The results across the 30 or so teams, the average results we've seen, most teams are improving about 30% to 50% cycle time just over that first six weeks, and that's building better practices around CI/CD and testing and even collaboration between product owner and engineering so that that work's happening faster.

Teams are saving money, both in how they're doing their delivery and how it's enabling the business results in the dojo as well. We've already hit almost 300 engineers. We've already upskilled about 300 engineers in the company. We have a long ways to go. We have thousands, but like I said, we're scaling that this year. We're going into next year.

What kind of challenges have we had doing this in this large, complex, multi-line-of-business, global model?

First, these are different constraints. When I work with different companies and collaborate with companies that are thinking about this right now, I think there's some good best practices and patterns around how to do a dojo, but every company has its own constraints, and I think you have to figure out how to adapt this model for your company.

One big adaptation that we had at Verizon is we are very geographically distributed. Even within a team, people are often spread in many, many locations. Getting people co-located for six weeks all on the same team sometimes can be a challenge. For some teams, we've been able to do it. For others, we've really had to start thinking about different offerings around how do you get remote people in, and different collaboration tech to enable that.

We have an offering we're standing up we call Dojo in a Box, where we're going to send the guidance and materials out to different locations, possibly with a coach or possibly with just guidance for those teams to do it themselves.

Integrating global staff. A lot of the work in our current operating model flows between the U.S. and India. They aren't all necessarily autonomous teams in each location. That's where I would love to see us get to with our operating model work. But figuring out how to integrate the India dojos with the U.S. dojos is something that we're continuing to work through so that we've got a better global experience.

Biggest learning, though: aligning the purpose of the dojo. Most teams, when they come in, they see this as an opportunity to just get work done faster. The managers, they're like, "Yes, I can put my team in with some of your experts, and in six weeks, you're going to deliver this faster than we would have, right?" And the answer is no.

Very rarely are you going to deliver faster in six weeks because the purpose of the dojo is learning. It is about learning, and we prioritize that goal above everything else. What that means is you have to slow down to speed up. We often have to really align leaders on that idea that, hey, over the next six weeks, you're not going to deliver as fast because we're teaching you all this stuff, and you got to slow down to learn these things. But you're going to get faster as an outcome of this, and that's something we've seen time and time again.

The other challenge is we have multiple lines of business. We have different states of maturity. We have teams come in on one end of the spectrum where they're learning very basic engineering skills, and on the other end, they're pretty advanced already from a CI/CD perspective. We have to adapt. What we do in our model is we adapt our coaching approach based on where the teams are at. When we charter a team coming in, we figure out where things are at there.

Then dealing with tech debt. Again, that goes into making sure there's time for learning.

Real quick on operating model. My hope is next year we're coming back talking about operating model transformation. One thing I'm really excited about here is more people are talking about product model and what they're doing. This is an area we need to keep focusing on as a community because there's just not enough guidance out there on it, and it's really, really hard to do operating model transformation. It's really hard to move to a product-based structure in enterprises.

For us, like many of you, we traditionally have been siloed, focused around activities, not focused as much on business outcomes, but we're moving our operating model to support that. This is an effort we've actually just really kicked off this year.

Our plan, our strategy is really four tracks to that operating model transformation. One is actually tackling the funding model, which is the last mile, I feel like, in some of these product model transformations. How do you actually change the funding and prioritization model to enable moving to full stack and outcome-based teams?

How do we figure out our product taxonomy across the organization? How are we going to map things across different portfolios?

Org change management is huge in this space. How do you drive that across thousands of people in your organization?

Then how do we align it with our technology modernization strategy and our architecture strategy? Because it's a lot easier to move to these product-based team structures when the architecture is decoupled and actually supports it. I've seen it work in monolithic as well as microservices, but it is a lot easier in the microservices space.

Let me talk a little bit about culture, and I'm going to hand it over to Nanda here in a couple minutes.

This is, to me, the most important part of DevOps transformation. I'm often labeled the culture guy because I beat this drum constantly wherever I go. You got to get people excited and empowered about changing and about the way they're working, and you got to figure out how to get the bottoms-up going and the tops-down going. I think that's the best way to make these transformations work.

I've had a lot of experience now running internal DevOps conferences. I've run across two companies now, over 10 events, over 10,000 people. Just at Verizon alone we ran five this year, 7,200 people, nearly 90 speakers, some external folks that are part of this community. But most importantly, it's about getting all the internal people that are doing things and changing out talking with each other and building community and learning from each other.

This is all about building community at the end of the day. When you're a large, siloed company and you've got multiple lines of business, culture largely is how do you get the collaboration happening across these silos? How do you actually get the engineers talking to each other? This is a really powerful event in enabling that.

We do it every quarter, and you can see some of our stats there.

The one thing I'll just leave you all with if people are interested in running these, I know I've collaborated with folks that are now doing these conferences internally as well. Like I said, it's about building community. It's about sharing those practices, those DevOps practices, and ultimately it's about how do you build this learning organization.

The quick steps to get there: you need an awesome event organizer. You got to figure out a great space. It's best when this is volunteer-driven, so figure out how to get people that are excited about doing this inside your company.

Get the right speakers. Get external speakers that are going to draw a crowd and get people excited, and there's plenty of them out in the DevOps space. There's great speakers out, both at this conference and many others, but get people internally to start sharing and talking. That's really the important thing.

Open it up to anyone. We had 3,000 people at our first one at Verizon. It was crazy, the scale we had at that conference, most via webcast because again, we're a remote company. Market it like crazy, make it fun, and just keep doing it. Use that to fuel the culture and the change that you're doing.

With that, I'm going to hand it over to Nanda. Nanda's going to share a little bit on this slide, but he came up with this really innovative idea on the DevOps Cup, and I'm so excited that we're sharing that here today. I want to give Nanda a little bit of time here before I chew it all up, so I'm going to give it over to him and let him go.

Nanda Kumar

All right. Thanks, Ross. Good morning. Hope everyone is having a good time at the conference.

Here's the reality. Especially, I'm assuming most of you are from large enterprises. The reality is some of the things that we learn about and hear about at these conferences sounds great, but then when you go back to your office and you start looking around, you got to deal with a lot of legacy applications and so forth.

One of the things that we wanted to give it a try was think about gamification and how do we use gamification to influence human behavior? That's what the DevOps Cup was about. Actually, this is a rolling cup. It's a physical cup that a team wins at the end of the year for showcasing mastery in terms of DevOps practices and so forth.

What I wanted to maybe take the next few minutes is to talk through why we did gamification, and what was the outcomes that we saw coming out of it, and then probably leave a few tidbits in terms of if you see value in this and if you want to embrace some of this going forward in your organizations.

First up, if you look at gamification, one of the problems, and Ross mentioned this a couple of times, we are a diverse organization. We are pretty much in multiple locations within the country, and we're also spread across the world in different countries and so forth.

When we initiated our DevOps initiative, we had to figure out how do you do this not just in one small pocket somewhere in one location or one team, but how do you scale this so that it's broadly across the entire organization? That was the fundamental driver for why we thought gamification would be a good way to have the teams get interested about this initiative.

The second aspect for why we went down this path is tools. This is a very sensitive topic if you go talk to any developer because the last thing that you want to have a conversation with the developer is to say, "You got to change your tool, and you have to use a new tool."

Instead, what we wanted to do with gamification was to say, all right, let's define the high-level guardrails of what we want to achieve with this initiative and let the development teams figure out, if they have to reach or achieve these outcomes, does the current tool satisfy the needs? If it does, great. If not, then they start gravitating towards some of these tools that makes it easier for them.

Just to give you one example, and this is probably a common thing with large enterprises, you have lots and lots of COTS products. We had the same amount of COTS products within our ecosystem. One of the teams actually went about, pulled in the vendor, sat down, and said, "You know what? We need to achieve these outcomes. We want to do CI with the particular product."

The COTS product did not have the capability, but because they wanted to participate in this event and be part of this gamification event, they worked with the vendor to build some of these capabilities into the COTS product.

The point to be made here is that when we focus more about the outcomes and let the teams determine, hey, what's the best way to get it, then you start seeing the changes needed, and in some cases, they start gravitating towards the tools which makes the job easier for them.

The third thing that we wanted to focus on on the gamification aspect is, and this is a common theme you all heard throughout this conference, which is you need to have leadership buy-in. But is there a different option? Is there a way where we could create a grassroots movement where it almost becomes like an uprising, where the leaders start noticing, "Hey, the teams want this. They want to do it this way. Maybe we should start to figure out how do we support it?"

That was the third driver behind why we did the gamification, was we wanted this to be like a grassroots movement throughout the organization and have the teams come up with ways in which they wanted to embrace the DevOps practices.

Finally, the last thing that we wanted to focus from a gamification standpoint is how do you collapse time? That's going to be the number one thing all of the executives are going to focus on. All right, this sounds great, and how do we make this happen quicker and faster and so forth.

How do you unleash the human creativity of solving for these problems rather than bounding them in a structure saying that you got to go do this training first, you got to do this next, and so forth? Instead, let each team be creative in terms of how they do it, and then you provide a platform for them to come and share and showcase how did they achieve it, and then other teams learn from it.

These were the four driving factors for us in terms of why we did the gamification and how we introduced the DevOps Cup.

What changed for us? When we started this initiative back in 2015, that was the very first time we put this into place for one of our business units. Then the next year, what we wanted to do was we shifted gears where we first focused on each team gaining their skills around CI/CD practices and the DevOps practices. The next year, we focused a lot more on collaboration. We want to make sure the teams are good first, and then they start sharing what they have done and how it works and so forth.

Once we got this in place, and now this year, we actually have gone and spread this across all the lines of business within Verizon, and we wanted it to be, going forward, much more repetitive, where instead of doing this like a yearly event, we want it to be more like a quarterly event so that gives us more insights and innovation to happen much faster.

In terms of the insights that we got, now we have the ability at a portfolio level to integrate and get insights into how teams are doing. I'd like to do a call-out to Topo and team at Capital One because we are collaborating with them on the Hygieia dashboard and seeing how do you get the view, not only just for an application team, but aggregating it at a portfolio level so you can see things from a VP perspective or a CIO perspective, so they can drill down and see how their teams are doing and where do they need help and support.

Now that each of the teams actually have their own Hygieia dashboards for the different applications, one of the things I like about the Hygieia dashboard is the product view. It's an excellent view of how your different components of your applications are working together, and the fact that you can track every commit and seeing how that commit is progressing towards the different stages of your software development life cycle, and what do we have to do to improve the velocity.

That's been one of the biggest benefits teams have gained from this, and they've started to now figure out how do we push the needle forward and what are the things we have to do from automation and so forth. Again, the point being is they are determining what is the friction point they have to solve for instead of us being prescriptive saying, "Okay, here are the five things that you have to do one by one."

Finally, from a Cup experience, what actually happens is, this is a yearly event, and as part of the event, we have something called the Tech Fair, where we allow teams to come in and they showcase what they've done, and it allows other teams to also participate and so forth.

Just to call out a small story on this one. When we did our first Tech Fair, my boss was nervous because we had actually invited the CEO of a business unit, and we had 15-minute slot for him to come in. We wanted to make sure that, hey, that we have invited the CEO and we just want to group up on this.

One of the interesting things that happened is while he only had 15 minutes, when he started to engage the teams and started to understand what they're doing and how this is going to impact the business and how it's going to transform how we deliver software, he literally canceled his complete morning schedule and he spent the rest of the morning with us trying to understand what this is and how it works and so forth.

The point here is that when we talk about leadership support, this is another way that you could get that. By showing what is possible, the leaders start getting engaged and then the transformation starts taking place. In fact, right after that session, he had arranged for a one-day workshop for all of his executives where we walked them through what is agile, what is DevOps. This is for the CEO, the CMO, the CFO, and so forth.

This can provide you the grounding that you need to get this initiative going and get the support that is needed.

The other thing that we do as part of this Cup is also we invite folks from the community to come and look at our applications and give us feedback on what we have done, what's the opportunity to improve, and so forth. You can see some labels out there. From time to time, we try to broaden the scope of this so that we can get more insights and more feedback on the overall event.

Finally, I will leave you with this. If you're looking at gamification, I think the basic model here is that defining the constructs of what you want to achieve and let the teams innovate towards achieving those outcomes. Then when you provide a platform like different types of events, like flash builds and Tech Bytes and so forth, it then allows these teams to collaborate and so forth. That's been our model for how we run this event.

Finally, I'll give it back to you, Ross, to talk about what we need help with.

Ross Clanton

All right, so this actually does conclude our talk. We would love to talk with you guys about any of these things that we're driving: dojos, DevOps Cup.

Where do we need help from the community? I know we're supposed to give one, but we gave three.

Product-based operating model. This has been on my list for a couple of years now. We got to figure out how to solve these problems as a community, and like I said, I'm excited there's more talks, but I want to learn more there.

How do we better gamify and measure engineering productivity? There's other people actually doing something like this, which was very exciting. We actually got connected to some other companies doing this here. I think there's some growth that can happen in the community there as well.

Then public cloud transformation can be really, really hard for large IT and large enterprises. We'd love to collaborate and learn more about how others are tackling that problem as well.

With that, I know we're a little bit over, so I'm going to wrap. Thank you.