Stefan Groot

Not having an IT background gave us the possibility to look at DevOps from a different perspective. And that's the reason why we're here today with our story. We want to provide you with this perspective today.

Let me first start by talking about: who is ABN AMRO? We are a Dutch international bank, headquartered in Amsterdam. We have over six million customers with almost 20,000 employees. It makes us one of the three biggest banks of the Netherlands. And we also have a long, long history, almost 300 years.

But we are quite modern today and heavily investing in digital.

So I want you to understand how we are structured, so how we are working together with the business. In August last year, 2017, we reorganized into a new delivery model, an agile delivery model. Therefore, we created squads. And squads, what we call a block.

A block is a cross-functional team. There's a product owner together with engineers who are responsible for a product. Multiple blocks together is what we call a grid, and a grid is responsible for one or more business capabilities. We have over 45 grids, and you need to think of credits, transactions, savings, or mortgages. They focus on those kinds of business capabilities.

And the way we are structuring our skill development is via circles. So a circle is a skill development group within a grid, for example, a development circle or a testing circle.

And the way we are organized around interest groups is via triangles. So this is not so much about having a dedicated team working on skill development. This is purely about interest groups, for example, public cloud or Java.

This agile structure gave us the possibility to collaborate with the business way better. So we are way more flexible, and it also gave us the possibility for continuous improvement, but also to automate pipelines based on products.

However, we still have one big challenge, and that is the handover to IT operations. And this is the journey we started one year ago.

Also one year ago, I reread the book Crossing the Chasm of Geoffrey Moore. So I'm curious, who in this audience knows the concept of crossing the chasm?

For the ones who don't know Crossing the Chasm, it shows that the phases of technology adoption or product adoption are different, and each phase of adoption calls for a different approach. For example, if you want to launch a new product in the market, you focus first on the innovators, followed by the early adopters and the majority.

We have seen that this concept also applies for a DevOps journey. We have seen that we go through different phases of adoption, and each phase asks for a different approach. And that means that one approach you have in an early phase does not necessarily work anymore in a later stage.

Therefore, we want to show you three phases of adoption we have been through at ABN AMRO. First, Mascha is going to talk about how we created the DevOps movement within the bank. And I'm going to talk about how we spread the word, and how we enabled teams to move towards a DevOps way of working. And I'm going to close as well how we are going to move to enterprise DevOps with over 500 agile teams.

Mascha Boender

Yeah. So let me start with the first phase. I'm going to tell you how we created a plan for developing a DevOps way of working within ABN AMRO Bank, and more importantly, how we got everybody involved and motivated to make it actually happen.

To be honest, it was a long shot. When we started one year ago, nothing was in place. So we needed to create everything: the definition, when teams are able to call themselves DevOps, what teams should do to become DevOps.

In addition, when we started DevOps at ABN AMRO, we had just finished our agile transition, so people were tired of change and management was hesitant to talk about DevOps.

I would like to give an example about this. One year ago, there was a town hall and the whole IT organization was present there, and our CIO was also there. People got the opportunity to ask questions to our CIO, and one person raised his hand and he asked, "What are we going to do with DevOps?"

And we were sitting there in that room as well, so we're really curious: what is he going to say? And he said, "No, we are not going to do anything. We just finished our agile transition."

And we are like, what? We were asked to start DevOps and our CIO is saying we are not going to do anything with it. So it was a challenge for us.

It felt a bit like the person who was standing there and looking at the sheep, but did not receive any reaction.

Moreover, our organization was still structured based upon the traditional way of working. So we had a separate development and operations department, with their own management, their own way of working, their own budgets. So it was hard to put these departments together.

So in sum, there was a lot against us, and we weren't sure where to start.

So what did we do? We first started with what DevOps meant for ABN AMRO. Therefore, we did two things. First, we created a general definition, which we sent to everybody who was involved. Secondly, we specified our definition into concrete practices, practices which we call building blocks.

This was captured in a maturity model. One year ago, we were working together with a large consultancy company, and they advised us to use a maturity model because they told us, if you use a maturity model, you could see where teams are in their journey. Based upon the level they are, you can give them more responsibilities.

So for example, if a team reached level two, you can give them the responsibility that they can deploy something into production without any approvals from other teams.

But when we created the DevOps maturity model, we did it based upon our own experience, the advice that the consultancy company gave us, and what we saw on the internet. But we didn't verify our approach. So that was the first thing that we needed to do.

But before I'm going to tell you more about that, I'm curious, who of you use a maturity model?

A few.

And for who was it a success?

A few. Yeah. Well, to be honest, in our company, it wasn't a success. And that was because of three reasons.

DevOps is not a fixed linear process. So when we came to teams and gave them the maturity model, they first started checking off boxes, and then they decided themselves where to start. So they did not follow the patterns of the different levels, but they were all over the model. So for us it was not clear where they are, and we could not give them extra responsibilities.

Secondly, there's no one size fits all. The maturity model that we created was not applicable for all teams because in some environments in our bank, it is, for example, not possible to automate. And the maturity model that we created requires automation for every team.

Last but not least, it felt like a control mechanism for teams. When we gave teams the maturity model, the first thing that they thought was, "Oh, this is a management document. Are they going to check how we are performing?" That was not what we wanted to achieve. They were focused on keeping management happy instead of improving their performance.

So we needed to change something. We said no to maturity models, and we replaced our maturity model for a DevOps capability model.

This capability model consists of separate, different building blocks where teams can work on to improve their performance and to become DevOps.

You can compare the capability model with a buffet. If you're at a buffet, it's up to you what you want to eat. And of course, a restaurant can recommend some dishes, but you can decide yourself where you want to start and what you want to eat.

This is the same with the capability model. It's up to the teams where they want to start with.

So this sounds great, right? But in practice, it wasn't that easy for teams to decide themselves where they want to start. When we gave the capability model to teams, they were looking at us and they didn't know where to begin. So we thought we needed to do something about that as well. Therefore, we created a moonshot workshop.

I'm curious, who of you have heard about moonshot thinking?

A few people. So for the ones who don't know, moonshot thinking is mainly used in product development and innovation, and it's about setting an unreachable goal or really ambitious target. And based upon that goal, you're going to determine all the challenges you have and try to solve them.

So let me explain how we use this in our DevOps journey.

When teams come to us and they say, "We want to become DevOps," the first thing that we do is we organize a kickoff with the team to create a general understanding of DevOps. And afterwards, we organize a moonshot workshop.

The first step in the moonshot workshop is that we are going to ask teams, "What is your ambition? What do you want to achieve?"

What teams want to achieve really depends on their context. Some teams, for example, say, "We want to deploy once a week something into production without any production errors because that's what the business requires." But there are also teams who say, "No, I want to deploy daily into production."

But for example, when you look at our transactions part of the bank, stability is more important, so they are not focusing on becoming faster, but more on: it needs to be stable. So the ambition really depends on the team's context.

But let's take an example and say, okay, the moonshot is: we want to deploy daily into production without any production errors. Then we put it on the wall.

And we ask teams, "Okay, if this is your moonshot, what is standing in your way at this moment to achieve your moonshot?"

And they come up with a lot of impediments. For example, "We are still dependent on one single person. We have a lot of manual tasks. We have a lot of technical dependencies." Then they write down all the impediments, and together with the team, we are going to prioritize them.

It's up to the team where they want to start. So they mostly make a top three of impediments, and with these impediments we are going to look back to the capability model and decide together with the team, "Okay, what are the first capabilities that you want to work on?"

Teams were really enthusiastic about this approach, because they understood why they needed to work on the different building blocks, and they were focusing on what really matters to them.

But there was one problem. Our approach was not scalable. We were a team with four people, and if we wanted to do this with more teams, we needed to change our approach. And Stefan is going to tell you more about this.

Stefan Groot

Yeah, indeed. So we created best practices out of our experiments, but we had a bigger ambition. Our ambition was to leverage the whole organization to create a bigger movement, to engage as many teams as possible. But indeed, the big challenge was, how do we scale this approach?

Therefore, Project X is a nice example. We had one as well in the Netherlands, in a small town called Haren, which has a couple of thousand people.

What happened? A 15-year-old girl, in 2012, invited some of her friends for a sweet 16 party. So what she did, she created a Facebook page, she made it public, so her friends could in turn invite their friends as well.

And that's also what happened, but in a different way than expected. Because one of her friends invited 500 of their friends, and that's the way the ball started rolling. Because the day after, 16,000 people signed up for her sweet 16 party.

And the father said, "You need to remove your Facebook page right now." But at that time, someone else already created another one. And the media called attention. So what happened? The day before her sweet 16 party, 250,000 people signed up.

And in the end, in a small town, a couple of thousand people, 15,000 people showed up.

So what did we learn? We saw a chain reaction, and it all started with one person. And other people just copy-pasted and spread the word because it was fun. It was funny to do so. That's what happened.

We tried to release that amount of energy as well within ABN AMRO. We didn't believe that we needed to convince every team to work on their DevOps capability, but we really believed in a sense of belonging. So teams just copy-pasting each other because they see that the neighbor is working DevOps. So it's fun and the cool thing to do.

So this is what we did.

Of course, we first started with giving a stage to teams, connecting teams, let them talk about their successes, about their failures, about best practices. Sharing is, of course, one of the key principles of DevOps, and it's really key for DevOps adoption. But I think it didn't bring us the required effect. We didn't go viral.

So we did something else. We attended as many knowledge sessions as possible, so the circle and triangle sessions. We did this because we wanted to tap into the networks of other people within the bank to reach a big amount of teams. And although this also helped us, we didn't go viral.

So the thing that really helped us was this small thing. And this is a hard copy, but this is our DevOps Playbook. Who in his organization or her organization is also using a DevOps Playbook?

I see a few, not so many.

So we co-created the DevOps Playbook. What is the DevOps Playbook about? Basically, it's a toolkit for teams, so they can start and build and grow their own DevOps capability.

It consists of three parts. The first part is about the principles and the definition of DevOps, so everyone has a common understanding about what DevOps is, what it means for the bank.

The second part is we created our approach, which is of course the moonshot. But we also created a canvas, a DevOps canvas. Teams can use this canvas to define and create their own journey and their own ambitions, so they can basically start themselves without having help from us.

And the last part, and I think this is the most important part, are the plays. The plays are basically the best practices we captured in the bank from all the places. We put them together in one document. And this document is not written by us only, but also by the DevOps teams, by the center of excellence, by process owners. So we basically captured everything in the bank that has to do with DevOps. We put it in one document, and we call it the Playbook. And that's fun.

I want to show you some slides. I will not stand too long still on these slides, just to walk you through, because at the end of the session, we can show you the hard copy of the Playbook. But normally this Playbook we use in a digital version, of course.

The DevOps capability model is in there. But because it's a 100-page-long document, it should be easy to navigate. So if you are a team and you want to learn more about multi-skilling and how other teams are doing it, you just click the button and you move to multi-skilling. And this is what we call a play.

A play consists of a definition and some practices which are around it. But the funny thing is, we defined a play like a user story or like an epic, even with acceptance criteria. So what happened? Teams were copy-pasting these user stories and putting them on their backlog in Jira. We could track the backlogs of the teams, so we could see exactly what teams were doing with respect to DevOps.

Of course, there's also a fun element in there. We have some MythBusters and some provoking statements, just to get attention from engineers and let them discuss about certain subjects if it's related to DevOps.

But we also gave them some nice tools in multi-skilling, for example. And again, this is just one building block. We have 17 building blocks. This is a really big document. But we just give them some tools and guidelines on what they should do and what they can do.

The reason why this was a success is because teams are motivated themselves to work on DevOps. Just given the right tools, they can start themselves. Also, the content was really, really strong because everyone in the organization was involved in creating the content, and also making it one source they can tap into in order to guide their journey.

And this is also the moment that it started to go viral, because management, even the business, everyone was sharing the Playbook with each other.

But we were missing one thing, one critical thing. If you talk about DevOps, this Playbook was primarily focusing on the hard side of DevOps, which is a team operating model. But we didn't focus that much on the soft side of DevOps.

And that's why we did the following. We partnered up with GamingWorks, and GamingWorks is a Dutch company. They created a game, so we started playing with it. They created the game, the Phoenix Project business simulation.

I want to ask you an ethical question: who didn't read The Phoenix Project of Gene Kim? He's not here at the moment.

That's a good sign.

So GamingWorks created a game based on the book of Gene Kim, and it's all about experiencing the DevOps way of working. So what we did, we started playing games with the teams. We put them together, and in one day they experience what DevOps is about, what they can learn from each other, what principles they should use.

Because Mascha played that game quite a lot of times, she could tell you some learnings we had.

Mascha Boender

Yeah. Well, it's really interesting about a game that it's not only about creating awareness and experience as a team what it is to work DevOps, but it also gives insights in the way of working of a team, the way they collaborate with each other, and the way they communicate with each other.

So what we did, we used the game also as a way to give teams insight into their team dynamics. Let me give some examples of what teams experienced when they played the game.

One thing that we saw is when they started playing the game, they created a sprint-for-sprint Scrum board, and they put all the features on it. But there was no space for incidents, for example. And during the game, teams also receive a lot of incidents.

So when the first incident came in, the team was like, "Whoa, what do we need to do? Because our sprint backlog is already full and we reached our WIP limit, but now there's an incident where we need to work on as well." So everybody was confused and they didn't know what to do.

So after the game, we had a discussion with the team, and they told us, "Yeah, we didn't make any agreements about it." And this also happens in practice. When we want to become a DevOps team, we also should take care of incidents. So we need to discuss with each other how we are going to deal with it, and how we can maybe make some place for unplanned work as well.

Another thing that I saw was when we played the game with a team, they were screaming to each other and they tried to overrule each other during that game. There was also a team coach present at that moment, and she started a conversation with the team. She explained to the team what she saw, that the people were screaming to each other, not listening to each other. The team recognized it and said, "Yeah, you are right. We try to overrule each other, but we are not really listening to each other."

So after that game, the team coach started to work together with the team to improve their communication and collaboration. So it's really helpful also to improve the way of working of teams.

Stefan Groot

Yeah. So it's more than just playing a game. It's about identifying team patterns, and we made it a part of our continuous improvement cycles.

Playing is really helpful if you're starting and growing a DevOps movement. So the Playbook really helped us in growing, and also the game facilitated us in creating awareness and engagement with the teams.

But if you want to move to enterprise DevOps over 500 teams, also this will not work. If you focus only on teams, you forget the bigger picture. And if you do not focus on removing the structural impediments, teams will get frustrated.

So the big question we face right now is: how do we cross the chasm, and how are we going to move to enterprise DevOps over 500 teams? So we are currently in this phase.

To answer this question, we had to break down with some of our strong beliefs we had in our startup phase.

The first thing is, I think I will swear right now in the DevOps church, is we need to say hello to top-down. This means that we need to have more planning and more organizing on a higher level. And right now we do it on grid level in order to make sure that we don't have 500 different journeys, but we have also an organizational journey and a grid journey as well.

But also some other things, for example, if you talk about changing contracts, about HR structures, you cannot solve it if you focus on teams only. You should focus on management as well, and on the top-down approach. So bottom-up needs to meet top-down.

Another thing is: one size does fit all. And of course, this is not fully true because everything lives in a different context, and every context asks for a different way of working. But free-for-all is also no option. If you have 500 different operating models, it doesn't work. It gives you a lot of chaos within an organization that is focused on reducing risk and control.

So what we are doing at the moment is we're creating more standards. We're creating more rules or more guidelines for teams to comply to in order to move towards the DevOps way of working. And this not only gives us the efficiency we want to have, but also helps us in reaching the majority of the teams who are more hesitant to change and more hesitant to innovate. They want to have more rules and guidelines they can simply follow.

And we're still using the capability model as our framework. Only behind that framework, we put more rules and guidelines in there, so teams understand it way better what they need to do in order to comply to our standards.

Last but not least, we expect commitment from management. What we do not expect is that they create plans, roll out plans, and they commit to those plans. What we expect is that they help the movement grow and make it sustainable.

What is also really helpful in here is playing the Phoenix Project game. And that's what we did as well. So we brought all leadership teams together in one room, and we started to play the game. So they experience what DevOps is about, what teams are struggling with, because they're playing it themselves.

And it let them see that they should focus on removing the structural impediments which are still in place in our organization, instead of controlling the change which is happening. So management should focus on paving the world for DevOps teams, so they can move on and move on and move on.

So what are the key takeaways?

As we already told you, when you start a DevOps journey, you will go through different phases of adoption, and every phase needs a different approach.

But what we can say: it all starts from the bottom up. It starts with the engagement of the workforce. So we would recommend to pick the innovators in your company, make them owner of their journey, and focus on what really matters to them.

And to determine what really matters to them, it all starts and ends with the business. Because if a DevOps team says, "Now I'm able to deploy six times a week or daily," the main question is: what is the added value for the business? Because if there's no added value for the business, it makes no sense. So make it a joint effort.

And last but not least, what we have experienced is that teams are eager to improve. Just give them room to play with the right supported tools.

Yeah, so this was our presentation about how we created a movement, or started growing a movement. We are interested in feedback, of course, and also if you have any tips regarding how to cross the chasm towards enterprise DevOps, we are still here in the hallway.

There is a stand of GamingWorks where you can find the DevOps Playbook. So if you are interested, come over and have a look.

Thank you very much.