Agile/Dawie Ruined My Life
Moving from the shadow of the valley of death to a culture of velocity, engagement and quality engineering is a hard and complicated journey. The temptation is to start with the methods, tools and techniques that drive the modern DevOps organisation.
This is the story of what happens when you start with the people and culture before anything else, and the magic that breaks out when we then add the velocity.
Chapters
Full transcript
The complete talk, organized by section.
Dawie Olivier
Good afternoon, everybody.
I think today I win the prize for the strangest presentation title in the entire conference. Or the best, yeah.
This morning we spent a lot of time listening to the stories of DevOps transformations, and in virtually every single one of them, the conversation at some point turned to the point of culture. Now, what we didn't do for many of them is explore what that means and the approaches taken to change this culture, because many of them were about the practices, about the automation, about the engineering that makes DevOps a thing.
First off, let me tell you who we are, and eventually I'll tell you what that quote means. Eventually.
I come from a bank called Westpac New Zealand. And for those of you who don't know where New Zealand is, it's not that uncommon. In fact, there is a website called worldmapswithoutnewzealand.com. And I do urge you to go there and have a look. On the maps where New Zealand does appear, normally we're the little messy bit to the right of Tasmania, just below Australia. That would be us.
Now, we're part of a larger banking group, but fortunately or unfortunately, in New Zealand, by legislation, we are required to run an independent bank. That has the benefit of carrying a large international brand. It also has the benefit of giving us the freedom to try some of this mad hippie West Coast stuff in a place where the rest of the group doesn't get too concerned about us.
The bank is about 5,000 employees strong, so it's not that massive. The technology team is about 660 of those, split over two places between Wellington and Auckland, so in the North and the South Islands.
Now, why is this story interesting?
Two years ago, I arrived in New Zealand from South Africa. For those of you who didn't pick the thick accent, I'm not Australian. You can pick any country except Australia for both South Africans and New Zealanders, and we won't be offended. But if you pick Australia, we're in deep trouble.
Many of those enterprise problems that you saw in the presentations this morning were absolutely present in this bank. We suffered from slow releases, bad reliability, lots of customer outages, lots of recrimination and fear and loathing and lack of empowerment, and nobody really interested in getting things done.
On further examination, it turned out that the majority of these problems that we were seeing, as much as there were many technology problems and many process problems, the majority of them referred to the fact that, frankly, there were 660 miserable people working in technology.
So how did I know they were miserable? Well, they tell me. And they told me that a lot at the time.
In fact, at the very first all-hands meeting I had, which was about two months in, I was standing in front of a crowd pretty much this size, and there were no questions and there was no feedback, and no one wanted to speak to me. And right at the end of the entire thing, somebody put up his hand and he said, "Dawie, I have a question."
And I said, "Oh, thank God, someone's got a question."
And he looked me in the eye from the back of the room and he said, "Dawie, I would just like to know, how long is your contract?"
So I wasn't really guessing too much that not everybody was particularly enamored of the organization at the time.
Turns out that just after that, we did one of the many organizational health or engagement surveys. We used a tool called the OHI by McKinsey. And it came back, and across 49 different quadrants or 49 different dimensions, it told us that we were in the fourth quadrant of organizational engagement. Now, I'm not a mathematician, but there are only four quadrants. Fourth one is the bottom one. Doesn't get any worse than that.
When the organization is in a state like that, there is no point trying to introduce hard technical practices. There is no point trying to introduce a new method, because virtually everything that is suggested is received with deep cynicism and suspicion. Everything feels like the new broom bringing his new sweep and, "Dawie, how long is your contract?"
What became clear quite early was that when we try, and this morning the question was asked, how many of you have seen failed organizational transformations? And the answer is all of us have in some way, shape, or form. Invariably, those transformations or those changes start by trying to change the mechanics of the organization. Sometimes they are a leadership change. Most often they're a middle-management layer change and an org-structure change and an op-model change and a method change. And invariably, a number of years later, the organization is in the same or a worse state than when you first started.
Because the one thing that we don't often start with is the culture.
Culture is a big word. It's a difficult word. It's hard to define until you boil it down to a very simple question. Culture is what it feels like to be at work every single day. And even more importantly, great culture is what it feels like to have a great day at work every single day.
All of us have had great days at work. You can take two seconds, cast your mind back. When is the last time you had a great day at work? Gives me goosebumps just to think about it.
What are those things that we see when we think about those great days at work? Invariably, they refer to us having slain a dragon, to us having climbed a mountain, having beaten a foe, having done it together, having done it with air cover, having had somebody say, "You are the best team in the place. You are the only people who can pull this through for us. We're going to put you in a room. We're going to give you all the support you need. Go out and attack that hill."
Unfortunately, there were 660 people in my team who didn't know what that felt like and hadn't for a very long time.
Interestingly, there was another strong correlator. This morning, there was a point made about leadership, many points made about leadership. But one of the keys to leadership and enduring leadership is the enduring part, is the fact that often just having a consistent set of leadership for a while really helps.
And this particular team were on their ninth CIO in 10 years.
Yeah, starting from a low base, right? Let me give you some advice. If ever you take the job of CIO in a bank, these are the symptoms to look for, because it can only go upwards from there.
So what did we do? We got our teams together, the leadership team first. And it was quite clear that we had absolutely no idea where to start and tackle this significant feeling problem that we had.
So we decided to ask. We got our teams together in a room, and we asked them the very, very simple question, "What would a great day at work feel like for you?"
It's not a generic question. It's a very personal question, and it's a question that conjures up very real feelings.
Firstly, let me apologize. I am South African, so this whole feelings thing is a little bit raw for me. It's not my natural state. And New Zealanders are no different to that. So there we had this entire team of technology people sitting there going, "Please make him stop. We don't want to talk about feelings."
And yet, once we got over that barrier and the words started coming out, they were relatively consistent across the entire team. The feelings that people liked were the feelings of being trusted, the feelings of contributing to something greater than the tech work that you're busy with now, the feeling of understanding the big outcome that we're all after, as opposed to, in the old waterfall state that we were in, only ever understanding your incoming boundary and your outgoing boundary.
Now, that leads us nicely to that little quote that I started my presentation with. It was originally just, "Dawie ruined my life," but the guy who wrote that, a guy by the name of Wayne, softened it a little bit for the presentation. He didn't want to offend me. By that time, I wasn't as reviled as I was in the beginning.
And the reason it ruined his life was because he was a guy who believed he was happy in his box. He liked knowing what his incoming criteria were. He liked knowing what the very minimum work was that he needed to do before he could hand it off to somebody else. And he knew exactly how to do only that minimum work so that he could meet his KPIs and get paid.
Turns out that in New Zealand, there is a very strong family culture and a tribal culture, especially among the Maori folk in New Zealand, where people contribute as elders in a community, and their job is to teach and to coach. And this gentleman, Wayne, was one of those elders.
And as we started exploring these feelings, something became really, really obvious to him. What became obvious to him was that he was experiencing two very, very different things when he was contributing in his community or when he was coming to work to spend time in the office.
And then it struck him that when he was spending two-thirds of his waking life in the office, that it really felt like he was wasting his life coming to that office every day. He should be spending more of it contributing, giving, learning, enjoying those higher-order outcomes, those higher-order feelings.
Now, it would've been quite easy for Wayne to say, "You know, Dawie, I'm done with this. I'm out of here." And to his credit, what came out was the elder, what came out was the teacher. And today, Wayne is one of our agile and DevOps coaches, and he spends his entire life helping others have these great days.
But it was a bit of a journey before we got there.
Having understood what the Promised Land could feel like, we had to codify this in a single unifying statement. We spoke about vision this morning.
Now, I'm going to keep coming back to the fact that none of this was driven from the top. All of this was facilitated and extracted. The teams came up with something which drives you towards virtually every practice that you would've seen today and over the next couple of days.
This is the statement that the team came out with, the Westpac New Zealand technology culture statement. I'll read it to you.
"We take pride in delivering great experiences that our customers love through empowered individuals in cross-functional teams trusted to do the right thing."
Wow. Wow.
So there I was, old-school technology leader. Love my waterfall, love me some control, like putting people in boxes, and this is the aspiration that the team came up with.
So then something interesting happened. We'd had this big love-in, and it was fantastic, and we all felt warm and fuzzy. And we went back to the office the following week, and it sucked, because we'd created this aspiration for what a great day could be and what we would like that great day to be and how we would want to operate every single day, and the reality on the ground was not that.
So humans rebel easily, especially when you create the aspiration and you put them back in misery.
The team started coming to me and saying, "Hey, Dawie, I know we don't do this agile thing here, but would you mind if we try and insert one practice here?" Or, "I know we don't do this DevOps thing here. Would you mind if we try, insert a testing approach here?"
Slowly but surely, in fact, not so slowly, but definitely surely, the momentum started building of more and more teams who were trying very many practices that make up the schools of both agile, lean, and DevOps. Both, all three of those.
And it was chaos. And it was chaos for a very, very long time. Until we started figuring out that what we wanted, and you heard it said this morning, we can't land nine out of 10 planes. We have to land all the planes.
It became clear that what we needed was to get the expertise in to help us live that aspiration of culture and run a safe bank at high velocity.
So this is where our partners came in absolutely critically. We had IBM on our deck. I see Sheila here from IBM. We had folks from Chef on our deck, and as many of the DevOps and agile experts as we could surround the teams with. Not to drive, not to create, not to build, not to apply a method, but to give access to the tools and the approaches that would allow people to feel the way they wanted to, they aspired to feel.
So where are we today? This sounds a bit like a hippie camp, but at the end of the day, we do run a bank. And as the CIO of a bank, we needed to make sure that what we were doing was very, very real.
We chose four sets of metrics.
The top left one, your left, structural efficiency, allows us to measure, for every single dollar that we throw at a problem, how many cents in that dollar go towards fixing the problem, and how many cents in that dollar go towards us doing business with ourselves and waiting and handoffs and all kinds of other waste in the organization.
When we started this process off, which was in October of 2015, the first time we measured was around 25 cents in the dollar, and I was absolutely horrified. And over the next two months, it got worse. It went down to 23 cents in the dollar.
Now, think about that as a shareholder or think about it as a responsible executive, where out of every dollar that I use to solve a problem, three-quarters of that dollar is going towards non-value-added stuff.
If you were taking your family's savings to an investment broker or to a bank, and you said to them, "Take these savings from my family and invest them in my future," and they turned around and said, "I'll absolutely do that, and the first three-quarters is going towards paying for my office and my assistants and my filing and my computer systems," that would be a pretty poor return.
It meant that when we saw this, there was no way other than to respond by allowing the teams to figure out what the very best ways were to deliver this valuable work for our customers.
The next interesting thing happened was that we repeated our engagement survey a number of times over the period. So the first one was in September of 2015 or October of 2015, came back at 53%. And virtually within six months of allowing teams to form these aspirations and start touching and feeling these new methods and engineering techniques, we jumped into the first quartile of engagement across those 660 people.
So as a salted executive, my first thought was, "Yeah, that's a flash in the pan, honeymoon phase. It'll go away." And two surveys later, we're still sitting in the high second and in the first quartile of engagement for our teams.
Something else interesting happened, is that we didn't do the classic thing of reorganizing or restructuring the business. What we waited for was the teams to come to us and ask us to change the structure of our business.
So by the time we eventually came through and we changed reporting lines and spans and controls and all those beautiful scientific organizational measures, the consultation process went like this: "Dawie, we know why you're doing this. We've asked you to do this. Please get on with it so we can get back to work."
Which is an amazing outcome for a process which is normally massively traumatic. And virtually no effect. We dipped into the second quartile when we did that restructure, which is not too bad.
Then there were two other important measures. The one is the efficiency of the run shop, because one of the classic fears here is that when we unleash automation and high-frequency change, and we're putting lots of small changes in every single day instead of quarterly or every five months or so, surely my organization's costs are going to go through the roof, for one, and secondly, reliability is going to plummet.
Not so.
Interestingly, year on year for the third year in a row now, we've showed 9%, so you can compound that out if you will, 9% year-on-year true technology run efficiency gains. So we've become cheaper by 9 to 10% every single year for three years in a row.
Even more startling is we measure a thing called frontline-affected downtime. The definition of frontline-affected downtime, quite simply, is that downtime that prevents us from serving a customer on any of our channels, be it in a branch, be it online, be it on the app.
In 2015, we had over 7,000 minutes of that kind of downtime. Now, that's a lot. It is across a whole bunch of channels, though. And in the following year, it was down to just over 1,000, and in this year past, we're down to just under 1,000 minutes in total. So that's a seven or eightfold reduction in the amount of time that we couldn't serve our customers.
So what do you think the impacts of that on the bank are in terms of revenue, in terms of NPS, customer experience, and satisfaction?
Well, our results unfortunately came out on Monday and I didn't have time to put them in here, but we've had two of the most profitable quarters that we've had in the last five years. Our NPS is slowly starting to recover from where we were down in fifth. We're in fourth now out of the six big banks, five big banks in New Zealand. And slowly but surely, the trend is starting to climb as we start getting on top of outages, creating new customer value, and driving that value straight into our customer hands.
So what did we learn?
You saw most of these lessons this morning.
The first lesson of these is that the job of leaders is not to lead. The job of leaders is to listen and to enable. So we spent a lot of time taking those egos that we'd built over 20 years as executives in big businesses and putting them in our pockets and spending our time on the floor hearing from people who understand how the job should be done, exactly what that looks like, and what kind of support they needed.
We spent a lot of time telling stories, finding those great stories where people were having awesome experiences, and making sure that they had the opportunity to tell them to as many people as they possibly could, so that our change could become an intrinsic viral pull change, as opposed to the classic downwards push, scientific style of change.
We had to let go of dogma. One of my favorite fights to watch. There were actually two. I'll tell you two anecdotes.
Right in the beginning of the journey, there was a session with all our project managers in the bank. We had about 70 project managers at the time, which is not a particularly big cohort, but it's not a huge team.
We got into this room, and the project managers spent an hour telling me how the bank was going to be run and how we were going to deliver all this stuff. And right at the end of it, they turned around and they said, "So Dawie, what do you think of that?"
And before thinking about it, I blurted out, "I think in 18 months' time, there will be no project managers in the bank."
Yeah, you can imagine. Firefighting happened.
And that was one of the key lessons for me, was that regardless of what people are doing at any particular stage as their job along this journey, we have to be deeply respectful of that job, even if we know that it's going to change.
The fact is that more than 80% of those people are still with us today. They're just doing very different roles. And perhaps the better way to have described what was going to happen was that the behaviors that made us effective project managers in a waterfall-style bank are not the behaviors that are going to be getting us where we need to be.
That's the one story.
The next story is today I have 60 to 70 Scrum masters. And if you think that dogma is a strong thing in the waterfall world, wait until you unleash the Scrum masters on each other.
I see PMI is having a session here somewhere today. And we have a strong cohort of PMI-certified folks, and we have a strong cohort of scaled agile folks. We have some disciplined agile folks, and it's always fun to me to listen to when the assumption of which method we've adopted becomes the topic of a conversation, because it's almost a fight to the death.
So we had to be very strong and pull back and say, "We're not adopting any method, but we do expect you to be knowledgeable of as many as you can. Because what we want to see is we want you to bring your knowledge and your experience to the teams who have to do the work and let them define what these methods look like."
So today, if we were to illustrate the method in Westpac New Zealand, firstly, it would be really hard because every squad and every tribe looks different, and they operate differently because they're doing it the way that works for them.
The second thing is that, depending on what your particular dogma was, someone will draw you the picture of a scaled agile release train. Someone else will draw you the picture of a Spotify-style squad. And it really, in the end, doesn't matter, because within the context in which they operate, it's the right approach and the right method for them.
Now, that's a very hard thing to do, is to not go and grab the first method you can find and tell the organization, "Do this."
So the bigger picture.
Oh, I seem to have broken that.
Over the last couple of years, I showed you the results. The results are quite startling. The change portfolio in the bank, it's not a very big bank, but the change portfolio is about $120 million year on year in capital expenditure.
Over the first year, we unlocked nearly $45 million worth of additional value out of that portfolio. So it's a 30% increase in the capacity for change. But the year after that, we delivered the full $120 million for the full $120 million worth of money, and we've delivered more change than at any time in the entire history of the bank.
There are no exceptions to the adoption of agile methods and DevOps methods. So there are no waterfall-style pieces of work left in our organization.
And if there's any thought that I can leave you with as we go through this or as we finish up, it's the fact that these changes are hard. You heard it said this morning. They're really, really hard.
But don't ever waver in your conviction that many of the old practices in our organizations are extremely poisonous.
We tolerated a particular set of work that was delivered in a true waterfall style and caused the first two P1 outages of this year, which was about six weeks ago, because of the fact that they hadn't had the opportunity to bake in those upfront engineering practices, the test-driven development, the automation, the monitoring, and the like, which we would've liked to have seen.
But you know what? That's not my regret. My key regret for that piece of work is the absolute misery in which that team has lived for the last 18 months.
So as leaders, knowing that our people aspire to things because they're people, knowing that they spend two-thirds of their waking lives with us and with each other, our true accountability in this world, besides for running great businesses, and that is a secondary consideration, our true key job as leaders is to ensure that every single person in our teams can have an absolutely great day at work every single day.
That's me. Thank you very much.
Q&A
Q: What was the hardest part of moving from waterfall to continuous delivery to DevOps? Within DevOps, there are different stages of DevOps, challenges.
A: Yeah, that's right. So for us, it was breaking down many of the traditional barriers, technology barriers around where these practices ostensibly can or can't be done.
Teams are quite quick to trot out the fact that you can't do DevOps on the mainframe. Yep. Can't do continuous integration on COBOL. And the fact is that none of those things is necessarily true.
But it's not the kind of argument that you can attack head-on. You have to lead people one step at a time to the realization that there are alternative techniques that allow you to operate in this way on any platform.
So that's really the hardest part, is the places of safety people go to when everything else is feeling new and strange is most often in their technical disciplines. And to break down the barrier that it can't be done is the hardest challenge of all.
Q: What percent or how many people just didn't make the move inside the leap?
A: Oh, that's my favorite question, and thank you for asking it.
We aimed for about a 20% turnover at the time. We never really got there. We got to about 18 and a half odd percent of turnover. And I think it stayed that low for a very specific reason, is that we never asked anybody to buy a destination, only ever to buy the next step.
It was essentially people who had become progressively uncomfortable to the point where the penny dropped, that they didn't like what we were doing. So we didn't have an exodus immediately. We had people trickle out of the organization over the space of a year or 18 months. So about 18% is your number.
Q: You allowed your teams to select their own methodologies. So what happens when team A has different methods from team B, and they're operating differently?
A: Yep. So dependency management is a maturity that came to us about six months into the journey. And the interesting thing about dependency management is it never comes softly, it comes with a loud bang.
Essentially what we had to do is just put in one of the most basic disciplines of all, is that there is no dependency between two teams or two environments unless both agree. Mainly because it got really easy for teams to throw something across a fence and say, "Hey, I'm waiting for the XYZ team."
Whereas making them visibly agree this in front of a board together meant that the method didn't really matter, as long as we knew that the dependency was in place.
Q: So your approach to culture was to allow the teams to come to you and say that they need this change. What happens if the team says no?
A: The question was that the approach was to allow the teams to come to me and indicate that they needed the change. What would've happened if they hadn't?
So the fact is that it feels a little bit more voluntary than it was in the beginning, is that what I had to create was the opportunity for people to have the conversation. And the benefit of being the new guy was that people were a little bit curious.
The second thing is that we actually launched a competition. We called it the Culture Quest, which was a real pain in the ass of a questionnaire that took about three hours to do. But it was a leading inquiry questionnaire, and it led you to organizations with great cultures.
And the invitation was that if you're unhappy with the state of the organization today and you would like a seat at the table where we start fixing it, this is your way to earn your seat at that table.
About 8% of the organization made it through, so we had a cohort of about 50 people that came to the first set of tables. But there were two important things that I knew about them at the time. The one was that they were really fed up, hence they were willing to go through this entire three-hour cycle to get to it. And secondly, that they had been pre-prepared with the knowledge of the art of the possible when they arrived in the room.
So it's a little bit sneaky, but certainly it was devastatingly effective. It was really effective for us, having those two factors in the room at the same time.
The second important thing is that we allowed ourselves to grieve a little bit over the results. Because in the process, the first thing that'll happen is someone will say, "Hey, Dawie, it can't surely not be that bad." And having 49 dimensions of engagement surveys saying the opposite was very useful.
So we put the verbatim results on the tables, we put the surveys up on the screens, and we allowed people to roll these things around for an hour or so, truly feel miserable about it, and then put them away and started on the process of the new world.
Q: And how did you drive transformation of middle management?
A: The question is, how did we drive transformation of middle management?
I always read with great interest when organizations go on, and there are two words to dread with your whole soul. One is agile, and the second is when they add that to transformation, because almost immediately thereafter, someone will say that we're going to cut through the frozen middle, or we're going to reduce our middle management.
And the fact is that we didn't do either of those things, mainly because most of the middle management in the organization had, not that long ago, been the custodians of technical disciplines. And given the opportunity for many of these people to grab hold of those technical disciplines again was very valuable for them.
Not everybody liked that, and that's where the majority of our 18% came from, was from that layer.
Good question. Thank you.
Q: How did your client feedback internally...? What did the reviews look like? How did you come up with an example? And also, how was your auditing? I guess it's not SOX for you, but how did you get audits incorporated for this change in your process?
A: Right. So the question is, how did the internal feedback change? How often did we get it, and how did our auditing processes change?
Firstly, it is SOX for us, so we are SOX compliant. And the only right answer for us was to have our partners, like the risk team and the compliance team, in the room with us, which we did. So we invited everybody in.
Today, our technical teams are run by business product owners, so the feedback loop is instant. Technologists don't speak for the solutions anymore. The business product owners speak for solutions and outcomes.
That said, the conversation is had in a triumvirate or a triangle of leadership in each squad, being a product owner, a technical lead, and a customer experience lead. And the three of them together decide and control both the backlogs, and then they manage the results of what comes back from delivery.
So there is no business versus technology conversation to be had.
In fact, our next iteration, which has just kicked off, is to do this at much more scale, is to have the entire business stood up in what we're calling villages, which are cross-functional teams of full technology and business spectrum. And hopefully next time I speak to you, I'll have the opportunity to tell you how that went.
Q: So you know SOX change is typically slow. Did you do something to speed up the writing, the narratives? How did you get the narratives to change quickly?
A: We basically made the statement that we were doing this, and we were going to make those changes at the same time or we were going to be in trouble, frankly.
Q: So you did have some risk of failure?
A: There was definitely risk of failure, absolutely. Important thing was creating the context of safety for the teams doing the work, that that failure would be mine and not a failure of the teams.
Clearly, it went okay.
Thank you very much, everybody. I think we are holding the room for the next session.