Host Intro (Gene Kim)

So the next presentation is from the amazing team from BMW.

Ralf Waltram is the VP of IT for R&D, and in five days he will become the VP of IT for production. He's presenting with Christoph Brinck, who is the VP of IT governance. Over the last three years, they have been on a tireless journey to modernize the ways of working across all of BMW.

I met them last year here at DevOps Enterprise London, and when they told me about what they've been able to achieve, I knew that this was a talk that everyone needed to see. I think the quote that best sets the stage for what they've done is this: "The work that we are doing is responsible for the biggest change to how BMW does business in the last 20 years." So with that, roll video.

Video

The future of driving. Nothing to be afraid of. Because we are working on safer cars for a safer tomorrow with large-scale data-driven development at the BMW autonomous driving campus.

Christoph Brinck

Welcome to our short talk about a huge change in our organization, which is, by the way, also nothing to be afraid of, like autonomous driving too.

I'm Christoph. I joined BMW almost 20 years ago as a software engineer, and after having several roles in business and IT, I'm now heading IT governance, which includes strategy, architecture, technology, and cybersecurity. That's why I'm here today. I'm supporting the agile transformation to BizDevOps for IT and for the whole organization. Today, I'm here with Ralf.

Ralf Waltram

I'm Ralf. As Gene mentioned, I'm VP for the IT with my team for research and development for five days, and afterwards I will be for production. As you saw in the video, large-scale data-driven development is an awesome thing to do in a DevOps environment, but we'll cover this later on.

I'm also more than 20 years with BMW doing DevOps, and we would love to share with you our story about 100% agile and becoming DevOps. But before, we would like to start with some facts about BMW.

I hope a lot of you are driving BMWs, and if not, you should go to a dealer next to you and check it out, please. Last year, we sold more than 2.5 million cars to happy customers. But BMW is changing from a world-class premium car producer, manufacturer, towards a mobility provider. We are absolutely happy that more than 12 million cars are connected via our ConnectedDrive backend. We bring services to our cars like real-time traffic information, news, online entertainment, but also give you the chance with the mobile app to stay in contact with your BMW whenever, wherever you want.

As Harald Krueger, our CEO, stated, the change is going on, and we want to become the leading tech company for premium mobility, as you saw with the development, for example, for autonomous driving. You can imagine you need a lot of IT people for that, and Christoph will tell us how IT is organized at BMW.

Christoph Brinck

Yes. A short look at IT. We have two main areas of IT in the company. One is called Connected Company. This includes in-car IT, all the lines of code being inside the car, and the backend surrounding the connected cars. Then there's BMW Group IT, where the CIO lives, and where we are living too.

We serve every business process in the company from HR and finance to sales, aftersales, engineering, production of cars, and so on. We are spread around the world. We are now over 5,500 people in Group IT in 29 countries. The main places are, of course, Munich, then South Africa, a DevOps hub in Portugal, a DevOps hub in the Americas, and then, of course, we are placed in many other places, in the plants, in sales, in the regional organizations.

What is quite important to understand our journey of agile transformation is that, of course, we have a brownfield of technology. We have around 4,500 applications up and running in almost every technology you can imagine.

Having set the scene a little bit, let us come to the why. Why did we find out that we had to change something?

Our working model as Group IT was very clear. It was described in hundreds or thousands of pages, the so-called ITPM, IT process model: a waterfall model including 40 roles solely for IT, most of them ending with "manager," by the way.

We were used to delivering in large projects, although we already knew that the so-called smaller project maintenance was much more efficient than the other ones. But we were used to doing it. We were used to delivering in a waterfall model, and to be honest, it was plan-driven, but it was not always following a plan, as you might have experienced in the past too.

Of course, putting all these things together, putting together applications which have dependencies, led us to release weekends from Friday 12 o'clock to Monday 6 o'clock with a 24/7 operations mode. Nevertheless, IT governance was setting up a bunch of rules and guidelines that only a few people stuck to or even knew about. That was the time before 2016, and you will see that something had to change, Ralf.

Ralf Waltram

Definitely. I was many years a project leader at BMW in an IT project, and as you saw before, it's not always fun to surf on big waves. You get washed out heavily. So we said two years ago, "Hey, something had to change." In times of digitalization, in times of new technologies, we have to get more flexible, we have to gain speed, and we should put more of our customer, internally and also externally, in focus. Not always plan-driven, value-driven. We want to deliver good things to our customers.

The answer at that time, two years ago, within BMW was that we are a bi-modal IT, two-speed IT. We do waterfall, and we do agile. 20% of the dev teams already worked in agile, and 80% are working in waterfall. Guess what? This was not the right thing. We said our answer must be 100% agile.

Why? Having two speeds means also we have two cultures in a company, and that's not easy to handle. We thought it's better to give a direction to 100% agile. It's a vision. It's not a one-day-off. It's not that you give everybody a book about agile, reading over the weekend, and then you are all 100% agile. This is not working.

The second thing was we saw that the agile teams were blocked out by the waterfall people because we had two big releases per year. The agile teams could release every two weeks, but they had to wait for the waterfall colleagues. That's why we said our answer is 100% agile.

We started two and a half years ago to go in this direction of transformation, and this is how we did it. It's not only about the process. For sure, we started with the process, getting agile, trying to bring agile methodologies to the teams and try things out. But it's also that we changed our structure. Before, we were separated into dev and ops teams. We decided to change our organization towards a DevOps organization and bring the teams together on the lowest possible level, on the team level, dev and ops together. Also, as we cover later on, we went to a product portfolio.

As Christoph said, we have a huge legacy landscape, and we saw that big monolithic systems are not very easy to handle in an agile way. That's why we also tried new technologies, to work out microservice architectures and cloud-based architectures.

Last but not least, and for me the most important point, it's all about culture. You need, for an agile way, a different culture: openness, transparency, every day in a stand-up being open to your teams. This was a massive factor, and we invested a lot, many workshops with the teams, with the leaders, to become agile in our way of working and to go this way.

This journey is not over. We started two and a half years ago, and it's not a straight path. I like this quote: "If the path before you is clear, you're probably on someone else's." So it's not that you copy the way of how Spotify is working. You have to find your own way. You have to put your own hand on the hot plate to find out.

Christoph Brinck

But you can take a close look on it.

Ralf Waltram

You can. Okay.

We picked out today four cornerstones of our agile journey. We picked out BizDevOps product portfolio, agile working model, Back2Code, and user experience. Christoph, you will start with the BizDevOps product portfolio.

Christoph Brinck

Right. I think it's the most important slide we have in terms of BizDevOps product portfolio. We have identified that 100% agile definitely means the end of projects, because projects naturally lead to a wall of confusion, you all know that, between projects and operations, and we had to get rid of that.

Furthermore, we identified that we have a second wall between business and development. We tried to break down both of these walls in one step. That's why we decided to go completely into a product world, to bring together biz, dev, and ops, which is not an org chart organization part of the model. It's a working model. It's how we are working together.

We were able to break down, and still are breaking down, silos in doing that. I think the most important point for that is that we were able to change the funding. In the past, we were funding projects on the one side and operations on the other side. Since the 1st of January this year, we were able to convince our board of directors and controlling, which was quite more heavy, to change the funding and accounting to a product-oriented model.

Almost 70% of our budget is now funded in products, and the rest in so-called initiatives, because we want to avoid the term projects for the next years in order not to fall back into a project model. The so-called initiatives are only there for coordinating the development in the products. All development is done in the product world.

We then, of course, trained the product owners and trained the so-called domain owners. I have to explain domain a little bit. We have now around 200 products to cover all our 4,500 applications in order to help our business processes grow. We combined them into so-called domains, process domains, which we steer actually with OKRs. That's our actual status where we are going to.

On the way to that product portfolio, we identified another opportunity we were not aware of before that. In the past, we were steering projects on the entity project, applications on the entity application, and operations on entities like fixed groups or something like that. There was a new opportunity to bring all those things together, to bring all facts and numbers, all figures together, to a product portfolio, which we set up.

We built our own tool, and we're still evolving it, where we can really have a product portfolio, be it a look on the Jira backlog, be it a look on the risks we are taking with that product, or how that product is used or is it used at all, on the budgeting, on quality, and of course also on cybersecurity risks and all these things. We are still evolving that, but that's a thing that helps us a lot in coming to real product portfolio steering, coming away from project steering.

Having set the scene for the product management and the structures, how do we really work together, Ralf? Can you explain that a little bit?

Ralf Waltram

Here we define our agile working model at BMW, and it was very important, like Christoph said, that we focused on the feature team level to give them agile bandwidth. We didn't say, "Hey, you all have to use Scrum." It's fine to start with Kanban, with Scrum, Scrumban, going up until scaled frameworks.

We also leveraged synergies between the products on the product level and even higher on domain or initiative level. Like Christoph said, we want lean synchronization. Decouple first, and where it's necessary, we synchronize it.

We defined our own working model. It's not a textbook. It's a living system where our teams are contributing. Christoph mentioned before our 40 roles. Now we have four roles: feature team, product owner, agile master, and line manager. They are all contributing to this agile working model. So it's inspect and adapt. If something is wrong, we put it in or we throw it out.

What's also important here is that it's supported by an agile tool chain. You need the right tool set and technology to support the team. I'm very happy. When we started two years ago, my idea was that we create a tool chain for 8,000 people. Finally, last month, we reached our 30,000 users on the agile tool chain, which is awesome, because all the teams like to work with this tool chain and gather together the experience with our agile working model.

So Christoph, now we know what our structure is from portfolio, how we work. What's next?

Christoph Brinck

Back to code. Here on the front, and not like the Adidas colleagues on the back. We have it on the front. Back to code.

What does that mean? One thing is very clear. You won't be successful if your core competence is project management and the software engineering competence is with your external partners. That will not work.

Ralf Waltram

That's not a good idea.

Christoph Brinck

It does not work in a waterfall model, but this does not work at all in an agile model. So we identified, okay, we have to define software engineering as a core competence. That's why we went to the board and asked for headcount: please give us more people. They said, "Okay, yes, a little bit. I give you some, 1,000 or 1,200."

But how to achieve that in a short time, how to ramp up, how to get that done? What we did then is we tried to find a partner, and we found a very good partner. We found a partner in Portugal, Critical Software company, and we founded a joint venture in September last year. Now we are already 400 software developers, DevOps developers, DevOps engineers in Portugal.

But that's not the last thing we are doing. There's one very important more thing. We have software engineers in our departments, but they do not develop software anymore. There is a chance to redevelop them, and that's why we started the Back2Code campus. It's a three-month full-time training. We pick out the people, the former developers in their actual teams. We retrain them to the newest technologies and then put them back, but to another team for sure, in order to not get them back to the old steering tasks they did before.

The third very important thing we did is we set up a much better developer experience, and that's something which is really, really important. We set up a new developer's workplace with a cool tool set. We set up a community. We made it more attractive to be a software engineer at BMW, because this is the problem: if you want to have and want to keep the good developers, of course, you have to be attractive for that.

Ralf Waltram

For the rest, of course, coming from a 25% software engineering competence, you will never come to 100% in a few months or a year. For the rest, we set up a project called DevOpsourcing, and that makes the working between internal development teams, feature teams, and external development feature teams much, much easier: getting office, have long-time relations to our partners, and have a very easy model to interact between the internal and external feature team.

We learn also from our external partners. We had great workshops with them, a very open and transparent discussion with them. We learned also from them, because also our partners are dealing with agile, with DevOps. It was a great exchange on that, so I'm very happy that we are going this way with DevOpsourcing.

Christoph Brinck

Absolutely. Coming from the developer experience, there's one thing missing.

Ralf Waltram

After developer experience, there is user experience.

Christoph Brinck

There's user experience.

Ralf Waltram

We have two examples of great tools at BMW with us. How many of you have mainframes still in your legacy landscape? Ah, a lot of them. So you know the dark mode. It's becoming more modern, but the problem is still that the user experience is still from the old time, and we have a lot of systems from experts for experts. The training takes weeks. If you are trained, it still doesn't make fun to work with these tools. There is no joy at all, and I think most of you experience this every day in your company too.

That's why we said, with 100% agile, something must change. If you want to become more user-centric, we should put also the user in the center and the user interface, because a user interface is like a joke: if you have to explain it, it's probably not a good one.

We have been focusing for two years on user experience. We tell the product owners: user experience matters. Don't put all your money into new cool features which an expert needs twice a year. Put your money into easy, intuitive environments, workflows, interfaces, and also mobile structures.

It's easy to write it down on a PowerPoint, but guess what? It's not so easy to understand. I'm absolutely proud that we created a space, 500 square meters, where we can show the teams user-centered design. Here you see the layout of the space behind me, and here you see it in action. We celebrated this month our first anniversary, and it's awesome. It's phenomenal feedback from the user team. It's an open-door philosophy: get in and you get help. It's easy access to experts. It's a different skill set.

I'm proud that more than 80 teams already used it heavily and changed their products, and the users really love the products. We put the real user into focus, not the expert user. That helps a lot. Ask your real user about their user experience, and you get the real feedback about that.

But after our cornerstones we showed, Christoph, what is the feedback overall from the business?

Christoph Brinck

Let's take a short look on it. We've sorted out the bad ones for sure, but I think there are some important things on it. I love the one in the middle too, to be honest: "I've never experienced bugs being fixed or new requirements being implemented so fast." That's someone from the business who says that about the IT organization, which was really new for us, this kind of feedback. So we have invested in speed, in cycle time, and we get feedback in terms of speed and cycle time.

Much more important for me personally is the right one, the one-mission thing. This faces the flexibility issue. We were facing real challenges with the WLTP regulation. This is emission, CO2 emission regulation, which changes the regulations really monthly. Our initiative, the process owner, stated that only adapting the agile way of working was the only way to get rid of that problem. That's really an honor for us to get such kind of feedback in terms of flexibility.

Ralf Waltram

Definitely. That's the target we have. It was one of the OEMs, automotive companies, which really could offer right away all our product setup, and this was really, really a great thing we did. The team was really encouraged to reach this, but as you say, it was only possible with the agile way of working.

What's the success factors of our strategy? We covered it here, and first of all, it was self-organized. We didn't hire a high-paid consultant from outside showing us fancy slides and telling us how agile is working. We did it on our own. The leadership team, a small team, started becoming more and more. With the right people, the willing people, they wanted to change really the way of working at BMW.

Stick to your vision. 100% agile is still our vision. We always tell it's a journey. It's not over. We did great things in the past. It was a winding road, yes, but stick to your vision. It's not easy, but you will reach it.

Last but not least, Christoph mentioned it before: to me it was in the beginning not so clear. I was focusing more on DevOps. But it was crucial for us that we started very early to do BizDevOps, to go to our business partner, talk to them about the change of how we work, because otherwise you are the crazy IT guys doing agile. Now, together, it's really all of us sitting in the same boat when doing this DevOps.

Christoph Brinck

Which is not the easy way, to be honest.

Ralf Waltram

It is not, but for us, it's the better way.

That was our success factors, and as I mentioned, it's a journey. It's not over. What's next?

Christoph Brinck

Here are some--that's where you might help us.

Ralf Waltram

Definitely. We are asking here for your help, for exchange with you. A lot of hashtags, and this is how we transport the messages to our teams. I want to pick out one: #transformers. As I mentioned, BizDevOps is a transformation started in IT, but is now really taking the whole business IT and business partners at BMW with it into the transformation, and we are going on with that.

Christoph Brinck

It's a massive change. It's the biggest change we had in the last 20 years in IT, I encountered. It's a good change because we changed from a cost factor to a real enabler helping--

Ralf Waltram

To transformer.

Christoph Brinck

--to transformer to the whole company.

Ralf Waltram

A second one I would like to pick out is #50releasesperday. Sounds funny. Do I really have to make 50 releases per day? But we stress our teams. We ask them to think about it, because when you are able to do it, it's so much easier to release software. You have to invest a lot into changing how you work in your CI/CD pipeline, especially testing. The internal phrase to reach this is "Viva la automacion." We put a lot into automatization to really achieve this goal: 50 releases per day.

Christoph, what are the two things from your side for them? What are the next things for you?

Christoph Brinck

For me, the most important thing is the DevOps platform. We have to get rid of our brownfield technology, but we don't have two or three billion to invest in the next five years just to rebuild the old thing. If someone has any good idea tomorrow, Ralf will be here tomorrow.

Ralf Waltram

And you will find me with this jacket, you see. Come into contact.

Christoph Brinck

We know that, and we heard it from colleagues this morning, there are some people which are quite far with the DevOps platform. We are very, very interested in discussions about that because we really have to catch up.

The second one is the API-first thing. How do we convince our product owners, our feature teams, our developers of publishing APIs, be it internal or be it publicly? That's really a concern I have, and really where we could need help and input from your side. How do we do that in such a huge company? That's, I think, our targets for the next one or two years, where our journey brings us to.

Ralf Waltram

Yes. If you should have fallen asleep the last 25 minutes, there's the five key takeaways you can, if you want, take home with you.

Be bold when you start your journey, but start small. Our vision was very far away from what we had at that point in time, 2016, and we started in small blocks, and then we scaled it up. That's advice I can give. Be bold, very bold, but start in small things. Ralf, what's next?

We always talk about the journey and it sounds so great, but it's not a five-star wellness cruise. Definitely not. There will come rough water, but the good thing is you can react very fast in these terms if you have a good team, and this is the message to you from me.

Christoph Brinck

Use the agile way in the transformation: inspect and adapt. We have changed so many things in the last three and four years and changed over and over. To be honest, that's a very hard way of learning in an organization that has the motto, "First time right." We all had to learn. We still are learning that inspect-and-adapt motto, and it's very hard to implement, but I really stick with it, and I really love it because it helps us a lot in finding the right way forward.

Ralf Waltram

Last but not least--

Christoph Brinck

Not last.

Ralf Waltram

Learn from other journeys. Gene mentioned it. Last year, I met the colleagues from Adidas. We right away got together. We had a great exchange. 50 people of Adidas came to us, to Munich.

Christoph Brinck

A huge bus.

Ralf Waltram

A huge bus. We had a great world cafe about agile transformation, and afterwards we had a great exchange about the DevOps platform, about cloud services, and so on. The exchange doesn't stop. That's a good thing, and I hope I meet tomorrow a lot of people, a lot of colleagues here, to share our journey and to get in contact.

And always, don't forget: enjoy IT. Thank you very much for listening.

Gene Kim

Thank you. I was able to find a picture of my wife, so I want to show you something.

So, when you shared that incredibly scary movie about autonomous driving, it actually reminded me of something even more scary that happened to me, but exactly in reverse, an X5. Three years ago, I had my six-year-old twins in the back of my car, and I had to drop off a birthday cake to someone's house. I left the engine running just to drop it off.

I come back and I look in the back seat to say hi to my twins, and they're gone. I'm like, "Where could they be?" Because I'm wondering, I have to explain to my wife, I've lost the kids. Then I look in the front seat of the car and what I see is even scarier. What I see is this. The more mischievous of the twins had crawled into the driver's seat and was actually moving the steering wheel, moving the car, and I'm just praying that there's no way they could reach the brakes and move the car into park or reverse.

Ralf Waltram

It's the right car.

Gene Kim

It is the right car.

Christoph Brinck

That's important.

Gene Kim

Superb engineering. Thank you. Appreciate it.