Stephan Stapel

Hello, my name is Stephan. I'm happy to speak at DevOps Enterprise Summit again. I already introduced parts of our DevOps and cloud transformation story in earlier conferences.

Today, I would like to tell you about our idea of leaving nobody behind in our transformation. I start with a short introduction to give you an understanding of our setup. I will then introduce technical patterns that we apply, as well as organizational development patterns. I will give a general understanding of how we share knowledge in our environment and conclude with some takeaways.

Hermes Europe, which is our mother company, is the largest post-independent parcel company in Europe. I am part of Hermes Germany. Here we take care of parcel delivery in Germany to both doorstep and more than 16,000 Hermes parcel shops. I'm working at Hermes as a head of development, and I am responsible for the customer IT.

At Hermes, technology plays an important role because we need to adapt to an always-changing market, which is growing with 5% to 10% per year for the last decades. DevOps as a culture plays an important role for us with the principles of automation, measurement, and sharing knowledge. And delivering as continuously as possible is one of the most important aspects we have.

We set that as a strategic goal in 2017, which means that we are now in our fifth year of transformation, of following this path, and we are quite happy with the results that we find. Overall, we see a good increase in delivery speed. On the other hand, the incidents decreased. And due to these circumstances and the pervasive cloud technology and automation we now have in place, our people feel safe to change and deploy their respective application whenever necessary. So we have a high degree of freedom concerning the processes.

So we generally progress well. However, in an organization of about 30 to 35 product teams in total, it is not the same everywhere. And we asked ourselves: how can every team participate in the transformation easily? Is this possible at all? And do we see a speed increase everywhere?

And the answer is: we found that teams are running and transforming at different speeds. To some degree, this is probably natural for a large organization. So we asked ourselves if this is something to pay special attention to or if we just wait and see what happens.

And we decided against wait and see. And you probably remember the famous two-speed IT metaphor, which was coined five to 10 years ago when technological innovation around cloud was still in its early stages. And we are convinced that taking this route makes it really difficult to have a healthy tech culture and even more difficult to have a healthy tech organization. It makes things complicated since you always have to argue in which context which approach applies or not.

But foremost, we don't want any losers in our transformation. We don't want to have anyone being left behind. We want everyone in the same working system, at least to some degree, as much as possible. And this is a hard way. It's not the easy path since it takes extra dedication. But from our point of view, this is worth it. I will introduce you to the things that we discovered and the patterns that we are using to deal with this.

Before diving in, before kicking off, a short disclaimer. In my presentation, I will use the terms slow system and slow-system team, and fast system and fast-system team. I'm clearly not using this in my day-to-day work, and this is also here not meant to stigmatize the teams. This is why I mention this now explicitly. I just needed some terms to describe the situations for you so that you can easily understand it.

And the question we had: why are some teams slower than other teams? And we found two main reasons for that.

Reason one is overly complex logic. Some teams have too many domains addressed in their application, too many business contexts. And this often comes with lots of dependencies that are then hard to manage. And the second thing is that a particular release is often made difficult by complicated testing. The necessary scope might be too large because there are too many dependencies, or the landscape that is needed to conduct the testing is too hard to set up.

Reason two is the technical setup. Often, slow teams come with a certain age. I will not use the term legacy because I think that is not really fair. Thus, the technological foundation of such slower systems was not meant to deliver continuously. Other paradigms played a more important role back in these days. One example of such an environment are JEE containers that we sometimes find.

Another aspect is architecture of a slow system that is not well crafted. For example, when that system was not meant to live for multiple years, but was thought of as a one-trick pony: we develop it, and one year later we will replace it by another application, which never happened.

On the other hand, we didn't find any example in our organization where the team setup itself was an issue. So capacity or small capacity is no typical sign of a slow team. And the same is true for the skills of the team. So in all of our cases that we have in our organization, the slower teams are at least as well-skilled as the faster teams.

And taking a look at those issues that we found, we found that it is not easily possible to overcome those reasons. So we had to develop patterns to deal with that.

Let's start with some technical-oriented patterns that we are using to ease collaboration between teams.

Pattern one: isolate dependencies to slow systems. This is actually the most common pattern that we find in our landscape. The fast-system team isolates the dependencies to the slow system by adding a dedicated adapter or gateway, and almost treating the slower system as a kind of external system.

And by applying this pattern, the test scope is reduced for the fast-system team when releasing their own application in high frequency. Because the adapter covers the changes and makes those internal changes invisible for the slower system. And the same is true when there are changes to the interface between the two applications. These changes can be coordinated well, and this often primarily affects the adapter. So we can reduce the implementation scope and the test scope for the actual adapter that we have.

Pattern two is formalizing the interface agreements by using contracts. So we are using the contract-based testing approach quite often in this scenario, because this allows to decouple the actual implementation and even leave out the thorough integration test. And on the bottom of the slide, you find the timeline for implementing such an interface using this methodology.

So the two teams that we see here have to synchronize when forming the formal contract. Then for the actual implementation, they are free. They can plan it as their process is. And just for the go-live, they need to synchronize again. And by applying this method, we successfully reduce waiting times for both teams: both for the fast team, which can plan their time as needed, but also for the slow-system team, which does not need frequent coordination with the faster team.

And the third pattern is joint development. Because the slow-system teams usually don't have time for changes that are not essential parts of their own roadmap, this is why we started to send members from the fast-system teams to make the necessary changes to the slow system. If possible, we do this collectively with members of the slow-system team.

This is still a pretty new approach to us because it sheds a new light on organizations and on team setups, since it almost autonomously allocates and reallocates capacity where critical. So the two teams find their path to making the necessary changes almost automatically. On the other hand, we need to take care that the fast-system teams are not pulling the rug out from under the feet of the slow-system teams by doing changes wildly all over the map. And this means that pull requests and a good review of the pull requests for these kind of changes is very important.

This third pattern is already a mix of a technical approach and a cultural-organizational approach. So let's take it from here, and let's take a look at further organizational patterns that we have.

So next to the technical patterns, we have some patterns for organizational development. And before I start with those patterns, first and foremost, even if it is difficult sometimes, even if the way is hard, we don't give up on improving the slow-system teams.

This is true because we very much value these teams. We value these people. They are a big asset in our organization. These teams, that's quite typical, they usually come with a long-living team setup with little to no fluctuation. We have very experienced colleagues in these teams who are working with the company for years or sometimes even decades. They have a lot of expertise concerning processes, development approaches, and for their respective business domains.

On the other hand, slowness shall not be used as a weapon with full roadmaps that are not accessible for implementing changes that are necessary for other teams. Because this might hinder innovations of other teams or faster teams due to the dependencies which are in place in the landscape. And therefore, we need to pay attention to the atmosphere between such teams that come with different approaches to development.

What we find is that these teams are often reluctant to pick up the new toys, the new methods, the new technology. So where the majority of teams, for example, naturally picks up continuous delivery, this is often still mad magic from the viewpoint of the slow-system teams. We then hear, "Look, this might work somewhere else, but certainly not in our environment."

And we think that this often comes with fear of change, fear of not knowing what will be next, what this change means. So we try to avoid the term change and the term breaking change. Instead, we try to start where the teams are and develop from there, improving step by step, little by little.

And one concrete example for this: where continuous delivery is really easy to achieve in modern cloud-native environments, it is hard to achieve with a slow system, which comes with the technical boundaries that we covered before or the process boundaries that might be in place. But what is often possible is to deploy on a cadence of a sprint, which is, in our case, every three weeks.

And by doing this, we give even this team kind of a beat. So we get the slow-system team in the same working environment as the other teams. And this means in turn that all teams understand when they can expect a particular change rolling in for the slow-system team. And it also allows the slow-system team to ease the planning because they know that every three weeks there will be a deployment. That makes it easier also to agree with the business on the deployment, because to some degree, at some point, it comes natural to deploy on a cadence of three weeks.

And the difficulty about going small steps, step-by-step transformation, is that it needs a lot of attention to keep track, not to lose the goal because we only see the next steps. And of course, we need to keep track for the development of all teams, but this is especially true for the slow-system teams. And the larger the organization is, the easier it is to hide from a particular change. So bend and wait until the transformation has gone by: a common pattern to organizational changes. And this shouldn't be an option.

The goal of our organization is to do this transformation, so we want everyone to have in this transformation. All in means all in. To whatever speed and in whatever steps.

Pattern two: ensure sustainability. In our estate, and probably also in your estate, the slower systems are often the more important applications. And we need to consider this when working with these teams. We need to find good arguments when promoting innovation. Eventually translate and adopt these innovations to meet the conditions of the slow system.

And a good example for this is observability monitoring. This comes naturally when doing cloud-native development, since this is the only chance to understand if and how an application is performing in production. And for the slow systems, the story is a bit different. They are often maintained in a more classic environment: virtual servers, JEE containers, and this is done for years or even decades, maybe even with a classic IT operations team around this system.

Thus, the necessity for observability might not be as clear because the team was successful without it for ages. But by taking small steps, introducing the first metric and a second metric, taking a look at it, refining it, we were able to convince these teams that this is a good investment. It takes a bit more time, but when we have the teams there finally, they usually have a high visibility of their metrics. They're using it day in, day out, and they are even using it for generating some KPIs.

And the final aspect I would like to cover is sharing of knowledge. And important to note here is that sharing knowledge is no one-way street. It's not only about sharing knowledge of modern technology, modern methods, modern processes to the slow-system team.

We find that often the opposite is the truth. So we can learn a lot from the slow-system teams. It can be around processes, development approaches, and concerning business knowledge. Because we find lots of business logic in these applications and have lots of experts knowing about this business logic.

What we do in our organization is that we are periodically organizing IT-wide fairs, so IT-wide conventions, meetings, so that all teams can present what they achieved, what they are doing, and let the other teams learn from that.

Secondly, our reviews are open, and usually it is well-invested time to take a look at what other teams do. For some common topics, like continuous delivery, for example, we also establish communities of practice to share knowledge. Honestly speaking, this is the hardest part of sharing knowledge, so it is not easy to run these communities of practice for a long time, but probably that is part of another story that I might present one day.

Let me conclude with some takeaways. I brought a quote from our business from Christine, who is one of our trainers in the customer service. And we asked her about the business impact on deploying on a cadence of a sprint, which means three weeks in our context. And she said, "Look, we used to intensively train our call center agents with each release. It takes a lot of time to do that, and thus, we were very reluctant to introduce new releases for the application on the cadence of three weeks. But we found that the changes rolling in every three weeks were quite small, and they were quite easy to explain, which allowed us to often skip those extra release trainings. So it made it even easier for us," which I think is a good story.

Let me conclude with some takeaways. What did we learn? First of all, there is no one-size-fits-all approach in DevOps transformation. We found that teams are running and transforming at different speeds. It takes dedication for teams which can't absorb modern technology quickly and easily. This needs to be considered and well-planned because that takes management capacity.

But this effort, this extra effort that we have, is worth it for having happy people in our organization and a healthy tech culture. I would like to encourage you to follow a similar path and leave nobody behind your own transformation in your own environment.

In case you should have any questions, this is my contact information. Thanks a lot for listening, and hope to see you.