Leo Kraan

Good morning, everybody. I think some people are trickling in, but I'm going to get started. I think we're on a tight schedule.

Hi, everybody. Thank you for coming. My name is Leo Kraan, and I'm going to talk to you about developer experience: seven pillars to invest in, to be more exact. I want to talk a little bit about the benefits of developer experience, and then I'll lead into the different pillars to consider when you're investing in it. I'll illustrate this afterwards with a use case of Booking.com, where I work. But first, I will provide some context on Booking.

I trust you're in Europe, so I trust that you're familiar with Booking.com. You kind of know what it is. It's a large, leading online travel platform, and we invest heavily in technology to take the friction out of travel. It's quite an old company, already founded about 26 years ago in the Netherlands, Booking.nl. This is a screenshot of what the website used to look like at the time. It was purchased about ten years later by an American listed company, Priceline, and it evolved quite a lot since then.

We started off with just accommodations, hotels, apartments, but today you can also book flights, attractions, insurance, car rentals, and taxis through our platform. Just to give a bit of a sense of scale, we have about 11,000 people working at Booking globally, a big part of them in Amsterdam at our headquarters. We have a bit over 2,500 engineers of different flavors. In terms of our business, I think last year we had near 900 million room nights booked by our traveling customers.

Our technology environment also evolved quite a bit over these 26 years. In the first 20 years, we were kind of chugging along, really investing heavily in our accommodations business unit. It was all focused on just go forward, innovate, experiment, don't look back too much. We just had this one travel vertical. It was all developed in a Perl monolith code base, very tightly coupled code and business functions. We did a lot of testing in production: just a bit of testing before you push it out, wrap your change in an experiment, push it to production, flip it on, see if it works and if it reaches your hypothesis. We had about 500 developers, so it was all a manageable environment.

Then around 2017, leadership decided to also start exploring and investing in other travel verticals, like attractions was one of the first ones that really kicked in. This required us to rethink how we build and how we develop. That monolith, tightly coupled code base didn't really work anymore when you want to have other verticals next to it. So we started extracting chunks of code out of this monolith, started building services, and that's why we started developing on Kubernetes and also started adopting more modern languages.

In this area, we then started also doing our first developer experience investments. At the time, I was responsible for our internal Kubernetes environment, getting that off the ground. We then realized, okay, we're going to develop in Java now also, and maybe also build in Node. These developers that build these services need a framework, some libraries, charts, and a new deployment tool. So we started creating different initiatives that would serve these new needs. These were really the first DevX investments.

A couple of years ago, around 2021 or 2022, our business started moving into what we call our Connected Trip strategy, really focused on delivering the ultimate travel experience where we combine all our business verticals, all our travel products, under one roof, one experience for both booking and experiencing your travel. We need to take more friction out of your travel experience. With this, we adopted a cloud-first approach, really honing in on AWS.

Then we realized that how we've been developing, our testing strategy, our environments, how we bootstrap and create services, was not fit for purpose anymore in this new world. So we realized that we need a whole new code workflow and environment setup to go along with this modernization effort. That led to a much bigger investment in developer experience.

I'm currently leading these teams, and we have a bit over 60 people now in our developer experience space and continuing to grow. This is also needed because we have over 2,500 engineers developing still on bare metal, bare virtual machines, still on on-prem Kubernetes, and moving to public cloud. This really requires a great experience to make sure that this works well.

The mission of our developer experience team at Booking really focuses on providing that fast, safe, easy-to-use path from code to production, really taking out the friction so our developers can do what they love to do and why they joined Booking: to develop and deliver awesome travel experiences.

If you look at different definitions of developer experience online, it all comes into this same area. You find things related to developer effectiveness, developer happiness, and still delivering quality products. This is also in the DevX realm; there's definitely close connections to this.

We really care, of course, about our developers going fast but doing this safely. Ultimately, we believe that if developers can really focus on what they joined for and don't have to tinker away with tools, don't have to do a lot of manual work to get their code released, that will make them happier. They enjoy developing travel products, not manually progressing through a canary pipeline or checking logs manually, just to give an example.

To build on this more, why developer experience at Booking? There's a big opportunity for developer experience. Think about, let's say you have 500 services with a weekly release cycle. If you were to shave off about 30 minutes of productivity, manual overhead, then you already gain around 1,600 developer hours, or five or six full-time employees. So there's a big opportunity to invest in this.

As I said, faster time to market is very important for Booking. We want to innovate. We want to move forward. Developer experience is really needed for us to support the modernization of our platform. If we cannot provide a great developer experience for our developers to work on AWS or to develop Kubernetes services, then this is not going to lead to happiness. This is not going to lead to that faster time to market.

We need to be able to respond fast to the market, to the needs of our business, and the needs of our customers, which also requires this quick iteration I mentioned. Of course, we want our developers to be very happy. We want to make Booking a great place to work. Because we have this 20 years of working only in Perl and bare metal, the market still often thinks of us as this big Perl shop. We want to change how people see Booking, how developers see Booking, as a great place to work where you can work with modern tools and have a great experience. That's also why we think developer experience is critical. And of course, quality is always important: reliable, safe, and catching issues early. I'll go into that a little bit more.

Thinking about the reasons why you would want to invest in a great developer experience, there are some areas to consider. These are the seven pillars I want to talk to you about. They're not completely unique to Booking; they apply to our environment, but I'm convinced that this applies to many different types of businesses. They will really help you when you start thinking about developer experience, when you start thinking about investments and how to approach this.

The first pillar, really the foundation of your developer experience, is a customer-focused culture. Think about your customer. Of course, customers in the sense of developer experience are the developers, the engineers in your organization. It's a bit of an open door: you need to talk to them. You need to know what they want, what they need.

But for developer experience, as you are very likely going to change maybe the way they work, the tools that they use, maybe even some parts of the freedom that they have to do certain things, because you're going to be impacting that, it's very important, it's critical, to have your customers, your developers, involved in the entire process. Make them part of your assessment, of your design, of your implementation. Create this ongoing feedback loop. They'll be much more willing to adopt what you're going to be changing. Of course, you have a better product too, because it's catered to their needs. So give them an active role in your process.

Something that's not always obvious is that you need to treat developer experience as a product. Often I've heard that DevX-related initiatives naturally evolve from engineers spotting challenges, things related to DevOps culture that are being done. Automation is introduced, which is all great, but at some point you need to elevate this and look at your entire developer experience and really treat it as a product with a lifecycle, with a roadmap, with customer feedback loops, with customer engagement, with a lot of active communication about it and evolution.

When you treat it as a product, that also means that you need to invest in a product. It's a job for people. So invest in product managers. If you have large projects that require a rollout across your organization, invest in program management and project managers. This needs to be somebody's job. Engineers, of course, can help out with this a lot, but it's a skill by itself.

As I said, in the developer experience team, we have about 60 colleagues driving that now and growing. We have a product leader in developer experience at Booking who really looks at our entire developer experience as a product. There are product managers in individual teams that focus on our Java ecosystem, or on our CI and CD tooling, or our coding environment. They all have a product manager responsible for that space, responsible for their engagement and feedback loop with our customers.

Another critical part of the foundation, a second pillar, is a shared developer workflow. A shared developer workflow can be seen as a framework of processes, tools, and best practices that really covers your entire software development lifecycle, from design all the way up to release, operations, and deprecation. This is your code workflow: the different stages that are part of your developer lifecycle, gates that you have in those stages, and a quality strategy that indicates what practices you must or should follow.

This differs per company. A banking company, an e-commerce company, a gaming company, they will have a different code workflow and a different quality strategy based on the characteristics of their software lifecycle and regulatory restrictions. The freedom, the company culture, also plays a role in your developer workflow. The amount of freedom that you provide to your developers will dictate the framework.

Defining this upfront and communicating it well is really important, because a lot of your tooling and processes in your developer experience come back to this code workflow. You need to have that well-defined so that the different systems, tools, and processes in your developer experience connect to it. That helps provide this smooth end-to-end experience.

The next pillars are the middle layer of your developer experience. The first is a unified set of tools. It's kind of an open door. I'm sure you all realize the importance of tools and why you need them, starting all the way from language frameworks all the way to your deployment and your operational tool set.

For developer experience, what I've seen is you can go nuts with a breadth of tools and different open source solutions. Let developers pick whatever they want, whatever works for their personal productivity. That's not always ideal. In developer experience, it's important that your developers have a frictionless experience and that they can go fast. That means tools need to be well supported, need to have the right training materials for them, and you need to be able to support them well. This might result in you needing to center around a smaller set of tools so that you can guarantee this quality of service to your engineers and make sure that you can really leverage all the automation they provide. Here, you get into the situation of what do I build? Also, in your developer experience organization, your engineers are costly, so you need to be very mindful in terms of buying versus building yourself.

In terms of tools, your engineers in your organization are smart, costly engineers. Ideally, you want to get out of their way and let them do what they do best. But still you need to consider where you give engineers freedom and where you are opinionated about this. Where you want to be opinionated is personal to your organization, and that should be very clear in the developer framework, the code workflow that I've talked about.

In some companies, security is maybe the most important thing. Things like compliance, quality restrictions, restrictions that allow you to operate a platform with greater ease, or architectural guidance that you want to embed in how developers work, and things related to efficiency if you want to reach that high productivity. You will very likely need to put in restrictions for your engineers.

A great way to do this is by templatizing a lot of regular tasks: how you bootstrap a project, how you configure and set up your CI/CD pipelines, different operational tasks. There's a lot that you can read online about how Netflix and Spotify have done this through what they call, for instance, golden paths or paved paths. This is really a way to simplify bootstrapping, provisioning, configuring, and operating your systems.

Booking, for instance, has templatized the deployment pipeline. Where we're opinionated, you have to use Harness as a tool for Kubernetes and ADOS deployments. You have to use this set of templates. But within the templates, we have a small layer of mandatory settings that we require from all of them so that we can safeguard security and compliance. A big part of the template is completely open for engineers to configure themselves: things related to triggering certain tests they might want to do, things related to release orchestration steps in their canary process, how big they want those steps. Those are the things we allow them to configure themselves. So we provide this freedom with the guardrails that we care about as a company.

Next are metrics. I think every talk so far has mentioned DORA metrics, so I'm going to do it too. DORA metrics are a very obvious place to start, but there are a lot of other metrics that are also critical. I care about these metrics for my developer experience team so that we understand where there are potential bottlenecks in our environment, where there are potential quality issues, adoption issues, so that we can prioritize those areas and then continue and have a feedback loop of assessing that our changes reach the right effect.

But I also want to make these metrics available to the engineers, my customers, and their leaders. When you have freedom with guardrails, if there's still freedom, then you need to give engineers the data so that they can decide how they want to use that freedom, so that they can make their own choices related to their testing and release strategy.

Besides DORA metrics, a very focused area, in developer experience I also care a lot about metrics related to adoption. It's a bit of an affinity metric in the beginning, like: hey, are our developers using the tools that we've developed for them? But it's also good to know, with adoption, how it is staying up to date. Have they adopted the latest versions of libraries that we prescribe?

Throughput is also really important: how many builds are being created? How many times is a deployment triggered? How many merge requests are being submitted? The quality of that: do things fail and pass? How many vulnerabilities are found? And of course speed: you want to understand how long developers stay in certain stages.

Segueing off the previous talk related to security and compliance, of course most businesses have some sort of regulatory compliance. Of course we all want to have a safe, secure product and data also. This is really something that should ideally be embedded from the ground up in the developer workflow. Shoehorning in security and compliance controls afterwards in tools is painful and usually doesn't lead to the best and smoothest developer experience. All these things are commonly referred to as shifting left, where you put security and compliance checks at the right place early in the developer process and provide that feedback at that point.

The last pillar, the roof over our developer experience house: at the bottom we have the customer-focused culture, but we also need to have a leadership culture that supports developer experience. You cannot create a great developer experience in an organization only bottom up. You need support to drive forward the change. As I mentioned before, certain parts of developer experience take effort. It should be somebody's job. Developer experience requires funding, which you'll need leadership support for.

Also, sometimes developer experience results in changes for engineers, changes to how they work, expansions or limitations in freedom or tool choices. You need leadership to reinforce the message about this so that it gets adopted and accepted. Lastly, developer experience is also a culture. We want this leadership also to enforce a culture of the importance of developer experience, both from productivity but also developer happiness.

Now I want to talk about a case that we had at Booking, in our developer experience space, related to how we build software. We didn't used to have a standard way of creating builds and creating release artifacts. When we moved to Kubernetes, we let developers take care of this themselves, put it in Artifactory for deployment, and that was the way forward. There was not really good support, no standardization, and we lacked a lot of visibility.

This was not ideal from a compliance point of view. This sometimes required teams, during an audit cycle, to manually provide overviews of all the deployments that they'd done. Not great. We also lacked visibility on tools that were being used, how long builds were taking, and all the performance aspects. We were sort of flying a bit blind. Additionally, we wanted to integrate some tools in this build process for security needs, like vulnerability scanning and code analysis. Integrating that in a process that doesn't really exist, where there is no unified tooling, is also not a good situation to be in.

So we developed something called Common Build. Not a very imaginative name, but it does what it says. It's a unified way of triggering builds. We defined this together with our engineering community and leadership. We talked to them about what are the things that you care about in your build process, and what are the things that you're willing to let go of, where we can be more standardized. We did the same with leadership. We talked to all the different stakeholders, like security, compliance, and change management, to come up with this solution.

In the end, it's not very complex. It's a set of GitLab pipelines that are triggered always at the same time when a merge request is finished, and it ends when your artifact is in Artifactory.

This unified solution meant that we could support it well. This was a small graph of how we were able to reduce the build time, which is quite drastic. Over a two-month period, because we had better visibility, we were able to improve this build time. Then because it was a unified solution, this was a big change and a big win for all engineers.

We were opinionated where it matters. We require that everybody has to trigger builds a certain way. We're going to use service accounts a certain way. But where we gave developers freedom is in how they generate their artifact. A lot of Java engineers like Maven, some like Gradle, some even like Bazel, and Node today likes Nx and Yarn. We're fine with that. That's their call. We provide some guidance, but they can pick this. This is something they cared about and we did not so much.

Essentially, visibility was a huge win. We got so many views on how it was being adopted. There was no more need for manual audit trails. We could see the performance and then optimize it, as I showed. We know exactly the throughput, how many builds are going through, their success rate, how long it takes. We have insights and quality that we get from all these scanners, because it's easily expandable. Since everybody triggers builds the same way and we run them the same way, we could integrate SonarQube easily. We're integrating Snyk now for static code analysis, integrating Sysdig for image scanning, all standardized.

It's very easy now for security and compliance to experiment with different tooling and different integrations while getting feedback on how that impacts productivity and how long it takes to create builds.

What I want you to take away from this is that developer experience is quickly worth investing in. If you assess what your workflow looks like and where you see opportunities to improve this, and you extrapolate this out to the time that it saves, ultimately it easily turns into a number of FTEs, a number of people that you could hire to fill this. Then it becomes almost a waterfall: that team will then be able to spot other areas and opportunities for change and improvements.

It will make your costly developers in your organization more productive and happier about this. The benefits outweigh the cost quickly. When you invest in developer experience, try to treat it as a product. Don't make this a side job. Really invest in product management for this, to design and operate as well. The seven pillars that I talked about can really help you think about how you would want to develop and invest in it.

Thank you.