Mieke Deenen

Welcome, everybody. I'm glad you all came here to see my presentation about how you get started with DevOps in government.

I've created a model, and I will share the key success factors of how you're going to implement DevOps within a government agency, and the important pillars to do so.

But first of all, let me introduce myself a little bit. My name is Mieke. I live in the Netherlands. I'm an independent consultant. I like innovative projects. I like driving change. I'm passionate about agile development and about DevOps.

My current assignment is with UWV, which is a Dutch government agency that is responsible for collection and payment of social security.

I like to be in the mountains: in the summer for climbing and hiking, in the winter for skiing. And this is my family: my girlfriend, Ivana, and my son, Olav.

We believe that people can function best when they can participate in society by working. And we also believe that society functions best when as many people as possible can participate in it by working. So it's our mission to help people to find work, and if work is not possible, then we make sure that income is available quickly.

So that's what we do. I have some figures about our organization. Our head office is situated in Amsterdam. I have about 18,000 colleagues. We execute Dutch social and labor policy. We have about 6.9 million people who are insured with us, so that's actually almost every Dutch adult. And we have about 1.4 million benefit recipients.

So these are the people that rely on us. Primarily people that don't have work, but also people that are ill or maybe are pregnant. They also get social benefits from us. We are the linking pin for government, for employers, employees, tax organization.

Those 1.4 million benefit recipients, those are our actual clients. And a lot of them are a very vulnerable group, because most of them are not very tech-savvy. Most of them don't have savings, so they live from paycheck to paycheck. So if we are only one day late with our payments because of maybe a failure in our systems, then we already get these people in serious trouble. They might not be able to buy their groceries or might not be able to pay their bills.

So for us, it's very important that people can really rely on us.

To understand why we started with DevOps, I have to share a little bit of our history.

In 2012, we had to digitize all our services. Until then, we had a website, which was a plain website with just some basic information, and we had to change it into a dynamic website where people were able to address the points themselves. They were able to upload forms, share their applications, share information with us, adjust things.

So that meant an increase of functionality and also an increase of usage. And we managed to create this in one year, but we suffered some minor stability issues, and that ultimately resulted in a major crash that lasted for several days. So we had a very large unavailability, and that was, of course, one of the worst things that happened in our history.

And the only good thing, though, was that that was our wake-up call. That was our sense of urgency. That was our justification. We really had to invest in our infrastructure and not only focus on new functionality for our customers, but also be aware of the infrastructure that we're working on.

So our main three KPIs were under pressure, and our primary goal, our main goal, was that we just wanted to be able to provide new functionality to our customers on a regular basis in a predictable way, without downtime, without being afraid of having unavailability.

So that's where our journey started.

And now we're getting to the interesting part. Then how do you get started with DevOps in government?

I tried to make a model based on our experiences, and I think that this model is also applicable to other government agencies and maybe even to more process-oriented enterprise organizations. But it is actually the first time that I'm going to present it, so I'm also very excited to hear your feedback on it. So if you have any suggestions or questions, then please feel free to offer me some feedback after the presentation.

Okay. Well, the first part of this model is, of course, strategy.

When you want to start, you first have to visualize your goal and define an approach to get there, because otherwise it just remains a goal. You also need a plan how you're going to get there.

And if you're working for government, then you already know that change is never fast. It takes time to change. And you probably all also know some examples of large IT government projects that failed or had an issue with overrunning their budget. I think this is not only happening in the Netherlands.

And that usually happens because there are large, multi-year waterfall projects, and somewhere in the second year or in the first year, you try to adjust your plan, and that usually has impact on your business case, or you try to do too much at the same time.

So when you're starting with DevOps, then you know one thing for sure: that you cannot know everything in advance. You're starting a new journey, and it is very exciting, and you don't know everything yet. So in order to avoid the failure in the beginning, there's only one solution. You have to start very small.

Start very small, very safe, and once you've gained a solid basis, then continue and expand, because then you won't make the failure of a multi-year project that's getting in the news.

So starting small, that means that you have to find a place where you can start, a place with a good rate of success, the most chances that you will succeed. So you have to check the departments within your organization and find a place where the management is the most eager to go faster and where they are open to change, because they will become your allies.

Then you have to choose an easy application, just mainstream technology. Don't go for the legacy systems yet. And for us, that was customer service division, and we chose a .NET application without a database. Very small, very easy.

And I tried also to show it with the picture. If you're thinking of a bridge towards the other mountain, you might have something more stable in your mind. But this bridge perfectly fits the purpose for this man.

So if you're thinking of an automated deployment pipeline, you don't have to create the best deployment pipeline in the world for this first application. Just make sure that you have a minimal viable product. Make sure it fits for this simple .NET application without the database.

And once you've done that, then you have to create a roadmap and make a plan how to move further. A roadmap is, of course, very agile, and most government agencies still are very process-oriented PRINCE2, ITIL organizations. So you probably have to translate your roadmap in PRINCE2 terms.

So you can write a project initiation document with different phases, with milestones, with a planning. And you can make an agile business case with it. You describe your first phase very clearly and very exact. And the second phase and the following phases, you can still leave them open. You just need a plan when you're going to write those.

And by that, you have created the technical or the theoretical possibility to adjust your plan, and not to already create the whole plan in the first year. But then it's also, of course, very important that you create feedback loops, that you get input, that you have retrospectives, and that you collect your input and your measurements, and that you know how to improve.

The second circle of this model is communication. Communication is, of course, very important. There is operational communication, which you have to do on a daily basis with your team, with your stakeholders. We have kickoffs for every new application that we are going to start. We have retrospectives.

You have to involve everyone. Don't underestimate the communication of this. And it's very important that you also pay attention to people with concerns. Just let them contribute to solve their own concerns, but really talk about it.

Another important part of the communication is about the results. You have to show what you've measured and what you've done. So create a dashboard. Management loves dashboards. Create management reports, monthly reports, and show the business results that you created. Show the financial advantages that you created.

And besides the results, it's also very important that you show what you've learned. And we all know that you learn, well, most of the time, first you have a little failure, and then you learn from it, and then you're getting better at it.

So how do you emphasize what you've learned without pointing towards the failures? And I did it by, I write, for example, an email to all my stakeholders after every successful onboarding of an application. And in that email, I write, "Okay, we now onboarded this application," but I also emphasize what we've learned during that project.

So I emphasize that this is the first time that we onboarded an application with a database, or this is the first time that we onboarded an application with another technology, or that is built on different stacks, or the first time with compliance with the new security standards. And by that, you trigger people, and you gain confidence, and you build confidence in what you're able to do.

Strategy and communications together is your foundation. And once you've set the strategy and the communication in place, then you're ready for the three pillars: people, process, and technology.

The first and most important is, of course, the people. First you need a team. And we chose to develop our own team instead of bringing in external DevOps engineers. We thought that our own people would perceive the DevOps engineers as some kind of rock stars, and probably too fast, probably hard to identify with them.

And we figured it would be best if we would create our own team with our own people, so that our own people could identify with them and could more easily collaborate with them.

So we started it from scratch and let them work together, created the scrum boards, created the campfire culture, and just started by working together and learning and improving.

Besides the team, there are some other people in your organization that are very powerful. So I think these are the stakeholders that you have to involve somewhere and use their strength to go further.

You have a steering committee, of course. They are responsible for prioritizing and for making sure that your resources are available. So involve them and let them contribute. That will also lead to a broader adoption of the change.

A second group are the architects. You cannot change anything in the infrastructure without their approval, so you have to involve them right from the start. And when you have to write project initiation documents, you probably also have to write a PSA, project start architecture. So ask the architects to help you and write down your plans, and make sure that your documentation corresponds with the infrastructure and with your plans, and have them approve it, because that's the only way that you're allowed to go on.

The third group is security. Security gets more and more important. In the beginning, I received a lot of questions about security, a lot of concerns, and I underestimated it. I just said, "Well, it's the same security level as with the manual deployments. It's no change. It might be a little bit better, but it's not worse."

And that wasn't the correct answer. And due to the uncertainty, we weren't allowed to go on.

So I changed my strategy and I embraced the security concerns, and we started to incorporate all the advice from the security board, and we made it as secure as we could. And by that, I could really prove that now we were secure. Our automated deployments are more secure than our manual ones.

And we even had it audited. We had a security audit, and we had a positive advice. So now it's for me very easy when some concerns arise about our security. I can always say, "No, we have been audited. We have positive advice. It is no issue anymore."

And the fourth person that you really need is someone within the department, someone who knows the people, who knows its way around, and who can make sure that you involve the right people. Because otherwise it might cause some issues in planning or in resources, and this person can really help you with the small issues.

So if you involve these people, you will use the strength of the organization. They will become ambassadors, and they will help you to make progress. So let them contribute.

The second pillar is the process. Once you have the strategy and you have a plan and you have the people, then you have to make sure that these people can work on your vision, that they can actually do that. And when you work in a government agency, you will find that some of the existing processes are not ideal for DevOps. So you might have to adjust one or two of them a little bit.

And there were two processes within our organization that I really wanted to adjust a little bit. One is that we are a siloed organization and I wanted people to collaborate, and the other is our change process, which is very slow, risk-averse, and I thought it was too slow for our plans.

So the first challenge was that we have software vendors on one side. They used to work even at their own firm. We have a hosting partner for acceptance and production, and in the beginning, they weren't even allowed to communicate directly with each other. So they communicated through requests for changes through our organization.

And I wanted them to become a team and to work together. And I figured, well, I cannot change the contracts, but I can remove the walls a little bit. I can ask them to work together. I can ask them to come over at our site and work together.

And that wasn't very easy at all in the beginning, because they didn't see any advantage of it. They didn't see the advantage of being away from their own colleagues, working at a strange company. But after a while, they started to know each other a little bit. They started to collaborate. They started to help each other, see who the next person in chain was and what he was doing and what he was actually needing.

So that helped us to get further. So we didn't actually change the contract, but it didn't bother us either.

The second challenge was our change process. This is our change process. If you want to change something in production, you have to announce it 42 days in advance. You have to submit a request for change. You have several tests that you have to perform. You have several quality gates, a freeze period at the end, and when you succeed with all these quality gates, then you're finally allowed to install something in production.

And I figured that was too difficult for us, partly because I didn't know 42 days in advance what I was going to do in 42 days, and also because I believed that this was the exact opposite of what we were trying to achieve.

So I asked change management to adjust the change process, and they didn't want to. So then I invited the change manager to become part of my team, and I created full transparency. I showed him everything. I tried to understand his concerns, and I showed him what we did, and I showed him that we tested everything, and that there was no need to wait 42 days. There was no need for a freeze period.

And I gained his trust, and then finally he said, "Well, okay, since your first release is actually only a technical release..." The first time we onboard an application, we don't do it with new functionality. We just onboard it as it is. So it's only a technical release.

And he said, "Well, okay, then you probably don't have to follow the main change process." And he allowed us to submit a request for change only three days in advance. And that flexibility, it helped us a lot. It helped us getting through the process. It helped us making progress.

So we didn't change the change process, but we were allowed to go on.

The third pillar is the technology part. This is how our automated deployment pipeline looks like.

We have several software vendors with their own development network. We have IBM as a hosting partner with an acceptance network and a production network. We ourselves have the testing network. We have instances of XL Deploy on every environment, and we are trying to create a single package that can deploy on every environment, one and the same package.

And sometimes it's very easy with mainstream technology, but sometimes it's more difficult because the software vendor might have another stack than IBM has, or the application sometimes consists of four or five different technologies from three different decades. Our oldest application is from the '60s, the former century.

So these are our technical challenges. And the only way to manage that is just to do it step by step. Start with the minimal viable product and onboard every application. Just try and learn and make it work.

So once you have the foundation and you have your three pillars, then you have your ideal environment to create results. And we created three sorts of results: technical results, business results, and the third one that might surprise you a little bit, the corporate policy.

A technical result is that you first create technical proof, and you first onboard some of the applications. And once you have onboarded some of the applications, then you can articulate the connection between deployment automation and stability of an application.

We measured the number of incidents, and we found out that the number of incidents for an application with deployment automation decreased when it was automated. So we could really emphasize that we contributed to the stability of this application.

What also helps is the business results. You have to make them apparent. So of course, you have the improved quality, you have a shift left, you have a better availability, and you can enable the business to increase the release frequency. And you can also show the financial benefits.

With our business case, we had to make an estimation of the revenue, and after our first phase, after we onboarded all the applications of the customer services division, we had to measure it, and it exceeded our expectations. And that helped us, of course, very much, because not every project in our organization has a positive business case. We also have a lot of projects because of changed legislation. And when you have a project with financial revenue, then show it to people. It helps them.

And the third was that our architecture board was now convinced of the advantages. They've seen the technical proof, and they've seen the business advantages, and they declared it policy that every application should have deployment automation.

And that was very important to us because that created a switch. It created a shared responsibility. Until that time, we were just offering the ability to the departments to onboard their applications. And after that, after it became policy, it was also their responsibility that they had to onboard their applications. So we switched from pull to push.

And I realized that, especially within government agencies, people are more willing to change due to corporate policy than to technical improvements or to innovation.

So that was a major accelerator, and it was, again, you just use the strength of the organization.

And once you have your first results, then you're getting to a point where you just have to accelerate and scale your project. And we did three things to grow and mature further.

We switched more to empower people instead of doing it all ourselves. We expanded our team, and I created the Center of Excellence and an end user community.

We still onboard most of the applications ourselves with our team, but we also encourage now other people to do it themselves. So when a project team starts to build a new application, they do it by themselves, and we only support them. So we try to switch more to servant leadership.

And we also do that for other project teams. So it sometimes will be confusing for my own team because we have more parallel tracks and we're not everywhere having the same role. But it also is very rewarding because people take more ownership and it helps in getting DevOps within the organization.

The second thing we did is that we added some people to our team. And I already stated that we developed our own people, and these three people are also very much our own people.

The first one is the most critical engineer from our first department. He asked so many questions, and he had so many concerns. He really did not believe in this, and the only way to convince him was by showing him everything and making him help us to solve his concerns.

So after a while, he knew exactly as much as we did, and he was already almost part of our team. And he switched and he turned into an ambassador. And he's now a great help, especially with new applications or people with concerns, because he understands those concerns, and he speaks their language.

And he now has created the SharePoint with a lot of information and all our lessons learned. And all the questions and concerns that people might have are answered on our SharePoint. So he's a great help now.

The second one is a lead developer from our toughest application. We started one application last year that I unfortunately could not onboard. This is, of course, a bit of disappointment. We succeeded with all the others, but this application was the most difficult. It had many stability issues. There were technical issues, organizational issues. It has a very severe service level agreement, so it was very hard to make progress there.

We are planning a second attempt, but we worked on that application for several months and we didn't succeed. And this lead engineer has learned so much, he has really faced so many problems, that he is actually the perfect guy to help other people.

So he was an external engineer and now external developer, and now joined our team as an internal developer, and he is helping other developers to onboard their applications.

And the third person is a trainee. She has no IT background, but she's full of energy and full of ideas and full of questions. And she's helping us making the switch to facilitate more, to help people more, and she's helping us with our Center of Excellence. And with all her questions and ideas, she helps us questioning and improving our own way of working.

So I'm very glad with these three people, and I think they're even better than the DevOps rock stars, because I really can see their contribution within our organization.

And the third thing we did is we created the Center of Excellence. I already mentioned some parts of this, but we have now an end user community. We have end user meetings. We have a SharePoint. We have newsletters. We distribute all our information. We share our lessons learned. We adjust and optimize our own way of working, and we created the space where we can learn and improve and innovate more.

So this was the model, and we are still actually in the outer circle ourselves. We have now onboarded 41% of our applications, so the road in front of us is that we have to onboard the other applications. I think we'll have about 60% or 70% at the end of the year.

We are learning and improving still every day. Every day I have new challenges and new solutions. I still want to integrate with test automation, something we have planned too, and there will be a tender for a new data center. So they'll probably have a new hosting partner and have to integrate with them too.

So this is actually what we are going to do. We want to finish our bridge. We want a stable bridge where everyone can walk on. And we just want an automated pipeline for every application. And it isn't ready yet, but this is what we are actually working on.

So to summarize, I have some advice for you.

You just have to create a cross-functional team, preferably with your own people. It really might help. It takes a little bit more time, but it really might help for the adoption within your organization.

You have to create DevOps values and techniques and use them even if your organization isn't ready to call it that way. So then maybe translate a little bit more.

You have to visualize your goals, visualize your results, make sure that people can see what you are doing. Improve every day.

Use the strength of the organization to move forward. Use the people. People are really wanting to help you and have different roles, so use them to help you. And celebrate your success to move forward.

So this was my presentation. Thank you for your attention. If you do have any questions, I'll be happy to answer them.

Q&A

Q: Just about the 42-day change process. I imagine that's evolved over time in that particular government department to actually manage risk.

A: Yes.

Q: And it's worked for them up to a point, but it doesn't work at the same time. So were you able to gradually evolve it using DevOps practices, but still manage the risk and take them with you? Or did you do something slightly different?

A: We have a discussion about that right now, yes. We're trying to change that as well. We're trying to make that more... Yeah. So it's still in place. Yeah.

Q: So this is not unrelated. I'm very curious how many people in this room are from organizations like the one that was on the top left of your change process diagram, because we experienced something very similar in the British government.

What I'm wondering is if that particular vendor, through the work that has been done for your department, have they changed how they are talking to other government departments, or are they largely taking the same approach?

A: I think that a lot of government agencies are going through this process, and I think it's now much more common that people are working together. There are more agile teams everywhere. So I don't think that we are the contributor to that, but yeah.

No? Okay. Thank you very much. Have a nice day.