On the Path to DevOps and Lean Government
Daniel Franzen is a co-founder and CEO of Bustard and a consultant in the Enterprise and Solution architecture area. He is helping organizations develop their Enterprise Architecture organization so it can strenghten the relationship between IT and business and provide means to develop strategy and innovation that is visible in everyday execution. Daniel has been a consultant at the Swedish Migration Agency for four years.
Jonas Elmqvist has a research background in the area of models and methods for development and verification of safety critical software. He has since then worked in the information security business with secure software development and information security management. Now, he is the manager of one of the software development units at the Swedish Migration Agency.
Chapters
Full transcript
The complete talk, organized by section.
Khaldoun
Imagine yourself away from your family for two years. You're not allowed to work. You're not allowed to mix with society. You cannot do anything, because without a decision and personal number, you are nobody.
I just tried to start a new life and enjoy life again, which I lost. I lost my smile for almost two years.
Jonas Elmqvist
So this is Khaldoun. He came to Sweden two years ago during the refugee crisis. He's originally from Syria, and he had to wait in Sweden for two years before he got his decision to stay and finally be reunited with his family, as he said, and start a new life.
But you might think, what does this have to do with Agile and DevOps? Well, first of all, thanks for letting us be here and listen to our story. We're from the Swedish Migration Agency, and this is our why. This is our purpose.
We're not here in a competitive market. We're not here to make money or make profit. We're here to help human beings coming to Sweden and minimize suffering for people like Khaldoun. So that's why, if we are able to create software or systems that can help our business to take faster decisions, we will minimize his suffering.
And also, the story he tells us, and other refugees that flee throughout Europe, they have one important asset with them, and that's their mobile phone. This is how they communicate. So we need to become digital, we need to become Agile, and we need to adapt to our users' needs.
The Swedish Migration Agency has a couple of missions. We process applications from people who want to come to Sweden and visit, or maybe work or study, or be reunited with their families or loved ones.
We also help people to return home to their home countries if they want to leave. And we handle applications for people who want to become permanent residents or become Swedish citizens, or protect people like Khaldoun, of course.
The agency has around 7,000 people employed throughout Sweden in around 50 offices, and we are from the IT department. The IT department is located at the headquarters in a city called Norrköping. Anyone know Norrköping? Some people do, right? Are you Swedes? Yes. Welcome. You don't need our services then, I guess.
The IT department has around 300 employees, and we have pretty much everything in-house. We have our own data centers. We have our operations teams working with ITIL, for those of you who listened to the previous talk. We have around 12 Scrum teams working as well.
My name is Jonas. I started at the agency around five years ago. Back then, I was the IT security manager, and we're going to tell you about the story that started two years ago, when we tried to introduce DevOps and Lean thinking into our organization.
Back then I was the IT security manager, but since then I had the nice opportunity to take over and lead one of our software development units as well. It's a unit which has four development teams, and a security team, and a platform team, and a middleware team.
Daniel Franzen
My name is Daniel, and I'm a solutions architect, and I'm a consultant. I've been at the agency, today, five years exactly. And my background: I've spent the last 22 years starting up and leading different startups.
Our story started two years ago with an IT department which suffered a lot of problems, both that impacted the business and with an architecture that really made it hard to do changes. Even small changes needed the delivery of up to 20 different modules and took about 30 days to deliver.
Of course, there was a lot of frustration. And the frustration was both within and outside the IT department.
Jonas Elmqvist
Because of this frustration, there was a lot of pressure on the teams to deliver things very fast, so they had no time to do things like clean up or innovation. So that made things even worse with technical debt.
Me, as the security guy back then, I also thought that we didn't do things the right way in terms of security. We were, as many other organizations, coming in late. We were finding vulnerabilities late in the process, which was expensive to fix. So we needed to shift left in terms of security.
Also, as I said, we introduced Agile around 2009, and we have Scrum teams working with Scrum, but sometimes we don't really follow the Agile Manifesto to the full extent. In this case, we sometimes focus more on the method and the processes instead of focusing on getting the actual work done.
And Daniel, you have an example you can show us.
Daniel Franzen
Yeah. This is a change request for a really tiny change. It's a misleading parameter name in a method. It's a really tiny thing. It takes five minutes to fix for a developer.
But we didn't fix it. We wrote this JIRA, and what did we do with it? Well, we put it in the backlog, and then we planned and planned and planned. So it was in and out of sprints and up and down in priority.
I think this is a good example of what queues do, and how much it costs with queues.
We were these three handsome guys thinking that we have to do something about this. So we had a lunch, and we had read The Phoenix Project book, and we started to talk, and we thought, "Okay, we have to start this journey."
So the first step was to convince the IT board that something had to be done. We got approved, and we got a committee, of course. We are a government, after all.
This small group started reading and listening to podcasts and taking a lot of discussions. We ended up in a concept we called Phoenix. What we tried to do was combine more than the traditional definition of DevOps. We tried to put in things like security, business architecture, culture, and so on.
We used different tools like Lean, Lean Startup, DevOps, and design thinking. And we made this poster to do some internal promotion of this inside the agency.
The vision was to be Sweden's most awesome workplace, and the mission was to minimize lead time to business value.
Last summer, we started the pilot and tried a tiny software thing just to try this out. You can see this like a minimum viable product for this concept. The results were good, and we learned that we have to learn more about continuous integration.
So early this year, we started a new initiative to learn more about continuous integration and continuous deployment. And now we are here.
Jonas Elmqvist
As Daniel said, when we started this two years ago, we had some problems that we had identified: technical debt, no time for innovation. But one thing that was even more alarming was when we talked to some of the team members in our teams.
They said that the engagement in the teams and the motivation was getting lower and lower because of this, because the process was slow. Sometimes we had old techniques. So this was alarming for the IT board as well, because we were perhaps losing employees who would find another job that would be more interesting.
So motivation was one thing that we wanted to address as well. The other thing we also heard was that the people working were feeling that there was low trust in our organization, and that's quite typical for being a government agency. There's a lot of oversight, and there's a lot of bureaucracy.
But those two things are really important, because if you read the State of DevOps Report from 2016, and if you read the Accelerate book by Nicole Forsgren, these two things are drivers for efficiency. If you don't have trust and you don't have motivation, you won't get any success in this.
So this was one of the drivers that we told the IT board two years ago that we need to address, because of not only the technical problems we had, but the employees were not feeling great.
We started looking at how do we drive motivation. How many of you have read the book Drive by Daniel Pink? Then you know this, so I'm not going to talk too much about it. But he mentions three things that drive motivation.
The first is purpose, and that's the movie we showed you: why you are at the workplace will drive motivation.
The second thing is autonomy. If you go to work and you are able to decide a little bit about what you do, how you do it, and when you do it, you are autonomous, and you will feel motivated. This is what DevOps is all about as well.
The last thing he says that drives motivation is mastery. If you feel that you're good at something, you're an expert, or at least you're improving your skill set, you will become even more motivated.
So these are the things that we wanted to address as well, besides the technical and the architectural problems we saw.
We started looking at the term that many people talk about. For example, Steve Mayner yesterday talked about transformational leadership. He has some great presentations from previous summits. I think it was last in San Francisco. So if you want a deep dive into transformational leadership, you can do that.
But for us, it's about leading with your heart. You have to lead with your heart and believe in your employees and build trust by having transparency as well.
What we learned throughout this journey is also that you have, as a leader and as every individual in the organization, to challenge status quo, because there are things that we need to change in our organization in order to change culture. In the end, it's about changing culture.
We wanted to create a space for collaboration because we saw that teams didn't collaborate. We wanted to change the culture to learning and improvement, and also to time for innovation. In order to change culture, you change how people behave.
So we started out by thinking, "Okay, so how do we change the behavior of people?" And we asked our employees as well, "What do you think that we need to do in our organization to drive motivation and innovation and collaboration?" And they actually came up with a lot of good ideas that we listened to.
Not all of these are based on the Phoenix initiatives. It's based on everything that the people have done throughout our organization the last couple of years.
The first thing we did was I talked to one of our employees who has a background in mobile development. At that time, we didn't have the capacity or the ability at our organization to build mobile apps. This was something we needed to do to follow our strategy.
So he organized a small hackathon, a code night, where we invited the IT department. We shared some pizzas, and those who wanted started learning building mobile apps in order to increase collaboration and learning.
What we also did, we wanted to spread good ideas and innovate. So we arranged something we called MIG Speak, which is an open space session, exactly the same as the Lean Coffee we have here at the summit.
We actually had four speakers that spoke about different concepts, and then we had roundtable discussions with different people throughout the IT department in order to get good ideas to work further on.
Actually, the movie about Khaldoun that you saw was part of one of these ideas that one of our employees had. He talked about empathy at the workplace, and he wanted to spread the feeling of purpose. So we encouraged him to do these three interviews. This is one of them, and he interviewed two other people, end users or clients to the agency.
These interviews we've used inside the IT department to show to all of the teams and discuss what this actually means for us. Also, the feeling they have when they use our IT systems, our electronic applications, for example. And we realize what our customers' needs are and what their feelings are when they're using our systems.
What we also did is our IT director, she wanted us to focus a bit on innovation. So we went to a conference site for a two-day innovation conference where we had seminars and talks about innovation. Daniel, you were actually one of those who organized it.
Then we actually had the teams had problems that they solved, and then they went up to a Dragons' Den situation, or a Shark Tank, and presented their solutions to be judged.
Daniel Franzen
I don't think any of them really went live afterwards.
Jonas Elmqvist
They were quite spectacular.
Daniel Franzen
They were spectacular, right. At least.
Jonas Elmqvist
Drones were involved in some of those.
Then, as I said, we realized that some of our teams didn't collaborate. We heard that they rather sent an email or made a JIRA instead of going and talking to each other. Even though we're in the same workplace, we are just another corridor away. This was one thing we really wanted to address.
Since we're from Sweden, you know about the Swedish fika, right? So what could be the best thing to combine collaboration and fika and cinnamon buns?
In Sweden, we actually have almost mandatory, you need to go and have a coffee at 9:00 in the morning and 2:30 at our workplace. So we actually bought some cinnamon buns and said, "Come along at 2:00 this Wednesday and talk about DevOps."
This started in February. The first one, only 30 attendees came. We were glad about the 30, but only developers. And now in May, the last one we had, over 70 people came, and both dev and ops started discussing ideas and started a bit of collaboration between the different silos we actually have.
What we also did is we arranged something we called Show and Tell, and that was an idea from one of our employees who worked at Ericsson. Ericsson has the same idea, that they invite people to a one-hour talk where you can submit a topic of interest, and you speak for 15 minutes, and you share ideas or just something you recently built that someone else could use.
We actually also had our business coming on to the IT department and talk about their experience of how they use our system, how they are affected when our systems are going down, so our developers and our operation teams understand the reasons why we need to have high availability, for example.
And as Daniel said, the last thing we've been focusing on is continuous delivery. We realized through this journey, if we want to introduce DevOps in our organization, we need to build another type of skill set, start learning more.
We have arranged some courses in both continuous delivery and DevOps in order to start learning these things, because we're seeing the teams are trying to improve, but the next step is we need to educate each other. And also the leaders as well have gone to some of these courses to understand what this implies to our organization.
Daniel Franzen
One of the things that we wanted to address with the Phoenix Project was aligning to strategy, because we think that the reason for doing DevOps is to be as fast as possible to bring business value.
But we also think that you cannot always ask the customer what the business value is. Steve Jobs once said that it's not the customer's job to know what they want.
Let me give you one example. One of our main focuses right now is to transform our government into a digital business. This is, of course, nothing we can choose to do, because it's happening around us. But we can choose to be disrupted or to disrupt ourselves.
And as a government, this is kind of problematic, because we are not going to be disrupted. There is no competition. No one else can do the job we are doing. So we have to disrupt ourselves.
That is exactly what strategy is. Strategy is the decisions you make to be competitive in the future. And we want to have these things in the Phoenix Initiative too. This means that strategy is the customer too. We see strategy as the voice of the future customer.
We want this transformation to be something that we push, not we react to. So waiting for customers demanding digital services is too late. We want to be preemptive.
But this also means that sometimes the feedback loops are hitting us, because sometimes the users are not that ready for the digital way of doing things.
We think this is a hard thing, because you can have negative feedback on something you know is part of the future, because you have to move the customers too.
We have had a lot of discussions about this. Is this some kind of big design up front? Because we want to rely on real customer feedback.
One thing that can be confusing is if you have a lot of internal customers. Do you have internal customers? Yeah.
And if you see internal customers and focus on internal customers, Steve Denning wrote this last week in Forbes: "In agile management, there is no such thing as internal customers. The only purpose of work is the ultimate end customer or the end user."
Don't get us wrong. Of course, the internal customers do a lot of things, and sometimes this is really value creation. But if you don't have a clear line of sight to the real value for the customer, chances are that you're sub-optimizing.
We believe, therefore, that you have to have a healthy balance between continuous improvement and strategy. You have to have both what Lean calls kaikaku and kaizen. Kaikaku is the disruptive change, and kaizen is the continuous improvement.
Maybe you have seen this quote: "The light bulb was not the result of continuous improvement of the candle." Of course, it's like that. So you have to have disruption as part of a process, especially in the digital age.
So should we be afraid that this is some kind of big design up front? No. We think that we have to have a value-driven solution design process that always validates the solution we do. I'm coming more to that later.
We like the pattern: think big, start small, fail fast or scale fast.
So how can we inject this strategy thinking into the everyday development process? We think that the solution architecture practice is a natural way to incorporate this strategy in the normal process.
But we have to rethink this architecture practice, because the traditional way of ivory tower architecture, and the geeks with ties that once now and then came up from their office and complained about things in the real world, we cannot have that kind of architecture.
We have to have architecture that brings real value and aligns strategy. So we want this value- and solution-focused architecture as a natural first step.
And we see there are two main goals. The first is to make sure that the solutions are aligned with the strategy. Because we can make decisions about the future, we have the possibility to think and make decisions about the future.
But at the same time, the second one is to make sure that the solutions are adaptive, because we cannot predict the future. We can do things exactly as we can plan a trip, but we don't know if a flight will be canceled. So we have to adapt to changes down the road.
And to force us into the right mindset, we developed this mindset of seeing architecture as a startup or entrepreneurial process. Just like the early phases in a startup, we think that you have to validate your architecture as fast as possible in the cheapest possible way.
To do this, we introduced this simple process that is based on both Lean Startup and design thinking.
I won't go through this in detail, but I want to give you one example of how we have used this. We had this big project where we were asked to create or develop or buy a new document production system.
We have one that is home-built, and the users' demands were quite clear. They said, "We want something that looks like and walks and smells like Microsoft Word, but with the integrations that we have in our home-built one."
But what we did was take a step back and ask why. Why do you need this software? And we found two reasons.
The first one was to keep records of everything we do, and this is regulated by Swedish law, that every agency has to keep records. But we found there are other ways that are better to do that.
And the second reason for creating documents is obvious. It's to communicate. So then we asked: what is the main communication channel we will use in the next five to ten years? Because if we are going to invest in a new, modern way to create documents, would this be the standard letter format?
And we don't think that printed letters are the main channel in the future. So we ended up with two principles. We call them architectural principles, but you can call them strategic decisions if you want to.
The first is we shouldn't compose documents manually. They should be generated from digital information inside our systems. And the second is, we should communicate with the best channel for purpose and not default to paper.
Of course, this is not rocket science, but for us, these were two really important principles when we designed the system.
So this is what we ended up in. Instead of the left one, where the users are composing mails by themselves, we now enter information and have that information digitally in our systems, and can use that information for everything that we can do with digital information. And then we can generate information for any channel that we want to.
So we can send this by mail, or by paper, or in the new Swedish digital mailbox that every citizen can have.
Jonas Elmqvist
So the results so far. Well, as we said, we tried this in a small scale in a pilot project, but we haven't scaled larger than this project so far.
One of our development teams, actually the one that was participating in the pilot, we have one of our team members actually sitting down there. One and a half years ago, we started measuring the lead time from commit to production. They had an average of 30 days back then, and now it's down to three days, right?
Daniel Franzen
Right.
Jonas Elmqvist
We're really happy that this team has come so far, but we have 11 other teams that are making progress, but not as far as this team.
Also, we believe that we should have a humane workplace. We believe that you should not be able to come to work on weekends. This has been the tradition at our place. Once a month, on a Sunday night, many people come in, and we do a large delivery.
A couple of years ago, this was actually 85% of all of our deliveries were made on Sunday nights. This is something we try to come away from. The mindset is wrong. We should work on weekdays instead of weekends. And now the last figures I saw were down to 35% of all deliveries on weekends. So this is something good.
And the most positive thing is the positive feedback we are all getting in the department right now, the positive vibe that we're feeling. Here are some of the example quotes we've heard during the last couple of months. Really optimistic quotes. People are feeling hope that there's something going on, at least.
But we still have lots of things to do.
Daniel Franzen
As you might have noticed, we say a lot of things like "we believe" and "we think." Of course, we are in a learning process, and this goes along with us.
We're now getting in the next phase and trying to scale this up in a couple of new initiatives which are slightly bigger than the things we have done before. But we have a lot of things to learn.
For example, if you have the solution for where we can put our architects, centralized, in team, or something between, please come to us and talk to us. Or if you have some other questions or something you want to share with us, please contact us. We are on Slack, and we are on LinkedIn.
So thanks a lot for listening.
Jonas Elmqvist
Thank you.