Host Intro (Gene Kim)

I'm so excited for the next presenter as well. One of the neat things I got to work on in the past year was working with a group of people who helped lead some of the most progressive software-firsts in the U.S. Department of Defense, including some of those well known, such as the U.S. Air Force Kessel Run, which I already mentioned yesterday.

What's incredible is this amazing natural experiment going on within that community, where there are so many efforts trying to solve similar problems, but in different ways. Among this team, we had this amazing opportunity to talk with over 30 of these efforts with the goal of understanding what are the elements that could explain why some are struggling while some are succeeding.

Our conclusion was that some are organized to succeed. It was exciting because the team that worked on this included Dr. Steven Spear, Trent Hone, who is the author of The Learning War book that Captain Andy Bean mentioned yesterday, and so many others.

But the person who led the team is the person who is presenting: Lieutenant Colonel Max Reele, who is currently deputy commander and materiel leader for U.S. Air Force Kessel Run. He's a longtime software leader, and I've learned so much from him. Here's Max.

Lt. Col. Max Reele

Hey, Gene, thank you so much for that intro. It's an incredible privilege to be here as we take in the journey of all of us into DevOps and digital transformations that we're embarking on for our organizations.

I find it so important to sit back and take a deep breath and really recognize what an incredible community we're a part of. Everybody's going through so much of these pains of digital transformation all together, and I'm here to share a few more with you at this time.

Starting at DevOps Enterprise Forum back in March of this year, I got a chance to meet with an incredible group of people who are also involved in Department of Defense activity trying to run digital transformations for their organization.

The whole reason I was there is because I was, at the time, deputy commander of an organization, Kessel Run. Gene introduced that a couple of times over, but that's the whole reason that I got the opportunity to join into this community.

Kessel Run is an organization that's really well known in the defense sector, but probably a little bit lesser known in such a large commercial and tech community as this, so I'll give you just a quick brief background. Established in 2017, so still a fairly new organization. We did have the privilege of having some of the founders here at DevOps Enterprise Summit this time, and over the course of the last four years, several of Kessel Run's founders and commanders have been a part of DevOps Enterprise Forum and Summit.

The reason Kessel Run was established is because the Department of Defense was historically and notoriously horrible at delivering IT systems that were producing great value for end-user outcomes. Kessel Run looked to break through that. It was really stimulated by the Defense Innovation Board; at the time Eric Schmidt was running that.

It stood up headquartered in downtown Boston. We engendered tech industry best practices for how to enable our personnel and how to bring great cultural norms into an organization within the Department of Defense, which is about as parochial as it gets. This has been a clash of cultural ideology from the jump, and it's something that we still contend with on a daily basis as we merge tech industry non-traditional government employees with traditional Department of Defense born-and-bred leadership.

That being said, Kessel Run has made quite a few advances in this area. We were able to achieve some operational outcomes by employing those best practices of DevOps that we learned from so many of the companies and partners that we've had here, notably Pivotal Labs, which turned VMware. They taught us how to build a user trust model, how to really employ user-centered design and other ideology out of extreme programming as a framework.

You see in some of the photos, you have the highest-level leadership from the United States Air Force. Our undersecretary of the Air Force is in one of these photos. Our chief of staff of the Air Force is in these photos. But what I'm more proud of is the photos that you see of Kessel Runners, our actual software engineers, product designers, and product leadership out in the user community, engaging directly with our users at their operational location so that we could continue to build that user trust model. It's imperative.

So what is the work that we're going to talk about today? The title said that there's a two-by-two, so sooner or later I've got to get to some data that's going to represent a two-by-two. I know everybody's always clamoring for that.

There is an inspiration for this work, though. We got to hear Gene, and we got to hear Dr. Spear talk, and several others leading up to this presentation. Even after me, there'll be another fireside chat with Admiral John Richardson, who is also part of the inspiration for this work.

This has been a really long journey, and it's been a journey for me specifically in which I've been a part of over five products that have gone fully into production with user adoption and still failed. Failed because of poor stakeholder management, failed because of any multitude of reasons that I still personally reflect on all the time and try and learn from, and figure out how I can better myself as a product leader to make sure that these failures don't occur again.

That really is what inspired me to get into the DevOps Enterprise Forum and start to have this session with other leaders of product organizations within the defense sector, so that we could all identify with each other: what are these patterns that we're recognizing about the failed programs that we've all been a part of, and what are the patterns that we're recognizing about the ones that are succeeding?

That was, yeah, sure, a very cathartic session. We all vented for probably about the first six hours of DevOps Enterprise Forum. Once we got past the venting, we started to really articulate what are the variables that we're balking at.

What we noticed was that there are a lot of immovable variables that seem to be there. Anybody familiar with the U.S. government budgeting system: it doesn't move fast, and it moves on the whim of legislators. It's not something that we can really control to resource our teams in the defense sector to make sure that they have very healthy resourcing all the time.

We're all sitting here complaining about not having the right amount of money to put the right teams on contract or to get the right IT tools, and that's just something that we were really wasting a lot of effort balking at, knowing that it takes several years to be able to move that type of variable.

Similarly, your talent management: how are you going to do hiring? How are you going to be able to grow that talent once you have them in-house? Also incredibly difficult to move. It's congressionally mandated, the force that we have, so you only have a certain number of government employees in any one of the services that you represent. Again, very difficult to move that variable.

Lastly is something called acquisition policy in the Department of Defense, procurement probably for most people in the industry. That is founded by law. There are statutes and regulation that are founded by Congressional law, and it's reissued every single year against us. Again, another thing that's very difficult to move. It takes a literal act of Congress.

We're all having our cathartic session about these variables that are very difficult to move, and we get an opportunity to talk to Admiral John Richardson, former CNO. He'll be up on stage here after me.

While we're talking to him about all these variables that we really want to be able to set the condition for, we want to have control of this. We are the leaders of our product organizations. Why can't we have control of these variables for our teams?

He started to talk about the immovable variables and how difficult it is, but the strategies that we could employ. In that time, I, as an acquisitions officer, a proud acquisitions officer, glutton for punishment, told him we have changed acquisition policy. I gave him an example of a very strong acquisition policy that was changed specifically to enable DevOps to exist within the U.S. Air Force's budgetary system, for the whole Department of Defense actually.

What was interesting is Admiral Richardson's response to me, as he stuck his finger in the air straight at me through the camera. It was all virtual. He said: but you're not organized for it. You're not organized for it. You set policy. You think you did something because you set policy. No, that's not the case. You haven't organized your team to be successful.

That was a real wake-up moment. Any time we have a four-star general point their finger straight out here, it's a wake-up moment. But nonetheless, our entire team recognized in that moment that there's something really powerful to the organization of how your team is going to get to employ DevOps.

So we came to a hypothesis. We talked about three huge variables that are really tough to balk at. All of your leaders need to be working that for you at all times. If you exist within a bureaucracy, they need to be working your resourcing, they need to be working your staffing plan and your talent growth plan, and they need to be working your policy to make sure that you could do everything. But that's not something that the internal team can move overnight.

What we came to a hypothesis on is a couple more variables that do exist that you can set the condition for overnight. If you have strong, competent leaders on top of your DevOps architecture, they can make this decision for you right now. You can leave Summit, get back there, and ask them to change these variables for you.

This is our hypothesis: the two variables that we think are very important to this mixture have to do with the lines of authority that your teams have to get through in order to push their code to prod. That's not just from a technical perspective. We'll see the entire continuum of what this variable looks like. It has to do with business operations as well.

The other variable has to do with the proximity to your mission outcomes. I say it that way because I'm a defense sector guy, but really it means: how close are you to your users' desired outcomes, and how close are your teams at being able to build that user trust model with those users to employ those outcomes for them?

I talked about the continuum of the variable. The aggregate lines of authority spans all the way from the far left to the far right. Far left being what I consider the worst of it, where you have to answer to many bosses, maybe over four bosses, to do things like your hiring, to do things like your talent growth, to be able to spend money. You're asking a different boss to be able to do any of those things. I haven't even brought up IT considerations yet, which is why we're all here.

If you look at the testimonials for the quotes that you see there, those are from teams that we interviewed. Gene mentioned in the intro that we interviewed over 30 teams that are executing this way within the defense sector. One of the reasons we're presenting it here is because we think that this holds true across all bureaucratic organizations, not just defense.

You read this testimonial: product leaders must make decisions to pander to the needs of many bosses. That sounds ideal. That sounds like a team I want to be a part of. That clearly defines why this is on the far left of the continuum of how this variable can come into play to impact the success measures for your teams.

As we step through the continuum, you see now we're saying move from the far left where you have many bosses, and you start to get closer to the far right where we're trying to get to a place where you have a singular line of authority who can make all of these important decisions for you and empower your team members to do their job.

I pause here for a second just to allow you to read the testimonials, because you'll see the accounts from members who exist on teams who have defined themselves to be in this place in terms of where this variable scores.

As we start to get to a better place, now you're really only reporting to two bosses. This is kind of like the separation of Dev and Ops that we heard one of the speakers really harping on yesterday. Or maybe your IT structure works one way, but your business operations works different, so you have to answer to two different bosses about whether you're spending money or about whether you're shipping code.

That's still an extra line of authority. That's still two different bosses that you have to serve, but much lower risk to your program having some sort of existential ending.

On the far right, of course, is the most empowering of them all: people with the right knowledge make the right decisions at the right place. Because when you have a singular leader who's competent enough to own this full path-to-production decision space, they can then empower all of the right teams to be able to make those decisions for themselves.

We get to the next variable, which is your proximity to your mission outcomes. In my mind, this one's even a little bit more telling. When you're at the bottom, that means that you have a bifurcation between your DevOps teams and their end user who's going to actually touch the product that they use. What they say here is: we ship things but don't know if they work.

What? You ship your code and don't even get a feedback mechanism to know if it works? So you have to work so hard to have in-app analytics on what you ship just to know if your user's actually able to use the feature or not. You don't even get to talk to them to see if they're using it or not. That's clearly a less desirable place to be as we span across this variable to see how we can get to a more improved place with it.

You get to a place where you're now having to work through a user proxy. It's not that you don't ever get to know whether the user is using your features correctly or not, but you're getting that feedback through a proxy or two, some sort of staff representative.

Next is you can meet with your teams and you do so on some regular recurrence, whether it's through phone calls or interviews, or even pairing interviews where you watch them go through the app. But you're not quite yet embedded in their operational environment, which, of course, is the most desirable place to be for that specific variable.

When you tie this together, you can see that the DevOps teams now are reporting that we can anticipate our user needs based on knowing the context for where they work, because we are embedded with where they work. So we can anticipate our user needs. That helps them build a much stronger product with much less rework at the helm because they're already understanding the context for which it will be used.

You tie these two things together, and all of a sudden you now have the two-by-two. Promised you I'd show it. Still not showing any data. I know everybody's getting hot and bothered for the data. But the reason I pause here is because I don't think the data is that important. The data tells us something about correlation, and it helps us understand whether our hypothesis has merit or not.

But what this really tells us is the power of these variables. When you tie these together, you can start to see different regions for which your teams would exist, if you made the right decisions for them or the wrong decisions for them about how they're going to get to execute.

The real exciting part here is that if you can get your teams into that top-right quadrant, giving them an empowered singular leader so that they can make one decision maker, allows them to run their full path to production, their staffing, and their financing, and you can get them really close to embedded with their mission users, or embedded altogether, you have now given them this incredible authority. You've given them the privilege of accountability.

Now you can hold your DevOps teams accountable for the success of their products or not. If you haven't gotten them there yet, you don't have the right to hold them accountable, because these are decisions that you can be making.

I'm talking to everybody here as though you're leaders in the DevOps community. What I would suggest, though, is if you're at the practitioner level, hold your leaders accountable for this. Go back and tell them, I need to be tighter coordinated with my users, or I need fewer people to have to report to to push to prod.

Let's see the system in motion and see if we can draw some better conclusions. Of the data that we got here, it is starting to unveil it. Now, the 30 teams: this is all self-reported, so we do need to do a deeper level of research if we want to have an empirical set of data to draw tighter conclusions.

But of the 30 teams we interviewed, these are the ones that are self-reporting that execute traditional or waterfall methodology. Interesting where they land. Again, not empirical data, but still telling.

These are the teams that execute some sort of hybrid of DevOps and traditional sequential release protocols. And then these are the teams in blue that are executing purely DevOps, some version of a DevOps framework. The ones with arrows are showing us some momentum for where they think their teams are going in terms of the cross-section of variables.

Your placement on this graph means something very specific, because while they're continuums on the X and Y axis, they are very well defined. You shouldn't be able to fudge where you are here, because you have a certain number of bosses and you know how you engage with your users. Your placement on this graph should hold pretty true.

What I show here is the graph in motion. The two that are connected by a line show the importance of how a variable can change overnight. That's why we need to consistently evaluate where you are on these variables.

Each of these, one of them is for a specific mission reason, and one of them is for an IT Ops reason where they need an extra layer of approval to push to an environment for which they don't have admin access. Nobody wants that in their IT Ops configuration, and you can show it right here on the graph how that hampers the team's ability to execute.

So why are we here? Here's our call to action and ask for help. Please take these variables. Hold your leaders accountable. If you're a leader, hold yourself accountable. Empower your teams to do better and then report back to us.

All of the contributors that have been a part of this work would love to hear how your teams are putting this into practice. Exercise the principles that we suggested here in the hypothesis. Let us know how that's going. Tell us we're full of crap if we're full of crap. If this doesn't pan out, if you make these changes and it doesn't pan out, there's nothing more important about that feedback for us to be able to make sure that we're still telling true stories to the people that want to hear how to better lead their teams.

This is all the collaborators we had on it, but there's a Birds of a Feather later today, and then there's a channel within Slack that I'd ask you to reach out to me on so that we can coordinate more on this work.

Really appreciate you listening. Please put the policies into practice and let us know how it's going out there. Good luck to everybody.