DevOps: Making the Middle Great Again
After gaining support from leadership and the engineers in the trenches, my initial success began to falter. After meeting with each business unit it became clear that middle management was not incentivized or driven to adopt a DevOps culture. My new focus became understanding the incentives, drivers and fears at the middle management level and addressing the factors that were blockers to DevOps success.
Chapters
Full transcript
The complete talk, organized by section.
Pauly Comtois
Okay. So when I wrote the title to this talk eight months ago, it was a bit of a tongue-in-cheek joke, and I never expected to be standing in front of you today with what I think of as an election hangover, even though I didn't drink. So we're just going to try and have fun for the next 30 minutes.
So I did something different for this talk. I actually decided to write a story. It's a fable, and it really encompasses the last two years of work with middle managers and how they helped to drive DevOps transformation.
I decided I drew some pictures too, so I guess maybe you'll just have to bear with me. It's a terrible self-portrait, isn't it?
Imagine you're an engineer at an enterprise organization. You love your job. You love who you work with. You even love your boss. See, I told you it was a fable.
Oh, wow. If you liked that one, this is going to go great.
You're part of a small team, but you're also part of something bigger than just yourself. You've spent the last few years streamlining your processes and consistently hitting every review goal ever put in front of you. Thus, you get promoted to management. Among challenges, rewards, lots of promises, you're excited. You're ready to take on the world. You're in the perfect environment, with the perfect people, in the perfect job.
And then you meet your new boss, the middle manager.
Now you know the type: the kind of boss who refers to himself in third person and says things like, "Teamwork is a lot of people doing what I say," and, "Wow, I didn't know there was a South Korea."
Now, the fact that you didn't know you even had a boss before now, that's just not going to get in your way. You're going in full of vision and promise and possibility. You're just going to keep doing what you've always done, what's always worked for you in the past.
And then reality hits you.
Yet you keep your head up. You plan to defy all odds. It's a bit like agreeing to go on a blind date and bringing an engagement ring, isn't it? There's optimism for you.
The problem is that you don't understand your new boss, this middle manager. It's as if he doesn't even want to be there.
Now, not giving up, you try to come up with innovative things, ways to make life better. You decide to take some risks, and before long, you're called into his office, and he says, "Kif..." Your name is actually Sam. "Kif, management is all about empathy and caring." Also, he sounds a little like Zapp Brannigan from Futurama. "It's about putting yourself last and being able to communicate and encourage your staff, like me. You must be able to appreciate them and ease their fears. If you can't get that through your thick, ignorant skull, Kif, you've got no future here. If you have any questions, this is Toby, our grief counselor."
And that was basically your introduction to middle management.
Slowly, surely, you're worn down like water on a stone, and pretty soon, you don't care either. This is how the cycle begins, how it perpetuates.
That was me. That was my life. I had agreed to go on that blind date, and now I was stuck out at a restaurant with someone using their own hair as floss. Not that I was any special catch either, mind you.
But that was many, many years ago, and in the time since, we've seen the advancement of executives to embrace Lean and Agile and DevOps. We've seen engineers adopting the principles of DevOps by the thousands. Look at all of you here today.
And yet, many middle managers hold onto the old ways of doing things, doggedly riding their donkeys to pick up their blind dates.
But why is the middle so averse to change, even when their organizations and their worlds are changing around them?
I sat in a coffee shop on a cold, wet Seattle Wednesday in September, or maybe it was July. There's really no way to tell in Seattle. We only have one season. It's called rain.
And I watched Howard walk through the door, hair drenched, matted to his head, water dripping on the collar of his Seattle-mandatory North Face jacket. He looks at me and says, "Umbrellas are for tourists."
Now, Howard's about this tall, and he's built like a fire plug. He has a long, pointed beard, and he's immensely proud of being a level 60 paladin and weekend dungeon master.
I love Howard. He reminds me of the guy in a video game that will sell you rare items you're going to need later in your quest. But Howard is facing a problem that no amount of casting magic missile is going to solve. He's pulling out his 20-sided die, and he's rolling saving throw by coming to me.
Flopping into his chair, he heaves a big sigh, and I say, "Okay, Howard, on a scale of one to Nature Valley granola bar, how badly are things crumbling apart for you?"
Howard's a middle manager, a senior director of engineering. He has 12 teams that report to him, encompassing developers and operations. Now, his boss, Dave, is the CTO of the organization and the main reason I was with Howard that day.
I had worked with Rachel, a dev manager within Dave's organization, and helped start a grassroots groundswell campaign. They loved DevOps. Now, this caught the attention of Dave, who said, "As an executive, why should I care about DevOps? This is something for engineers, right?"
Now, I encouraged Dave to read books and blogs for context, attend conferences like this one for fellowship, and use the State of DevOps Report for tangible clarity.
Now, Dave really became a believer. But the problem was, he couldn't make any progress with Howard, and he liked Howard. Howard was still the right person for the job. But Dave was changing the way the organization saw middle managers, so he reached out to me, which brings us back to the coffee shop.
"He's crazy, you know," Howard tells me over his coffee.
"Who?" I ask.
"My boss. He says we're not competitive in the market anymore, and that we need to rethink how we create and run software, starting with how we form and lead teams. He actually asked me what the best organizational layout should be."
"What do you think it should be?" I asked.
He sits up quickly and says, "It should be the way it is now. I have talented developers creating software. I have talented operations running that software. We get features out. We manage outages quickly. This stuff has worked for years."
Howard slumps back into his chair and sighs and says, "Now Dave wants to blow all this up. It's going to cause problems that I'm going to get blamed for. I sure hope I like the underside of the bus, because that's where I'm going to be spending most of my time."
Howard continues and says, "Listen, I tried to tell him that this was all nuts, but it's like bringing a horse to water, and I can't make him drink. I even tried holding his head under, but all he did was blow bubbles."
Now, I think back to my own conversation with Dave. I'd asked Dave what his new vision was for this company, and especially for middle managers. Dave wanted to provide context and build an organization that shared responsibility and accountability. He wanted everyone to feel empowered to make decisions and take on some form of leadership. He wanted to remove the middle management command-and-control structures of the past.
Now, Dave knew he had to model these behaviors for Howard, but Howard just wasn't interested in these changes.
Now, Dave came to the realization that his organization was misaligned. Goals were putting middle managers at odds with each other. He had to work to regain that alignment. Having each middle manager be successful at their own agendas in their own silo did not make the organization a successful one as a whole. He wanted to break down barriers between teams, share that accountability, and really work towards a unified goal.
I told Dave, "We look at the world through two panes of glass, a window and a mirror. Now, the healthiest of us look through both. When we look in the mirror, we look to ourselves for accountability. And when we look through the window, we look outward to others for accountability, and oftentimes blame."
It's very hard to follow a leader that only ever blames someone else, isn't it?
Ironically, this was the tactic of my son when he was very young. As an only child, he had to be very creative on who to blame when things went horribly wrong. Like when one of his imaginary friends cut his hair for him.
Now, his parents didn't believe in imaginary friends, of course, so the family pets became the next targets of blame. Like when one cat somehow magically shaved the other one.
He had some sort of fascination with cutting hair. I don't get it. Nine-year-old. But we were on a pet reduction program, and as the last of three cats went to the great scratching post in the sky, he had to face reality. He had to look into his own mirror.
I said to Dave, "Now we just need our middle manager, Howard, to look into his mirror."
Now Dave, wanting to hold himself accountable for starting the change, decided to shift the culture from one that rewards people only because they have mastered the current system to one that rewards those that focus on growth and innovation.
Changing a culture is very hard. Instead, Dave focused on influencing the culture by creating incentives that would drive behaviors to reinforce desired outcomes. Dave held an all-hands meeting where he laid out these changes, starting with why, the context. Providing this context exhibited trust and sharing and openness from the top, and Dave, he hoped it would pave the way to the same types of changes in Howard.
The problem was, you see, Howard didn't have aspirations for executive leadership. He was perfectly content in the little world he had created for himself. This was the old-world position of middle manager, and Howard was great at it.
He was not really interested in DevOps, or "Dev Oops," as he called it.
But Dave had changed things on him. Gone were the days of having a neck to wring when something went wrong, and instead, technical leadership tasks were becoming a shared role.
Now, back in the coffee shop, I said to Howard, "We need to broaden our sense of middle manager identity."
We're born empathic. As a baby, we cry when we hear other babies cry. It's innate. But if we are taught not to be empathic by parents or education or business or even government, it makes creating a DevOps culture almost impossible. Creating empathic organizations starts with those that set culture and beliefs. In other words, it starts with you, Howard.
Now I sip on my coffee and continue saying, "You're in the unique position to have influence both up and down in your organization. Dave has empowered you to focus on innovation and growth, which means you must close the gap between the business and the technology. Focus on letting your teams drive their own automation and speed, provide mentorship and guidance, and then provide contextual feedback that the business needs, and in terms the business can consume."
Now Howard looked doubtfully at me and said, "Listen, if I automate everything, and I let my managers make all the decisions, I've automated myself out of a job. I've worked too hard to get to 12 teams and a $4 million budget just to throw it all away."
I sat up in my chair and I said, "Empowering your managers does not diminish your value to the company or your teams. Quite the opposite. Your value is in sharing your experience and guidance. You focus on the success of the organization by providing the leadership that the business and the technology require. If you judge yourself on the results of your teams, then you'll see increased benefit and value from DevOps. If you value only yourself and what you can collect, you're likely never going to see that benefit."
Howard sighed and said, "Okay. Look, I get it. All this stuff logically makes a lot of sense to me, but you got to understand, I've been successful at doing things the way I've been for years. This is a huge change. I'm not even sure I know where to start."
I said, "Okay, well, that's fair. How about we start small, and we iterate, and we build on that? How about we start with how to lead with a DevOps mindset? We focus on leading with humility and chutzpah. Then you can encourage your managers to do the same thing.
"Now, humility means you don't have to be the smartest person in every room, or the one making all the decisions to be successful. You must build trust with your managers by showing you don't have a need to be invulnerable. Having a huge number of employees or large budgets won't add value to the business or, let's be honest, secure your place in it. Give your managers more control and require humility from them, and then make sure you model that behavior.
"Now, chutzpah is being courageous, but courage is more than removing a headphone jack from a phone or an escape key from a laptop. It's bravery to the point of almost audacity. It's enabling risk-taking for your managers. Now, when I think of chutzpah, I think of the story of the boy who killed his parents and then, when hauled before the judge, begged for leniency because he's just a poor orphan after all. That's chutzpah. Not that I suggest you kill your manager, even if you sometimes fantasize about it.
"Well, let's be introspective for a moment," I said. "Howard, tell me something about yourself, something about you, not work-related, something that will help me understand who you are."
Now Howard, a shy person by nature, starts dubiously. He says, "Well, I'm from the Pacific Northwest, so I'm overly concerned and passive-aggressive. For example, I drive 40 miles per hour in the fast lane, and at the same time, I'm always aware of my windshield wiper speed, looking at other cars to make sure I'm not being too dramatic."
I said, "Well, that's a start. How about we spend the rest of this hour getting a sense of the type of leader you want to be and the type of leader your managers need you to be?"
Now, at the end of our meeting, we had uncovered the following challenges for Howard. And while the final list isn't exhaustive, it's where we decided to start.
Howard was too involved in the day-to-day, borderlining on micromanaging his managers. Maybe more than borderlining, if you ask the managers. We decided to push management and decision-making down closer to where work was being performed. Howard would create guardrails and clearly define what those were so that they could take risks, as well as remove a lot of the ambiguity from the day-to-day. Stepping in only when absolutely necessary or when asked, this was a more subtle DevOps type of management approach for Howard.
Howard's leaders were locally incentivized and individually incentivized. Our solution was to align team-based goals rather than individual goals, and align those around the objectives of the overall business. This created transparency between the business and the engineers, and Howard provided that transparency while giving the teams and leaders the freedom to make the decisions and risks.
Time constraints were killing productivity. We decided to set strategy around new processes that could add visibility and identify issues with work and process and bottlenecks. We could experiment with cross-functional teams and reduce meeting burnout. As a middle manager, Howard was uniquely positioned to have visibility across teams as well as the technical context to know where they would work best together.
Work metrics were hidden inside the teams, with Howard managing the executive status reports manually. Think pivot tables here. Our solution was to automate report and metric collections and remove Howard from that process altogether. Howard would make these visible to everyone within the company, not just the executives, regardless of whether they were good or bad, and then make this visibility an incentive to innovate and improve.
And of course, all of these challenges would encompass a continual focus on putting the customer needs first. This sounds like a simple goal, but sometimes we lose sight of this crucial objective.
The following week, we focused our meeting at the coffee shop on how we could begin to move decision-making in Howard's teams closer to where work was being performed. I said to Howard, "You need to lead this change, not just manage it. When people are uncertain, they will look to the actions and behaviors of others to dictate their own."
He needed to humanize himself with his employees. He knew that he was too closed off with his teams personally and micromanaging them professionally. He needed to start leading conversations with questions rather than statements and orders. We decided an AMA, or an Ask Me Anything, would be a great way to open up communication and allow Howard to show some vulnerability as a leader.
Our next meeting in the coffee shop, Howard told me how the AMA went. He said, "I brought everyone together, and while the conversations were a little slow in the beginning, we actually had productive output. They asked if they were all getting fired, or outsourced, or reorged, or automated out of a job. There was a lot more fear than I had expected."
Howard said, "I explained we were all in this together, and I took that opportunity to have a culture session where we reinvented our culture as a group. We listed out all the things that we wanted to see in a new culture."
Now, I got some eye rolls at this, and one guy even sat in the corner with his arms crossed and huffed and said, "Nope, I don't believe in culture. It's like the moon."
I said, "You don't believe in the moon?"
He said, "Nope. Everyone knows it's just the back of the sun."
Howard gave me a guilty smirk and said, "SharePoint administrator."
Giving a knowing nod, he continued.
Howard said, "Most people didn't want to share a belief. They thought it was too hippie trust fall, like they were afraid we were going to start burning patchouli in the office. So instead they decided to share a benefit, something about a company they'd love to work for. Then we'd convert that benefit into a belief. For example, 'I'd like to work for a company where we can work in a coffee shop,' became, 'Our culture believes that work happens where people are, not just within a cubicle.'
"We used Lean Coffee voting to get to our top 10 core beliefs, and then we talked about how everyone would encourage the behaviors that would drive this new culture. Our focus would be on empowering teams to make their own decisions."
Now, at this point, Rachel brought up how language and attitude play such a vital role in DevOps and innovation, how it can bring people together or drive them apart. She said, "We need to work on our paraverbal communication. It's not just what we say, but how we say it."
Howard continued, saying, "As a group, we decided we'd work together to move away from negativity. Negativity can be acidic to a culture, although just once I'd love to hear someone say something nice about the Perl ticketing system I wrote."
I said, "Don't hold your breath."
Howard laughed and said, "MVP stands for minimum viable Perl, right?"
I said, "No."
He said, "Okay. Well, how about we work to remove complaining? Now, I know that complaining is the national art form of my operations team, but complaining can become viral misery. If our world is miserable, what are we doing to make it better? And no one is in a better position than the people who have to live with it.
"So we would focus on solutions rather than only complaints, and instead of asking why, we would ask how. How you did something invites conversation. Why you did it can sometimes lead to people feeling defensive.
"We'd move away from, 'That's just always how it's been done here,' to, 'How can we do it better?' And finally, we'd move away from, 'That's not my job,' to, 'How can I help?'"
Howard said, "We closed the session by agreeing to encourage each other to grow the culture. Transparency and openness would be our north and guiding star."
I asked Howard at this point, "How can we start creating a tangible plan with measurements to achieve these goals?"
Sighing, Howard says, "I honestly don't have any idea."
And I said, "Well, let's think through this."
And Howard said, "Oh, I have an idea. How about we use one of those value-streamy things you were talking about?"
I said, "A value stream mapping workshop? That's a great idea. We've already seen what bringing people together can do for starting to rebuild a culture. Let's see what it can do for building a tangible plan."
Howard looked at me for a long, long moment and he said, "I'm just not sure this is going to work if I'm not at the wheel."
I said, "Okay, listen. Remember, you don't have to be the one driving all the time. This will all work out in the end."
"What if it doesn't work out in the end?" Howard asked.
I said, "That's easy. In DevOps, it just means it wasn't the end. There's more work to be done."
Howard leaves the coffee shop and goes to work on the VSM.
Now, the next week we meet again, and he tells me how things went. He said, "The managers picked the development value stream to do the mapping on. It was very enlightening. Even the executives understood it by the end. I think it was all the colored boxes and arrows.
"Bringing everyone in the value stream together at one time allowed us to see a view of software development we don't normally get to see. We saw the full life cycle and where everyone fit into it. We even had a couple of aha moments where people found processes most didn't even realize we were still doing.
"The team was all on equal footing, and everyone had an equal voice. It was pretty amazing. We did have one person that didn't believe in it and folded his arms and huffed at the whole thing."
"The moon man?" I asked.
Smiling, Howard said, "Oh, yeah." He said, "Doing a value stream mapping at this company made about as much sense as studying cyberbullying among the Amish."
It's a slow-burn joke, isn't it? A way-homer.
But even by the end, he begrudgingly admitted, "At least we have a plan now."
So we even won him over, and we were able to show that everyone had a place in the new value stream, even after all of the automation was put in place.
Now, the team decided to track the VSM on a big board in the middle of the dev bay. We put up a Kanban board under it so that we could show work moving through the system, and we could also gather statistics to feed Little's Law.
Howard continued, saying, "The information flows on the VSM showed us where we could sit teams together, reduce handoffs, and even reduce friction. We started this already and it's going great. I have to be honest, I didn't think any of this DevOps mumbo jumbo was going to work, but I'm starting to think this insanity may have a chance."
Howard put his hands behind his head and rocked backwards in his chair and said, "You know what? Dave has completely changed all of my goals for the end of the year, and I was starting to panic a little bit. But these DevOps tactics are actually helping me achieve them. I only have one problem remaining: that other middle managers, they're not following the same methodology."
I said, "Let's work with Dave to help create objectives and incentives for the other middle managers. You can evangelize these approaches within your own organization. Be creative and open to helping your peers, and lead the way to increasing interaction across levels.
"There are lots of opportunities within your organization. There's meetups, even outside in your community, book clubs, conferences, webinars, podcasts. Most DevOps communities are welcoming and eager to help, just like all of you here today. Encourage other managers to join in, too. Share the success that you've already achieved and start to build that internal community."
Now, I wish I could say that everyone believed in Howard and these ideas, and that they all made it through to the end. Sometimes a job starts out great, but changes over time, or maybe the person does.
Remember that blind date? Maybe after all that hair flossing, it actually ended up okay. In fact, maybe you hit it off, only now much, much later, you've been struggling. As employer and employee, you've both put in the effort and tried your hardest, but you have to say those five words everyone dreads to hear.
"Hey, we need to talk. It's not you, it's not me. Well, it's mostly you. I guess what I'm trying to say is that we've been drifting apart for a while now, and I think it's time you saw other companies."
It was a joke, of course, but in reality, ending a bad relationship, even a professional one, or maybe especially a professional one, is better than letting it drag on. The toughest conversations are usually the most important ones to have, and doing so openly and honestly is part of being a strong leader.
I wish I could say, too, that every middle manager had the support of a CTO like Dave, or staff that were eager and interested in change. Howard isn't real, of course. Neither is Dave. Heck, even these glasses aren't real, right? They're all made up of people that I've known in my career and my life, including myself. Well, not the glasses. They were made in China.
I've spent the last few years focused on how middle managers can drive DevOps change, and in some cases, I've discovered that they become the forgotten few, the ones that get left out of the transformation. Value stream mapping, culture sessions, goal alignment, and others mentioned in this little tale are tactical approaches that have worked so well for us in our DevOps journey.
Now, sometimes middle managers just don't want to change. They simply want to stay hidden and use WhatsUp Gold to monitor their IBM 704 mainframes. No offense. Especially to you, who's just on. That joke was already written. There was no way I could take it out.
I love you guys, seriously.
They don't want to make things better because if they did, it would take away their main reason for complaining. I call these people engineering Eeyores. Maybe they can't let go of the past, the that's-just-how-we've-always-done-it-here mentality. They have that Eeyore mindset of, even if I tried, it would probably fail anyway.
But why do they have that mindset? I found most often these individuals started out wanting to really make a difference, but the culture in their organization was so caustic, it wore them down over time. Simply swapping out this middle manager would only serve to ruin yet another great potential leader.
Middle managers serve an important role between senior leadership and engineers. But like everyone else going through transformational change with Lean, DevOps, and Agile, the middle manager must evolve with the ecosystem. Nothing is static in technology for long, including leadership.
It really does take a village to raise a middle manager to a middle leader, the type of culture that can grow the managers into the leaders of tomorrow. Now, we need the Daves of the world to step up and help create the DevOps culture from the top. We need the individual contributors and other managers to open their minds and support each other. We need everyone supporting the transformation in their organization, even if they don't always agree with it.
Now, not everyone gets a mentor like Howard did. If you want these types of changes, but you don't have a mentor, look around you today. There are many people that are willing to help. If you have these types of skills already, please share them as a mentor. Remember, you're the village. You're the village people, I guess is the word. You get to choose your costume.
After 25 years of working with engineers and leaders, I have had my best thoughts about them confirmed as true. And if they are true here among the most cynical of cynics, they must be true most everywhere.
We all have our own ways of doing things. Some of us, as hard as it may be to fathom, and you can count me among them, truly do want others to succeed. Our worlds are changing, and we must change with them. High-performing teams and companies are composed of individuals with commitment to working closely and openly together, regardless of title or station. DevOps grants us the space where we can live apart in our organizations and still have space to grow together.
I hope you've had a great conference. I know I have. Please feel free to hit me up with questions or comments or concerns. Please enjoy the rest of your time. Thank you so much.