Log in to watch

Log in or create a free account to watch this video.

Log in
San Francisco 2017
Share
Download slides

Bringing DevOps to Product

We join Howard from last year as he attempts to break down yet more silos. This time his focus is further up the value stream, and includes the business. After discovering an increasing divide between the product management team and development team, Howard seeks to get to the root of the problem. After digging in, he finds the situation to be worse than he thought. A lack of trust between the teams has made collaboration an insurance policy, rather than a competitive advantage. Communication is sparse and often through suboptimal means, resulting in miscommunication when it occurs at all. Outright anger has caused meetings to be held in secret, cutting out key players. Silos have formed and what’s worse, there are guards at the doors. Quality is suffering and the company is now shipping software that no one wants to buy.


Howard will showcase how we have been taking DevOps principles and applying them to the point where the business meets the engineer. With the future of the company at stake, Howard compels the Product Owner and Scrum Master to find solutions that draw the team together. Moving away from individual judgement to recognition of the strengths in their differences, Howard shows that DevOps can solve cultural concerns for more than just Dev and Ops. It can transcend engineering and include the business to create an enjoyable and productive workplace.

Chapters

Full transcript

The complete talk, organized by section.

Pauly Comtois

All right. So welcome, everyone. My name is Pauly Comtois, and I was hired three years ago at Hearst Business Media to support and grow a DevOps movement across three unique market verticals and 12 distinct business units.

And for the last year, we've been focusing on finding the largest barrier to flow in each of those businesses. And after 15 value stream mapping workshops, we kind of nailed it.

But interestingly enough, it was really more between product and dev than it was between dev and ops. And I thought the findings were pretty interesting, so I wanted to share them with you.

And what follows is a story, or a fable, that I wrote that I'm going to read to you today that includes the efforts of product owners and developers within those businesses to adopt DevOps to drive real business value in their organizations. So I hope you enjoy.

It begins simply enough. It begins as many things begin, not with a sudden epiphany or a moment of clarity, but with a long, slow buildup, ending with an engineer throwing his hands up in frustration and feeling like giving up.

Now, this in turn leads to an idea: how do we leverage DevOps to make the entire organization better?

Now, some of you may remember Howard from last year, our muscle-rippling and dashingly handsome middle manager, struggling against the flow of change. He eventually aligned with his organization, and now he fights for DevOps, justice, and Gene Kim's Three Ways.

Now, for those of you that don't remember Howard, he's almost as tall as he is wide. In fact, if you turn him on his side, he's still the same height. He's built a little bit like Gimli the Dwarf from Lord of the Rings. And much like any good dwarf or middle manager, he's fiercely protective of his people and easily distracted by shiny objects like mithril or a new JavaScript framework.

Well-liked and respected as a leader, Howard returned to his small Seattle startup after a brief mandatory vacation. He'd been so stressed that Karen, his boss, told him he had to take some time off. Howard had ended up at the doctor, and the doctor told him, "Listen, you're going to have to start psychotherapy or meditation. Choose one."

He said, "You've got to be kidding. Surely there's a chemical option. This is Seattle."

Now, this explained why Howard's office smelled like patchouli, and he had taken to wearing crystals all over his body. Not that there's anything wrong with that, but it was during one of these mandatory meditations that Jack Burton decided to stop by his office.

Now, no one called Jack by his real name. They just called him Snark. And he was called Snark because he'd spent essentially 90% of his adult life saying the opposite of what he really meant.

At 5'4", he was small in stature but grand in presence. He was incredibly smart. He loved being a scrum master, but he was the type of person that felt that if he wasn't arguing, he wasn't contributing. His witty comments always held a vital truth if you could just get past the sarcasm and cynicism dripping out of every word.

Slumping into a chair, he said, "I don't know, boss. I can't continue to carry this team. I'm an engineer. I don't have any upper body strength."

Now, Howard...

That's actually me, right?

Howard had been expecting this conversation. You see, Snark was the scrum master of his best team, but he was always at war with their product owner. The PO was a guy named Joel Nichols, but everyone just called Joel North Georgia.

Now, North Georgia loved the product. He loved his job, and by all accounts he was smart, easy to talk to, and incredibly nice. Now, the opposite of Snark in stature, he was tall and lanky. He towered over Snark in conversations, but they just never seemed to see eye to eye.

"Come on, Snark. Is it really that bad?" Howard asked.

"He keeps making commitments for dev without asking. He changes things at the last second without warning," Snark said. "Listen, as a rule, I don't trust anyone that isn't at least 94% beard and 6% grease stains. Besides, when we went to the customer site, he clapped when the plane landed. Then he turned to me and went hashtag YOLO out loud in front of other humans. I would've thought that was against his alien prime directive or something. He's a former customer that doesn't know anything about technology. He doesn't even have a real job. It's like he's in sales. He still wears a Members Only jacket."

Now, this wasn't unusual for Snark. Several times a week, he'd deliver a rant. They run about 75 minutes, including a bathroom break, about how someone, typically not Snark himself, was a complete idiot and causing all the problems. To that point, this one was blessedly short.

Howard held up his hands and said, "Listen, you had me at clapping when the plane lands, but nothing you said, other than maybe that YOLO thing, makes him a bad person. All right, he doesn't understand technology. That's true. But he understands the product and how our customers are using it way better than we do. Let me talk to him, but I'm going to need you to be a team player on this."

"Okay," Snark said as he rose from the chair. "But for the record, I'm the best team player on this damn team."

Now, Howard knew there were two sides to every story, so he set out to find North Georgia, walking past a large table of developers chatting loudly to where North Georgia sat alone eating his lunch.

Howard said, "Hey, North Georgia, you mind if I join you?"

North Georgia kicked out a chair and said, "Sure thing. You eat yet?"

Howard smiled and said, "No, not really hungry. What are you eating?"

Looking at his plate, North Georgia said, "Well, it's a pureed nut spread with a grape relish reduction paired with a beautiful brioche bun."

"Peanut butter and jelly?"

"Yeah, man. See, the wife and I, we've been watching that there Food Network every night, and the only problem I got with it is there ain't no recipes for squirrel."

Now, Howard folded his arms behind his back and said, "Yeah. Anyway, seems as if there's some problems with the team, and I'd like to better understand why that is. What are your thoughts?"

Now, North Georgia looked a little taken aback. He said, "You want to know what I think?"

I said, "Well, yeah. Is there any way we can fix that? All right, maybe. So it seems the team is having some growing pains, and I'd like to better understand why that is. What are your thoughts?"

Well, North Georgia thought for a moment and said, "Well, I wouldn't say it's blowed up, but pert near. It has gotten all cattywampus. Snark uninvited me to the standups, so I just started sending in feature requests through the interwebs. Backlog's taller than a Georgia pine, but I keep her in order."

Now, apparently, the relationship between product and dev had become the Windows Vista of relationships.

"Well, I think you should definitely be at the standup. Your input is critical to the team's success," Howard said.

"No, that dog won't hunt, boss. We tried it once, and ain't no one ever got nothing from me being at that there standup. You see, my goals are different from the developers'. As long as they keep doing what they do well, or good, I just need them writing code, okay? If I need more features, it ain't hard. It's just more typing."

Now, clearly, Howard was surprised. Apparently, neither Snark nor North Georgia really understood the value the other brought to the team.

Howard said, "Look, you and Snark both have important roles. They're just different roles, that's all. Neither of you can be successful if you can't communicate and work together as a team."

Howard sighed, saying, "You and Snark both seem to think the other person's job is, well, easy. We're all working towards the same goal, or at least we should be, but somehow, collectively, we've lost sight of it. Will you come back to the standup, see if we can figure this thing out?"

"I reckon."

"Great. All right, I'll see you in the morning."

Now, when Howard turned the corner to join the standup the next morning, what he saw made his heart sink: Snark and North Georgia standing toe-to-toe, arguing.

Snark saying sarcastically, "Listen, I believe in the standup meeting. I do. No, seriously, I do. I've been in solid standups, ones that start with a clear structure and purpose, and move through the entire team, and finish with a sense of profound accomplishment. This ain't one of those standups."

"I agree," Howard said. "The standup should be a representation of how we communicate as a team. We should be sharing things like information, tools, even outcomes. But to do that, we have to start by understanding what we each bring to the table."

"Well, we create the magic," Snark said, and he hooked a thumb at North Georgia. "It's hard to create magic with old Hufflepuff here."

North Georgia said, "Really? Doesn't seem that hard. How's writing code here so different?"

Now, Snark gave a smile bordering on a sneer and said, "Okay, well, we create software here the old-fashioned way. Tiny, incremental improvements to an antique, artisanal code base with fundamental flaws that no one's brave enough to touch, with randomly scheduled manual releases onto a production infrastructure that hasn't been updated since 2005. But it's better than the past when we would dive right in without any preparation or research, and work a feature for weeks to months, only to find out it's 100 times more complex than we thought. Oh, and someone else has already done it, and they did it better than we could have. Our code is deteriorating faster than we can lower our standards. But at least I'm not bitter about it."

North Georgia said, "Oh, man. Y'all ought do something about that."

And Snark went, "Do you think? Listen, we never know what's hitting us next other than it's guaranteed to be yet another feature request. We're sitting on a mound of technical debt, and it's growing like Tribbles."

"What's a Tribble?" North Georgia asked, looking confused.

"Never mind," Snark said, looking disgusted.

Now, to North Georgia's credit, he tried to defuse the situation, saying, "Listen, I used to be a developer, too. I understand y'all's life. I ran my own WordPress site."

"Mm-hmm."

"I even modified the PHP. Y'all ever thought about using open sores?"

Now, Snark's eyes gleamed as he said, "You always do that. You're offering solutions when no one's even asked. And did you just say open sores? Listen, I'm sure your rusty PHP and FrontPage 97 skills are crushing it back home. Why don't you leave these technical problems to those of us with tape on our glasses, and you can mosey on back to your Gateway desktop."

Now, Howard saw that this was spiraling out of control, saying, "Listen, if we can't communicate as a team, we're never going to really succeed as a team. This isn't the best place to work that out. Let's just try to get through the rest of standup, and then after, we'll go get some coffee, and I want to introduce you to a friend of mine. She really helped me out last year. I think she could really help us out this year."

Now, after the shootout at the standup corral, Howard decided to reach out to his friend and mentor, Emma Watson. No relation.

Emma agreed to meet, and the walk was quiet, giving Howard a chance to think. Could they use DevOps methods to break down the silos between the business and the engineers? Just making them go to a 15-minute daily standup clearly wasn't going to be enough.

Now, after introductions, they sat at the table with their coffee, and Howard explained the situation to Emma. And after a moment, she began by saying, "Listen, I'm coming into this situation new, but it sounds as if you guys have some stereotypes, maybe even some misunderstandings, that are holding you back from gelling as a team. That sound about right?"

There were reluctant nods around the table.

Emma continued, saying, "Listen, we're going to want to destroy those stereotypes, and you can't do that with talk alone or a daily 15-minute meeting. What we want to do is, we want to become the contradiction of the stereotype. We want to prove it wrong through our action, not just our words. We could try a DevOps approach to this."

Now, Snark rolled his eyes at that, but Emma continued. She said, "Last year, when dev and ops habitually put DevOps into practice, it organized their lives. But when they faithfully put it into practice, it changed their lives. In other words, when they believed in themselves and the change, they saw the change in themselves."

Now, Snark nodded at that and almost seemed to agree, saying, "Wow. Deep."

North Georgia said, "Listen, that may work for technical teams, but I ain't operations. Hell, I ain't even technical."

"That's true," Snark said. "He couldn't even tie his shoes if they didn't have that little Velcro strap on them."

She said, "Okay, that's enough, Snark. These comments aren't really helping." Although, she did peek around the table to look at North Georgia's shoes, just in case. And they were slip-ons.

She said, "No more witty comments. They're not really beneficial to what we're trying to accomplish here. And besides, I don't think you have to be dev or ops or even technical for the principles and patterns of DevOps to work. In fact, pushing these beliefs outward from dev and ops can drive real business value for everyone in the organization."

Snark sighed, defeated, and said, "All right. Where or how do we start?"

She said, "Well, we can learn a lot from each other if we can just find a way to communicate more openly and effectively. We're not looking for a full stack developer here. We're looking for a full stack DevOps life cycle. We want to change the way we feed, communicate, and process information through our value streams. We need to hold ourselves and others accountable, but still give everyone some room to take some risks. You don't need to know how to do the other person's job, but you should understand it and how it integrates with your own.

"Why don't we start by working on a definition, a team definition of a mutual purpose that we can all agree upon?"

And they started with a common purpose of working together to build engaging software that solved customer challenges. Sounded nice, but they recognized that to have this common business goal, they'd have to refocus on building the team around collaboration, communication, and even a little bit of tooling.

Snark said, "Listen, I'm all for trust falls as much as Captain Patchouli over here, but this isn't going to work, okay? Product's only goal is selling as many features as inhumanly possible, leaving us no time to reduce risk or add stability by lowering technical debt. We're building a skyscraper on a foundation of matchsticks here, people."

North Georgia raised his hands as if to surrender and said, "It ain't my fault. The salespeople keep cutting unachievable deals that land them their bonus."

"You could just say no," Snark said. "I mean, we're supposed to be on the same side."

Emma stepped in, saying, "Well, we should stop focusing on blame and maybe start taking accountability for the part we each play in creating a safe enough environment to say no. Successful common purpose requires trust, and trust requires safety."

Snark said, "Well, I can safely trust that sales is going to sell something we don't even make yet."

Emma said, "Well, perhaps. But both of you blindly defending yourselves and blaming others is like arguing over who has the better participation trophy. Both of you are afraid of saying no. No can be the hardest thing you do in your career."

Snark said, "Not if you're in our IT department. Other than grunts and clicks, no's the only thing they know how to say."

Sighing, she said, "Well, you don't have to say no exactly, okay? We want to find a way to communicate our limits and concerns in a manner that still builds trust. How can our customers trust us to deliver if we can't trust each other to do the same? When people spend their time protecting their own interests instead of focusing on the goals of the team, you end up sacrificing innovation and productivity."

Now, Snark and North Georgia agreed to begin having one-on-one meetings with Emma, as well as frequent trips out of the office together to work towards strengthening their understanding and even trust of one another.

You see, in the past, when North Georgia was on site with a customer, he'd often make decisions and even commitments on behalf of the team without any communication or collaboration. Now he sought to build trust by saying not just yes to everything, but instead, "Yeah, that seems all right. Let me verify with the team before I commit."

Now, calling Snark on the phone, he hadn't quite mastered texting yet, he set the deadline as the business, but only the Scrum team could set the commitment.

Snark said, "Well, we can commit to 80% of a high-quality product by that deadline, or we can commit to 100% of crap by that deadline."

North Georgia said, "Well, bless your heart. I'll get him to agree to the 80% by the deadline. We done worked it all up on the chalkboard."

"Whiteboard," Snark corrected. "Could you send me a picture of it?"

"I didn't bring a camera."

"Well, just use the camera on your phone."

Now, looking at his Jitterbug flip phone, he said, "You reckon?"

You see, collaborating in this way, North Georgia was able to set proper customer expectations and even team expectations early, instead of making excuses later.

The team defined accountability and responsibility. You're accountable for what you create, but you're responsible for each other.

Now, the following week, Emma met North Georgia at the coffee shop, and she said, "Remember last time we met and we talked about trust? Trust isn't a binary concept. It's nuanced. In fact, it's often situational. We want our efforts to benefit the whole rather than just a single part, and for that, we really need trust."

So Emma pulled a small stack of papers out of her bag and pushed them across the table. She said, "These are writings by a brilliant man named Russ Ackoff on something called systems thinking. We're going to start focusing on systems thinking combined with DevOps to really break down these silos between the business and the engineers. You see, you've been focusing on improving your independent part without focusing on how it impacts the whole."

Now, Ackoff speaks to the essential or defining properties of any system as the properties of the whole which none of its individual parts possess. For example, your hand can't write, but you as a system can. Cut your hand off, lay it on the table, it doesn't do much on its own.

She said, "The essential property of your software value stream is that it discovers, creates, tests, and operates the correct software for its customers. No single part of that value stream can typically do that entire process on its own. Therefore, when a system is taken apart, it loses its essential property. That is the negative consequences that we experience when we inject silos into our value streams."

North Georgia nodded in agreement and said, "So we still work on the parts, but we focus on how it improves the whole."

She said, "Exactly. We do that by building an environment that fosters active and open feedback. We want loyal opposition. Basically, a culture where people can share a strong position weakly held. We need to communicate more often and openly so that the developers aren't always so surprised by what's coming out of product.

"Why don't we start with something that's working further down the value stream?"

He said, "Yeah, I was fixing to ask you about them task boards the devs use. Y'all reckon we could do that for product?"

She said, "I don't see why not. Seems like a good place to start. How about we try that, and in a couple of weeks we'll swing back and see how it's going."

Now, North Georgia copied the format of the dev agile task boards and began doing a standup with the product team. It was open to anyone that wanted to join, and it made their pipeline highly visible.

Now, including the business priorities in their weighted shortest job first model meant that the business needs no longer trumped the backlog prioritization at the last second. This created the much-needed consistency in feeding that pipeline, further building the trust needed to increase the value of the system as a whole.

Now, this reduction in lead time meant that they were getting product to market 30% faster. The new collaboration meant that those products were ones the customers actually asked for.

He said, "I want to get off the porch and take this one step further. I want to provide the why of the pipeline. Why are we building the software that we're building?"

She said, "That's a good idea. Providing the critical component of context means that you're elevating the process. You're moving it from writing code to creating solutions. What else have you been working on?"

He said, "Well, the biggest seed in the devs' craw has been tech debt. No one in the business seems to think it mattered, and I think that's mostly because the business folks didn't understand it, and no one was really held accountable for it.

"Now, Howard helped me convince the management that we should focus about 20% of our time on tech debt. That includes time spent on identifying the technical debt that impacts the business and what amount of risk we can accept. We made the tech debt a red card on both the product and the dev task boards. That way, we could track it in a transparent and clear way. That 20% is tied to product's review and their bonus.

"Now, refocusing our metrics to be more customer-driven allowed us to drive the right features. We used that time we got back to reduce tech debt."

Now, once North Georgia had support for aligned incentives from his leadership, they began to publicly track the efficacy of every feature, holding product accountable, retiring those features that weren't in use any longer, and doubling down dev time on the most profitable.

Working together, product and dev built a pipeline and an API bridge between their applications. This enabled the automated flow of information and accelerated the new culture of collaboration and communication.

Now, Emma met Snark at the coffee shop the following week. She said, "How are things going?"

He said, "A little surreal, to be honest. North Georgia asked me to mentor him, but I got to tell you, he's mentoring me as much as I am him. Although half the time I can't understand him, and the other half I don't think he can understand me. He's showing me what it's like to be a product owner, and I think I finally get why we need him on the team. I mean, he can articulate customer needs better than anyone I've ever met, and he can get them to do just about anything."

Emma replied, "That's great. Listen, by understanding each other's value and just creating some very simple channels to share that value, you're starting to embrace DevOps. Now, why don't we talk about how you can communicate with the product team to build more trust?"

Snark said, "What are you talking about?"

She said, "Well, quick, name three things that are high risk to you at work."

Without a moment's hesitation, Snark said, "A developer with a login to the Oracle database, terrifying. A sysadmin refactoring core code, offensive. And a product owner with a brilliant technical idea. Really?"

She said, "Listen, we want an inclusive culture, but sometimes your humor, while funny, can be kind of divisive. I mean, we want to break down silos, not reinforce them with caustic language and behavior. DevOps isn't about requiring anyone to be a different person. It's really more about modifying how we interact and behave with one another."

She thought for a moment and said, "A clever and thoughtless person can be one of the most dangerous opponents to DevOps. Let's start by moving your communication patterns away from hyperbole. I'd like you to start leading and contributing conversations by asking questions. Try to resist stating opinions or even facts."

"Hey, I ask questions," he said.

She said, "Yeah, but your questions are all rhetorical. Lastly, and most important, stop the personal attacks, the personal snark. It tends to build walls rather than break them down. Think of it a little bit like Nickelback, right? A little bit goes a long way, and no one wanted to hear it to begin with."

Actually, as a Canadian, I should apologize for these gentlemen. And that one, too. So sorry. Or, oh, geez, sorry.

So Emma said, "Listen, we're not going to tolerate brilliant jerks any longer. Remember, reacting is automatic, but thinking is not. If you have to make a joke at someone's expense, try making it at your own. I helped Howard make a similar transition last year. Are you willing to give it a try?"

Snark said, "Ugh. I don't know. I kind of feel like it's my Kobayashi Maru. But okay, I'll give it a shot."

Now, Snark's efforts at mentoring North Georgia is a story unto itself. What started out as Snark seeing every interaction with product management as a conversational cul-de-sac became a relationship built on mutual respect and common purpose.

North Georgia took Snark to product management meetings, customer sites. He led classes and provided one-on-one mentorship to teach engineers the proper way to speak to the business.

And in turn, Snark took North Georgia to different standups and showed him how DevOps worked in their organization. He walked North Georgia through the high-level overview of the application, even the architecture.

They took every opportunity to cut each other down and instead built each other up. Heck, Snark even started saying y'all around the office.

Now, they may not ever choose to be each other's company on a weekend trip, but they did eventually build a solid working partnership through open and shared experiences.

Breaking down silos between product and dev wasn't accomplished with lone brilliant individual contributors. Instead, it was with those same individuals finding a way to trust in each other.

Now, solving the impedance mismatch between the business and the engineers required a focus on communication. In DevOps, we need engineers and business leaders to be able to openly communicate with one another, and in order to facilitate that dialogue, we focused on mentorship and training at all levels.

Now, this focus on raising people up got us to market faster with products that delighted customers and provided intrinsic reward to our own engineers.

Embracing DevOps principles outside of dev and ops, we removed cliques and exclusive groups by embedding product owners into the teams, both physically and through shared experiences, knowledge, tools, and goals.

Focusing on the business's cultural readiness for DevOps was key. That allowed us to bring them into alignment with engineering first, then we automated the tooling and process only where it made sense.

Focusing on the handoffs from product to dev, we reduced waste, idle time, and even confusion. Metrics were created to track not only technical value but business value as well, with a focus on customers, compliance, and regulatory needs. Metrics were shared openly and not just through weekly email updates or meetings.

Now, we've been able to adopt DevOps throughout our organizations at Hearst Business Media, not just dev and ops. Product management, content creation, and yes, even finance to some degree is adopting the practice. We focused our efforts on bringing the business into the DevOps family instead of just seeing the business as the crazy old uncle at the family reunion.

We did this by tying business goals directly to the release cycles and being transparent about both. True, the business and engineering do speak different languages, but they are two languages trying to tell the same story.

Now, we weren't able to wrap up every DevOps mystery in 30 minutes. Some we're still trying to peel the mask off of, like finding creative ways to tie business value to DevOps, such as competitiveness, profitability, and even new customer acquisition. Or addressing key individual contributors in engineering and leadership that still think that any innovation is just a fad.

It takes time to move to a sense of trust and safety, especially if the old ways have been in place for years. Getting the teams to hold each other accountable in a healthy way, and then using technology and lean process to enable that trust-based accountability was a big win for us.

In bringing these two teams together to form one team, we put the people first and the technology and process followed naturally. In DevOps, you're not competing against the other team, you're competing with the other team.

If you'd like specific help on starting this journey in your own organization, come find me and let's start it together.

Thank you to Gene and DOES17 for allowing me to continue to share Howard's ongoing journey of DevOps. Have a great conference, y'all.