Hello, my name is Marc Nemecek. I'm a senior director of IT infrastructure at CDK Global. When I named this presentation, I kind of meant the title somewhat literally, as in we are solving the Dev IT cultural divide with operational agile, not as in we have solved. This is not a reference work.

And I want this to be consumed by everyone, but really who I'm talking to primarily are people who are beginning their transformation or are about to begin their transformation, might have some self-doubt. Hopefully, you can learn some of the lessons from some of the stumbles that I have made and my team has made, and even if you don't, I promise I will continue to stumble and create more lessons for you to learn from down the road. So you have not heard of CDK Global, and the reason why is because before three weeks ago, that company name did not exist. But our company is 40 years old.

And I do not have my slide clicker. Thank you. I could give the whole thing on the first slide. I can do that.

Let me quickly bore you with my past. It's just a couple of lines. The first 20 years of my career was spent in QA. I was on the R&D side.

I spent 12 years at Microsoft and three years at Tripwire before I jumped over to CDK Global. The move to CDK was significant for me, not just in terms of a job change, but because it was me leaving R&D and jumping to the IT side. Now again, CDK, you have not heard of. However, you have probably heard of who we were before.

We were a division of ADP. The primary division at ADP is called Employer Services. Those are the guys that have the logo on your paychecks. That's not us.

We were a smaller division called Dealer Services, and three weeks ago, we spun off and became our own company. But we kept our business, we kept our infrastructure, and our clientele. We have tens of thousands of customers. We are global in over 100 countries.

And we bring in around $2 billion of revenue in per year. Inside CDK, we have about 9,000 employees, of which 1,400 are R&D, and about 200 are my organization. That's the Global Hosting Services organization, and Global Hosting is responsible for the infrastructure upon which all of our cloud services and products rest. That's tens of thousands of virtual machines, hundreds of pre-production environments and production environments for over 60 product streams of varying shapes and sizes.

The types of products that we have are distinct. Some are web-tier solutions with database backends, some are legacy machine systems on-site at dealerships. Some are digital marketing solutions. And the technology stacks that we have are as varied as it gets.

The reason why is because we have teams that are legacy, that are 20 or 30 years old or beyond. We have teams that have been acquired or we've merged with. We have even startups internally. And so if you can think of a technology stack name, I can guarantee you we've got it somewhere inside our infrastructure.

And so my intent originally, was to put up a slide that had logos of all the different technology stacks that we support internally. But the conference refused to put a third screen up to support enough real estate for me to put those up. So instead, what I'm going to show you is something that I find equivalent, and that is magnified grains of sand. They are as unique and beautiful as all of our product and technology stacks, and they are also just as distinct.

The kind of problems that we run into are the ones you would expect in scaled enterprise IT. Scaled enterprise IT tends to grow and change to the point where it has extracted the maximum possible work out of the smallest possible teams. By definition, scaled enterprise IT teams are underwater. And so this leads to some cultural issues.

A cultural circle of life. You may see people who are at 100% utilization and high context switching. That causes them to, in defense and self-defense, construct high fences around their areas of responsibility, which in turn makes them think about pushing back on customers with phrases such as, "Not in my job description." This leads to the dreaded dropped baton, which then leads to dragons, which just leads to misery, and then the cycle of life continues over and over and over again. I came on board at CDK in July of 2013, and the primary mandate for me was, "Help us with the culture." In fact, the word DevOps was in my title originally.

And so I took time to kind of, in addition to my day job, study what was going on and try to decide what can I do that might be opportunistically... So I can change opportunistically. And of the things you see on that screen, the area I really wanted to zero in on was the dropped baton. The others kind of seemed like much larger scale efforts.

But the dropped baton in particular was frustrating to me because it was of a very distinct nature, and that was always between Dev and IT. This is not news to any of you. This is how it works. Developers will drop batons, communication batons from dev to dev.

IT teams will drop batons from IT teams to IT teams. Dev to IT, it's like they all get dropped.The sheer volume of it is exponential by comparison. That's enormously frustrating. It's not too surprising, though, for some reason.

So I started spending time drawing pictures like this on my whiteboard and other people's whiteboard, and whoever would have me in their office to talk about it. And the reason why batons get dropped is because people don't communicate. They don't communicate because they don't feel accountable to one another. They don't feel accountable to one another because they don't work well together or they don't see each other every single day.

We have methodologies that solve these issues, but we primarily deploy them in development because that's the shape of how Agile works best. I've hit this issue in previous jobs and in CDK of bringing Agile into IT. And in every scenario, every time I tried to simulate it or figure out how to deploy it or propose it, it hit an enormous wall of some sort. It just didn't work.

And in every scenario, I had to walk away and say, "Forget that. I'm going back to my day job. There's no way we're going to put Agile into IT. It's just too big." And so initially, I walked away from it, and then eventually it became my job, of course.

We have a product. We have many products. We had one that is over a decade old, and it's got a database that is being increasingly crushed under an increasingly large amount of technical debt. The beautiful thing about databases and technical debt is that you don't have to misbehave to make it worse.

It self-feeds because data just gets bigger. And so my boss calls me one day and he says, "Mark, as you know, this is getting worse and worse. It's getting executive attention. I think it's going to a war room." Just a show of hands, does anyone here, not this, but does anyone here know what a war room is in IT?

About 70% of you are liars. War room is when somebody fires a panic flare and 20 engineers and three executives go into a conference room, and then for six hours, they watch two guys fix everything. And it makes me angry. I would walk into those rooms and my vision would get replaced with enormous piles of cash on fire.

My boss knows this, so he calls me up in January of this year and he says, "This is getting worse, and I think it's headed for a war room. And Mark, you did not make this scenario. You're not responsible for making this happen. But what can we do in IT?

What can we do in hosting?" Something rather unfortunate happened at that moment. I think he spurred some kind of other presence inside my mind that controls me to answer and say, "I know what to do." "I will find a small group of people who are accountable and responsible in IT and dev, and I will pull them into a standup and we will solve this." And he said, "Oh, you mean like what they do in development?" And I said, "No, no. Something else, but I'll let you know how it goes." And he said, "Great." And so I get off the phone and I realize that what I have just done is sign myself up and my team up to do something that I don't believe in. More specifically, that I have no idea how to do.

So I'm back to this picture because I know that if I try to stick an IT standup together, it's doomed for failure. And besides, by the way, I hate the term dev standup versus IT standup. It makes no sense to me whatsoever. The distinction doesn't make sense to me.

And so I'm thinking about this, and again, about the fact that dev to IT batons keep getting dropped, and why does this get dropped? And I've decided... By the way, this is the part of the presentation where you're going to begin to think that I'm a little insane. I decided that I'm going to unpack this thing all the way, because if I put an IT standup together, I know it will fail.

And so I start asking philosophical, existential questions and going straight down the rabbit hole on who is dev? Who are the IT engineers? Who are these people? Who are developers?

Why do they get up in the morning? Oh, they write code and they check it in. Wrong. They don't, no.

That's not why they get up in the morning. That's what they do. That's not why they get up in the morning. Developers take a bag of nothing and they pour ones and zeros into it, and they shove it into a box, and out comes Twitter for your iPhone and amazon.com and online banking and Angry Birds.

They literally make magic out of electrons. Who wouldn't get up for that? I wanted to distill it into a single word, one word, what do developers do? They create.

They make things. That's awesome. I would get up for that. What do IT people do?

Who are these people? No one in IT gets up on this planet and thinks, "Mm, time to keep the servers up." They're caretakers. They're like that annoying neighbor that has a Camaro that they wash every single day. The thing's not even dirty, and they wash the damn thing every single day.

And they check the air in the tires, and they check the oil, and they put that stuff on the seat, that stuff like MotorTrend's going to show up or something like that. Like they love this thing. Like they want it to be ready. This is what they do.

Distill it to one word. What do they do? What does IT do? This is a harder word for me to get to.They operationalize.

It might not be the best word for it, it's just the one I landed on. So now, this is starting to come together for me, and this is real. I did do this. This makes more sense.

A dev standup doesn't make sense to me versus IT standup. But people that are focused on creation versus people that are focused on operationalization, this is starting to make a lot of sense to me because these people are from different planets, they speak different languages, they do different things. They care about different things. They get up and go to work for the same or different reasons.

I'm starting to see why it is that these batons are just falling left and right all over the place. And indeed, I'm starting to see how I could actually construct a standup that doesn't suck. So this is what we did. I grabbed a couple of people from the creation standup.

Not all of them, I left all the developers alone. I grabbed a product owner and a dev manager, and then I grabbed some people from my team. I grabbed a NOC monitor engineer, a platform engineer, a DBA, and an app tech individual, and I pulled them into a room and I said, "Here's what we're going to do. We're going to write some user stories, and we're going to write them properly.

We're going to say, 'As blank, I would like to blank so that blank.' We're going to file them into epics. We're going to sort them in a backlog and frame them into sprints, and we're going to sprint, sprint, sprint, until this database is no longer a suspect in any performance problems in this space. That's what we're going to do." For IT engineers to hear this, they get stoked, like, "Oh, Mark, this is great. This is great.

What's a user story?" Whoops. So I said, "All right, we're going to do all that, and I'll teach you Scrum while we go." What could go wrong, right? And so we did this. We did this operationalization standup.

Don't try to say that too fast the first time because you're going to hurt yourself. That's a tough word to crank out. You have to practice that for months the way I did. We started, and know that in past lives, I was an agile purist.

I was like an agile hipster. I would judge people based on how agile they were or how well they were in their standups. It was stupid. And if you were one of those and you saw this standup, you would throw up in your mouth a little bit, I think.

Right? It might have resembled it on the outside, but then you look in, and it's just ugly. We didn't use story points, we didn't check velocity, we didn't do retrospectives, we barely acknowledged Scrum boundaries. All we did are the things that an operations team would need to do to improve communication, visibility, accountability, right?

We tracked our stories. We had a centralized, visible backlog of work. We had peer-based accountability. We met every single day.

And it was a stumble. Make no mistake, we stumbled through this thing, and not pretty. Not pretty, but it worked. It was ugly, and it was just so much better than the way IT and dev organically talk and work well together.

And work together, I shouldn't say work well. So this standup was a reference work, and by that, I mean we've done a dozen of these since. And that first one, I started in February of 2014. This is all new stuff.

In 2014, we started that one, and in a week, we had a groomed backlog. This is a crew that never seen agile before. So in four weeks, mission accomplished. There was no more suspicion on that database.

To this day, it has never again appeared in conversation around, "Hey, what's up with this product?" It's always something else. In eight weeks, we disbanded. We're like, "Oh, we got all the high priority things done, and we got other things to do. This isn't worth looking at anymore." We disbanded.

It was exactly the way this should go. I just had no idea about that at the time. This was all brand-new territory for all of us. So this thing became...

Here's the problem. Not a problem. Here's the thing with introducing a new concept in a highly visible space where the CIO and CT are watching, and have it be a stupidly wild success. It becomes a go-to structure.

You start to shove everything into it. I had to unfortunately cut a really painful story about a failure we had with that by trying to stick a massive business unit multi-product migration into a single standup. It was brutal. Ask me later about it.

But this thing became a go-to construct, and by and large, across the board, it still continues today and has been greatly successful. We did notice one thing, by the way, before I go on. We noticed as we spun up more of these that we had some common epics between them. This shouldn't be a surprise, but I didn't know to expect it because, again, all new ground.

We would see documentation, monitoring, runbooks, backup, HA, done is done criteria, infrastructure delivery. Eventually, we institutionalized that, and every time we would start up a new standup, we'd throw those epics over and the baseline stories to begin with and say, "There you go. There's your jump start." We actually commoditized how we operationalized our products. And just to make sure you know, this good news, these good things that came out of it, touch about 5% of our entire product line.

Like I said, we are very early in our DevOps transformation. I don't know how to scale itSo what happened? Across the board, especially with one partner in particular, we saw deployment frequency go up, success rate go up, Dev IT relationships improve dramatically. They started talking to each other.

That's good. That's an improvement. But not only that, they started talking to each other before they talked to me. In fact, entire conversations and conflicts and sprints would go by and I'd never hear from them.

Like I said, it was just good, actually. Commoditization of application configuration, that's hard to say. The focus on documentation and change control process meant that our mean time to repair dropped, and a year ago, you might've seen a C-level executive in a war room, my favorite thing, at CDK. I'm not joking when I say I can't remember the last time that I had a C-level executive in a war room.

My context with C-level executives and outages right now is once per month, I'll write a report and present it to them directly and say, "Just so you know, this is the stuff that happened. You didn't know it before, but now you do." And it's a yawner. So if I could steal that DeLorean from the party last night, and go back in time to when I first started my DevOps arc, and that predates CDK, I'm just going to share with you a couple of lessons I would impart to my younger self. And in my attempt to keep this interesting, I'm presenting these lessons in increasing order of importance.

So, big finish. You are not going to have large victories in DevOps. It does happen. Right?

You look at Gwynne Groer. He transformed HP. That's amazing. That's awesome.

It's just that volume-wise, that's a very, very small percentage of the positive tick marks in DevOps. Don't expect that. You'll have small victories, and the problem with that is that you will then forget those things because they were small. It is very, very important if you are a leader in this space that you identify these things, capture them, honestly, journal them, and keep them, because there will be days that rain and you are going to need this.

It is important if you are a change agent that you keep your own morale up, especially if you're the kind of person who might be like the top of a chain of change agents in your organization, and that's the best hope for moving forward. You're going to need that. You will not throw for 200 yards and win the DevOps Super Bowl and get a parade. You just shouldn't expect that.

What does a little victory look like? You might solve 10 years of technical debt in eight weeks. That's a little victory. You might have an employee who talks a lot about, "I think I'm going to leave the company," but then they don't, and they don't talk about it either anymore.

In some cases, it could just be like a thank you from one of your employees or from somebody you partner with in your organization, and they've come to thank you because you just made their life easier. The most poignant one that I had, the most poignant little victory I had, the one I keep written down on a piece of paper in my pocket... No, I don't actually do that. Was the night that I had to call one of my directors of development, one of my partners in the dev space, and let's be real frank about this.

This guy and I, and his team and mine, we'd had some contention, and they'd had some executive attention, and you don't want executive attention. And I had to call him to tell him that my team had rebooted his web tier in the middle of production hours because they were trying to reboot a pre-production box, and it just biffed. Whoops. Yeah, it's pretty awesome.

So I call him up and we talk for a few minutes. I notice that his blood pressure hasn't gone through the roof, and that's strange. We talk. He checks some things.

I check some things. He comes back and says, "Mark, I think we're fine." I said, "Okay, look. When your QA people come online overseas late tonight, my team will hook up with them. I will get online with them, and we can begin to run a smoke test." And he said, "Mark, stop.

I said we're fine." I said, "Okay, and look, I'm not the kind of person that would punish someone for a mistake like this. That's not going to happen. But it is unacceptable that it's possible for us to reboot your production tier. We're going to have to think about a protocol for handling pre-production and put that in place." He said, "Mark, seriously." He cut me off.

"You're not hearing me. You are not hearing me. Stop. I know you're going to do this.

And when you do it, you don't have to tell me you did it, because I trust you." Trust you. That's the one I keep. If you are a leader in this space, part of your job description, whether you like it or not, is selling better culture. Don't insult your customer.

I've had experiences with some of my dev partner teams who I thought might not be on the proper curve for adoption of automated deployments. And the what I did not do is say things like, "Dude, do you even ALM? I was into continuous delivery before it was cool." Don't use big words and make people feel small. When you do so, all you're going to do, especially in an enterprise environment, is you're just going to alienate them.

Instead, catch their imagination, catch their attention. The incident I'm talking about, what I did is I used a quote that someone gave at DevOps Days Portland last year. He put it up on the screen, and unfortunately, I can't remember who this guy was. I cannot credit him, but he had this great single slide aboutWhat is the fastest possible journey for a single line of code change from commission to production?

And I just asked this guy that same question, just repeated the same question to him. And he's looking at me, and then he kind of begins to look out over my shoulder with a thousand-yard stare. And I'm thinking to myself, "He's fantasizing about writing code directly to production, isn't he?" And then he comes back and looks at me and says, "Uh, I don't know. But I like the way you think." Velocity.

We want to talk about it. I didn't insult him, I excited him. This slide is dedicated to Dominica DeGrandis, who by the way coached me on this presentation. You will manage exactly what you measure and not a thing more.

The precision of your metrics will define... Sorry. The quality of your metrics will define the precision of your management. And by that I mean, if you go and horse whip one of your IT engineers and say, "I want you to respond to P1s faster," and you don't have a recurring SLA meeting happening frequently, then you're just abusing your employees.

Say you don't have any metrics. Should you still start a meeting? Yeah. Start a meeting and put an empty grid on the wall and pull your leadership in and say, "Check it out, we have nothing." "Let's get something, and then next time it will suck less." That's not a joke.

This one's subtle but important to me, and that is enforcing roles, not boundaries. People tend to react to boundaries a lot. This happens a lot in my organization, and I try to coach people often on this. This is not to say let your employees get hijacked.

If that kind of thing happens and you have an engineer that objects to it, maybe you object to it, a boundary-focused employee will respond by saying, "You got the wrong person. Find someone else." A role-focused employee will likely dig in, get tactical, help them get through the business issue, come out the other side and start asking questions about, "Was this the best use of my time?" Because if you want me to go help them debug their code every day, game on, bro, let's go. I'm ready. But if not, let's figure out how to staff this properly.

It's subtle, but it's an important shift. If you're starting out in this space, and you're a leader, and you've got some people around you that are ready to execute, and you've got some plans, and you're excited, you're going to throw the doors open and walk out into the sunlight, and you're going to catch your foot on the curb, and you're going to smash your face right into the concrete, and then you're going to do it again, and again, and again. My history at CDK is replete with stumbles. I stumble now, I will continue to stumble, I promise.

But this is the nature of it. This is how things advance. It's how you learn. You're going to do it publicly.

Be ready to be wrong. Be ready to make mistakes. That's going to happen. This one is just about don't...

Originally, I was running seven operationalization stand-ups a day for a while. I don't hate myself, but I did want to make sure that things were running well before I cut them loose. Today, I don't run anything. Well, I don't run those stand-ups.

And that's because I have focused on spinning off leadership in other people. And that is, I don't mean to say spin off yourself, make clones of yourself or an echo chamber. I mean an action chamber. People who can align on your same intent, but not even necessarily your same methodology.

My boss and I do not always agree on methodology, but we are exactly aligned on intent. Lastly, this is one of the most impactful leadership quotes I've read recently. And this is just about making sure people understand why you're asking what you're asking. If you're going to hammer a team and say, "You should adopt the NoLeo by next Wednesday," you can say things like that, but make sure they understand what you're getting at.

Why are you doing this? And don't be afraid to regale them with the tales, right? Talk about the ocean symbolically to say, "Hey, in the land of continuous delivery, nobody fears change." You could beat up on your IT team again on P1 management or on workflow management. Don't.

Just tell them, "Look, if we were black belt ticket managers, everybody would trust the IT workflow. How cool would that be?" And so in this space, if you're a leader, you need to gather your leadership around you, set intent for them, set them in motion, stop telling them what to do. They will stumble into greatness, and they will drag you along with them. I have some ambitious things I think I could maybe help out with.

I have failed enough to sell DevOps, but I think I might be of use in a conversation about how to sell DevOps, or at least how not to sell it. I also think there's some very simple, quick tricks you can do to disinfect a blame-based culture. I have not dove deep into Idle and DevOps as far as if they can coexist. I use elements of Idle in my DevOps, but I haven't walked that path.

I need help with this. And not to say we're making any changes organizationally, but we are growing. How to evaluate in introducing tiers in my organization is something I would like to get more attention to and more help on. And just final thought, it has been an honor to stand amongst everyone here this entire week.

I think you are all leaders in the DevOps space, and I don't care what your job title is or who you work for. If you're here, you're at the forefront. And so I just want you to know that my door is open, always. If you're in Portland, Oregon, and want to get a beer, reach out.

Send electrons my way or for any other reason. Thank you very much.