Heather Mickman

So I'm Heather Mickman. I lead Target's API and integration team at Target, and I'm here this morning with Ross Clanton, an ops leader in our infrastructure services organization.

We're literally thrilled to be here today to talk about this journey that Target is on with DevOps.

I am going to tell the story of my dev team in the early days of our journey. Ross is then going to talk about the fantastic work that he's driving on the ops side, and then about our combined efforts for what we're doing to drive DevOps at Target across our enterprise.

But first, I'll tell you a little bit about who we are, what we do, and some of the complexities that we face.

Target's first store opened in 1962 in Roseville, Minnesota, and we now have more than 1,900 stores in two countries. We have headquarters locations across the world. We're located in Minneapolis, Minnesota, but also Bangalore, London, even Cairo. We have 40 distribution centers and are the second-largest importer in the U.S.

Over the last 50 years, Target has also added new businesses, including banking, pharmacy, and healthcare.

Over the years, I'm sure you can imagine that we've added a lot of technology to keep up with our growing business. I'm sure you've seen Target's in-store technologies, like our price checkers, cash registers, the handheld devices that team members will use to restock shelves, the new gift registry iPads that are rolling out, our guest Wi-Fi that we have so you can easily use the Cartwheel app on your mobile phone.

But there's a lot of technology behind the scenes as well, from the applications that run our distribution centers to ensure we're getting the right inventory to the right stores at the right time, our dot-com site, our mobile apps, the servers in our data centers, the phones in our guest contact center. It takes a lot of technology to run a modern retailing company.

Now, over the years, of course, complexity in our technology footprint has expanded. The number of applications has increased, and that means new technologies, new tools. Of course, our infrastructure has grown. That also means that we have a lot of technical debt that's grown over the years.

So to keep up with that complexity and to keep up with that technical debt, we've added process and organizational structure. And we have silos, and we have a lot of silos. We have silos within our silos because we really want those silos to be as efficient as they can be.

And so we've got this technical complexity and technical debt. We have organizational complexity with lots of silos. And of course, the world has changed a lot in the last 50 years since Target first opened that store. How we communicate, how we dress, how we listen to music, watch TV, that's all changed. And how we shop, that's changed dramatically in the last 10 years.

I'm a mom with two kids, and I think about myself and how differently I shop now than I did even six months ago, and that's because my expectations continue to evolve, because Target guests have changed how, where, and when we want to shop.

Technology is in the hands, it's in all of our hands, and it's not enough for us to be a big-box retailer anymore. It's about connecting with our guests anywhere, anytime, anyhow.

Retail has been completely disrupted, and it's being disrupted at an increasing pace as innovation and changing technology just continues to explode. Target needs technology to remain a leader in retail. Retail's broken away from brick and mortar. We need technology to be flexible, scalable, and have the ability to quickly test and learn new strategies, whether that's for our guests, within our stores, or even within our supply chain.

All of those complexities, the organizational complexity, the technical debt complexity, the evolving, changing marketplace, that was what was behind a key technology strategy that we kicked off in 2012 around APIs. We had to start creating APIs to expose core assets so that Target could create these cool digital guest experiences quickly and also to simplify our internal architecture.

Those legacy technologies that I've talked about, those have grown over the years with a lot of point-to-point integrations, and we have a very tightly coupled architecture. That means that quarterly release cycles are the norm. And I think you know probably as well as me, when you're doing big releases just a handful of times a year, there's a lot of dependencies in the ecosystem, and that always adds a lot of risk and typically a lot of instability as well. So it was important for us to focus on building out some APIs to decrease that complexity as well.

Initially, we focused on building out APIs that you would imagine would be important for a retailer, like a products API or store locations, promos, pricing. And we have. We've built and exposed a lot of those APIs from many different back-end providing systems that ran across a lot of different platforms.

These APIs now are enabling internal applications as well as our mobile applications and a lot of capabilities on our dot-com site. So we can quickly test and learn with different guest experiences internally and then also when we're working with partners like Pinterest.

That makes a lot of sense. Probably no surprise to anybody here, it's important to have APIs to be able to move quickly and innovate.

But what was more interesting--I shouldn't say more interesting. The reason I'm standing here today is because, yeah, it's been great to lead the strategy for Target, for one of the world's largest retailers. But just as challenging was the fact that we were talking about how we were going to deliver those APIs, and that was a fundamental change to how we were doing work in a large enterprise IT organization.

So we started talking about things that were going against the norm, like agile, not waterfall; smaller, more frequent releases; automating all of the things; continuous integration; continuous delivery; and of course, DevOps. These were entirely new concepts for a large enterprise IT shop and very different than how we had been doing our work.

Now, in the early days of this DevOps journey, I actually had to stop using the term DevOps. It had become a loaded term with a lot of misinformation and misconception. And as I would have conversations with my peers and other senior leaders in the organization, I would literally see them shut down as soon as I would say DevOps.

Some laughter indicates that others have had the same experience. Probably not unusual.

So, right, it was going to be hard. Being a change agent is really hard, but I knew that we could do it. So I focused on three things for my team in the early days, and that was talent, culture, technology, so that we could deliver results.

There was a lot I needed to do as a leader for an incredible team to pave the path forward so that we could achieve these results. And the only way forward was I had to figure out how to start removing roadblocks and being a constraint buster for my team so that we could demonstrate by doing and showing results.

So this is my awesome team. They build cool stuff. They like to innovate and try new things. We started as a pretty small team in the early days with some really great people like Dan Cundiff and Greg Larson.

Now, the fastest way to demotivate them, the team, well, probably all of us, and to stop progress in its tracks too, is bureaucracy. Nothing is worse than running into a roadblock that stands between you and building stuff. And so what my job is, is to empower my team to get stuff done and to not be paper pushers.

I wanted to minimize grunt work, and we've made some really great progress. One of my highlights from last year was when my team presented me a lifetime achievement award for dismantling an entrenched process that was just one of those processes that everyone was doing because that's what we had always done.

So I started asking questions. Why are we doing this? Why are we adding hours and hours and hours of work in order to just bring new technologies in or to try and innovate with different ways of doing work?

And as I started asking those questions, just like, "Why?" and the answers would come back, "Because that's what we've always done. We have to go through that process, right?"

No.

And very quickly, everyone's kind of started scratching their heads and getting on board, and you're like, "Yeah, you're right. We don't need that process." So sometimes it's as easy as just asking those questions to really remove inefficiencies that we have within our system.

My focus is always on results, not process adherence, and identifying those types of roadblocks and working with folks across the IT organization to drive those efficiencies.

To make that work on my team, I created a safety zone because I needed my team to feel comfortable with innovating, failing, learning, and moving on. I needed them to trust that everybody on the team had their back and that everybody was empowered.

So I provided cover so that my team could make awesome happen and trust that I could remove roadblocks for them. That meant that it was important for me to build trust with my peers and leaders across the organization, and I needed them to trust me as well. That was really hard.

Has anybody ever been called a hippie that wants to live with the unicorns and rainbows?

I have a few times, and it was just a result of the fact we were doing things differently, and the rest of the organization just didn't understand what that was in the early days.

So I needed to prove that it wasn't just a bunch of hocus-pocus, fairy-dust magic. So I demonstrated results with data. I took the emotion out of the equation and used metrics.

Initially what I did was I ran parallel development efforts to capture cost and quality metrics for a more traditional delivery model versus the changes that I was advocating for. And the results showed faster delivery, higher quality, with overall less cost. Awesome, right?

So now I wasn't just preaching and talking theory. I had actual data and metrics to demonstrate to the rest of the organization what it was that we were doing.

To do that, we needed the right tools in order to get our work done. We have full-stack ownership for our APIs and our API platform, and our tooling does matter. So we brought in a number of new technologies to create the automation toolchain for Target. That included OpenStack, GitHub, Jenkins, and Chef.

We built transparency into our development process. Code reviews are dead, so we use pull requests, and our ops team loves this because they can actually see what's happening. We're not throwing code over the wall, and they can provide us feedback, and they're part of the development process. And also when something breaks, we know it, and we can fix it really fast.

We also use HipChat on the team, which has been really awesome because the team not only can chat throughout the day, team including our operations and support teams, but also we can stream in alerts and events, and we can real-time see what's happening across our ecosystem.

Now, of course, as we brought these tools in, we worked with a number of partners across the organization, including Ross, Jeff Einhorn, and Elizabeth Carpenter, all of whom are here today. It's been really great to see the adoption of these tools and some of these new processes start to expand across our organization over the last year.

So this is my brag slide.

We now have more than 30 APIs in production with monthly volumes of more than 1.5 billion. Ross and I gave this talk back in the July timeframe, and that number was, I think it was just a half a billion at that point. So in the last three months, we've more than tripled the volume that we're seeing through these platforms and APIs, and the metric that I like the most on this slide is the stability that we have across our platform and our APIs.

We have less than 10 incidents per month, and that number hasn't changed as the volumes have gone up. We also do more than 80 deployments a week. We have 250 commits on an average week. So there's a lot of change as well that's being pushed through the system, but it's small changes.

And so that means more stability for us because we know when a change is going in, and if something goes wrong, we can fix it quickly.

I also want to call out, just from a business case perspective, how successful this program has been. We have an IRR of more than 80% with a really significant NPV as well.

Now, these metrics here and some of the others I just mentioned, I didn't have those three years ago. I didn't have transparency into my development process. It wasn't easy for me to see how many times we were doing deployments or how frequently different developers were committing code. Now I can, and it's really powerful.

So it's been a really fantastic three-year journey on this dev team that I've been leading. But what's more exciting is the change that we're driving across the organization. Target's a technology company, and we are committed to making DevOps a reality, and Ross is going to tell us more about how we're doing that.

Ross Clanton

So those results are extremely impressive, especially if you saw the cultural and organizational hurdles and barriers that Heather and her team had to go through to make this happen.

While Heather was driving awesomeness in her own space, I was kind of going through my own DevOps journey. I was moving into an ops role after starting my career in ops, spending time leading in engineering, security, went to enterprise architecture on the dev side, and now I was coming kind of full circle.

Needless to say, I'd gained enormous empathy around the challenges that these different functions face within the organization. And over that time, I definitely broadened my perspective on the systemic issues that were actually impacting our service delivery.

It wasn't any one function. It was a problem with our organizational system. And that problem was misaligned incentives, too much manual process, too many handoffs, many silos, and not enough accountability across the organization.

I think that's a problem that many of you are probably familiar with today. And it's very difficult to get to good results when you're in an environment like that. So at this point, I was clear on the problem, and I was left thinking, "There must be a better way than this."

Now's the time for the token Phoenix Project slide. I'm guessing you guys will see 20, 30 versions of this slide over the next few days. Most DevOps presentations tend to have one in there.

And I think this is really, really important to me. When I was coming back into ops, I reached out to a trusted thought leader in infrastructure at Target, Matt Walburn, and asked him, "What should I do to get back up to speed? I've been out for a while."

He just recommended one thing: "Read The Phoenix Project." And that book was so eye-opening to me because I kind of felt like I had lived the role of all the main characters in that leadership journey I'd been in. So it really, really resonated with me, and it helped me start to see the importance of DevOps more as a cultural movement and really how it can connect people across the organization.

And it got me excited and passionate to learn more. In fact, I felt so strongly about this book that later on, after I did move into ops, I bought 22 copies of the book, assigned it out to my management team underneath me, key partners, key team members that had a role in driving this agenda, and it was like a homework assignment. I even had an offsite on it where we virtually recreated some scenes in the book.

And I got passion. And when I get passion, I want to learn. I want to understand. I invest deeply to understand things.

So I researched, I learned, I talked to experts internally, externally. I learned about the innovations that were happening in the industry. I started looking at the unicorns and looking at what Netflix and Facebook and Google and everyone was doing.

And one important thing I learned was how to start seeing and treating failure differently. It's not something to avoid, which is how I was brought up to believe, but something that's really key to innovation within an organization. And we, as an organization, needed to learn to think differently, and that's hard. Changing minds is a very difficult thing to do in a change agenda.

So at this point, I worked a lot with some close partners, many of whom are in the room today, like Jeff and Elizabeth, to not only mind the gap, but recognize that it existed in the first place. And that was hard. These were different concepts.

And some things that I had to do at this stage, like Heather, I tried not to overuse the word DevOps. I used outcome-based language like, "We're driving leaner, faster, higher-quality service delivery." There's no DevOps in that phrase.

People don't want to hear about unicorns and rainbows when they aren't part of this community and this culture, and they don't understand what that means yet. Once they become part of the community, they do like talking about that stuff.

And one story, actually, at this stage in the game, I went to Velocity last year, and it was shortly after I moved back into ops, and I brought a bunch of my managers with me. I didn't bring any engineers, which is kind of weird. That's an engineer conference.

But my goal was to expose them to this thinking, and I wanted them to see the passion and the magic that happens and what happens when you have highly empowered, highly enabled engineers that feel passionate about doing things. And it paid off in spades.

Those leaders came back, played key roles in helping drive this change agenda once we got back from that conference. And now we're sending engineers to these conferences, obviously.

I also made a business case to build my own rockstar team, and the goal was really to accelerate building our competency in service management, infrastructure as code, and performance engineering across the organization. And I really wanted to extend the tooling that our architecture partners, both Heather and Jeff Einhorn from infrastructure architecture, had brought into the organization.

And my goal was not to make this team yet another silo. We have plenty of silos. But really to leverage them to enable and empower others to learn how to do these things and to make awesome happen in their own spaces. And we do that through a number of different ways that I'll talk about here in a second.

So like Heather, I had to create a culture where everyone could level up. And that meant doing something scary. That meant experimenting, testing, failing, and ultimately succeeding.

I've learned that this is a key to establishing a learning culture within an organization, and ultimately, that's how you enable continuous improvement. And to do this, I had to fully empower this team, and I'm really proud of that team because they embraced these principles even before they were as culturally acceptable as they are today at Target.

And if that wasn't enough, we also started doing Lego Ops. We embraced the new toolchain. We started looking at our infrastructure as building blocks and started building reusable components, primitives of infrastructure code that can be stitched together in ways that give us flexibility in how we do service delivery.

And with this flexibility, we're starting to test and learn on some different service delivery models, and our goal is really to find the best models that will go the furthest in delighting our customers.

One way we've done this is I've been a big proponent of introducing agile approaches to our infrastructure and operations organizations. So we had our teams doing Scrum, sprints, Kanban, different techniques, more modern agile-based techniques to get our work done.

And we even took that a step further and started doing what we call flash builds. And really what was happening is demand for the team's time was ramping up and we needed more partner folks involved, and it's hard to get a partner team and an infrastructure team that isn't oriented around agile to give up one of their people to be in one-week sprints or two-week sprints.

So a flash build is an eight-hour day filled with two sprints, two four-hour sprints filled with sprint planning, the development work that you do, retrospectives, demos, highly engaging, high-speed, high-energy process. Got to have a really disciplined Scrum Master facilitator to lead that type of an event.

And the outcomes or the benefits of that is it allowed us to optimize our time because we had everyone in the room working together on these problems. It allowed us to bring in those partner SMEs because they could free up a day at a time. Maybe we do a flash build once a week or something that's different for every engagement that we do.

And it's more fun. The engineers and the folks in the room are having a blast doing this work, and we get faster results. In fact, our very first flash build that we did, we got done in an eight-hour day what took many, many weeks following our traditional delivery processes.

So where are we at in terms of enterprise results?

The below metrics here tell a powerful story in how collaboration, new tools, infrastructure as code, and a DevOps mentality are bringing Target team members together to achieve great results. In doing so, they're utilizing social coding in GitHub, creating over 270 repositories to date with ever-expanding test coverage.

Whenever anyone updates any of those builds, it's going through automated testing to ensure that we're building a quality and consistent product.

And one thing that's really powerful here, or even more significant, is the momentum that we're getting. If you go back a year, there was really just a couple teams doing this stuff. It was basically Heather's team and our dot-com organization, and there was some POC work in our infrastructure architecture organization.

In July, when Heather and I gave this presentation, we had 11 teams and about 71 team members that were all contributing to infrastructure as code at Target. Three months later, that's gone from 11 teams, 71 team members to 30 teams and 134 team members. So you can see the momentum and the growth that we're starting to get across our organization and how we're doing this.

I'm going to shift gears and talk about how we started to drive DevOps within Target. And what we recognized at this point is we weren't the only ones that were looking to change things. We've already mentioned some of the other partners here today as well.

Many groups were starting to focus on automation across our technology organization. Some, like our dot-com organization, were even exploring DevOps.

And at this point, we really worked hard to establish partnerships with a lot of these stakeholders because the vision was really to enable and empower our technologists to automate across Target and to drive more end-to-end improved service delivery.

And the time was really ripe at this point to expand the conversation on DevOps. So Heather and I talked about this, and we decided we're going to do our own internal DevOps Days.

We decided we were going to champion calling it what it is. We actually started calling it DevOps, and that was a hard step. We had to overcome some of the challenges that Heather mentioned earlier with negative connotations on what that meant.

But we also recognized that DevOps has a significant meaning to folks in the community, and we've got a big, strong, growing internal community. We have about 400 members of our internal Target DevOps community now, and it has a lot of meaning to them. We needed to call it what it is. It had resonated with them and had meaning for them. So we started doing that.

And we held our DevOps Days. Our moniker there was "Connect, Share, and Learn." That's really our goal.

Our goal is to connect people across the organization that were interested in this topic, give them a forum where they can start to come together to share experiences through demonstrations, Ignite presentations, open spaces, where people that are actually working on these types of things, these different ways of doing things, both in our development organizations and our infrastructure organizations, tests, wherever, have them come and start sharing what they're doing. And ultimately, to learn.

We wanted to get people together so that they could learn from our internal experts and a number of external experts that have come in and presented for us as well. Rob Cummings from Nordstrom was actually our first kickoff speaker, and that was awesome. We've had a number of other folks. I won't go through naming who they all are today.

Being DevOps folks, we capture data on these events. So we've held three DevOps Days. We do them quarterly. We started in February. So we've had three events, we've had 50 presenters across those events, and we've had over 650 attendees over those three.

In fact, one event we coordinated globally and had DevOps Days in our Bangalore, India offices the same day that we were having it in our U.S., Minneapolis offices.

And, of course, we get survey feedback: eight out of 10 on the awesome scale, and 100% of the attendees have said, "Keep doing these." And so our one-year celebration of our DevOps Days will be coming up here in February.

And in doing this, we were able to start building a community. And as a community, we started questioning whether the way that we were doing things was the way they should be done.

And we viewed questions and resistance as a negative, but the important thing here is we have silos as an organization. We're always going to have silos. We have a massive IT organization at Target. Silos by themselves create boundaries, and those boundaries create barriers to communication. We needed to break through that.

And what we really did with building this community is allowed our stakeholders to start boundary-spanning and start connecting together.

Yeah. And so we did this by challenging the norm, questioning the way that things were, and we didn't view resistance to our new way of doing things as a threat or a negative. We actually viewed it as an opportunity to start doing things differently and to pull people closer and be inclusive.

One way we did this was by publishing our own Flipboard magazine, Make Awesome Happen. Anyone here today, by the way, can subscribe to this if you have Flipboard and you want to scan the QR code. This will be in the presentation that goes out later as well.

But we built this magazine about five months ago. It's curating content from around the internet, and it's open for anyone who wants to subscribe to it. And we have 208 readers today, 109 articles, almost 4,000 page flips.

And our goal was to set a new course, to do things differently, and to enable and empower our technologists to be free. That's really why we hired them in the first place: to be free to innovate, to start using our outside voice.

Whether that's us here presenting today, we finally have an external-facing tech blog that we're encouraging these participants, our members, to communicate externally. Whether it's Target sponsoring DevOps Days Minneapolis, we're doing a number of things to get far more active in the external community.

Oops, I went backwards.

And our goal is to really start something new. It can be small. It just needs a place to grow, and this largely started as a bottoms-up movement that is very much being fertilized top-down now. And ultimately, to create a culture of awesomeness that everyone wants to catch, a culture that's focused on getting stuff done and progress over perfection.

And if I were to leave you with one final takeaway, it would be to take the leap. It's working for us, and being a change agent in an organization can be very difficult. It takes perseverance, passion, resilience. Most importantly, it takes partners that are either willing to take that jump with you or they're standing on the other shore encouraging you along.

We've got a lot of work left ahead of us to continue moving this journey at Target, whether it's figuring out how to do continuous delivery across the enterprise, testing and learning different models of getting stuff done, and continuing to create and share our story. But even though we're still early in our DevOps journey, change is happening and we're having a lot of fun.

And with that, we're the first ones to share our slide of where we could use help from this community.

And so here's what we'd like help on: accelerating our DevOps transformation, aligning incentives, and shifting to a blameless culture. I think that's a very hard cultural shift to do in a very large enterprise context, and that's the journey we're on right now.

So we welcome any advice that you all have there. Thank you.