Log in to watch

Log in or create a free account to watch this video.

Log in
San Francisco 2016
Share
Download slides

The Need for Speed: Enabling DevOps through Enterprise Architecture

Have you ever wished you worked for a unicorn like Netflix or Amazon? Have your colleagues ever told you, “DevOps just won’t work here!” Many people hold the belief that DevOps is not achievable in distributed product teams, large enterprises, or highly regulated industries. This session is for the haters.


This is the story of a DevOps transformation inside the world’s largest healthcare company: how a highly siloed, matrixed IT organization is using enterprise architecture to leverage challenges and identify constraints, run experiments, and ultimately evolve into a highly resilient, customer-centric delivery organization that continuously re-aligns IT with business intent to continuously deliver value to the customer.


What began as a need for speed, led to experimenting with enterprise architecture to find ways to decrease lead-time across all of IT (versus optimizing specific functions or products) and focus on throughput. Through these experiments, the enterprise architecture group uncovered guiding principles that encourage the natural adoption of DevOps rather than the common, mega-enterprise practice of mandating the a top-down Framework or big-bang installing the hot new transformation of the year methodology (aka Bi-Model from Gartner).


Ultimately, horses (enterprise IT organizations [aka Clydesdales]) must learn the 3 Ways of unicorns or face extinction, but the key to the horse’s journey will be the most unlikely of guides: enterprise architecture.

Chapters

Full transcript

The complete talk, organized by section.

Mark Landy

I'm Mark Landy. This is Will Evans, and we're going to go. I'm from Johnson & Johnson. Will is from PraxisFlow. Will is Chief Design Officer in Residence at NYU Stern, also one of the bright minds at Praxis.

We've been knowing each other for about, what, three years now?

William Evans

Yeah, about that. I first met Mark about three years ago, and we kind of embedded some of these ideas inside of the talk itself.

When I first met him, he had some very interesting challenges. He had just previously been the chief technology officer at a company called Medco. He had moved over to Johnson & Johnson, and it's kind of like you've been sailing a 34-foot boat for quite a while. You've become relatively good. In fact, you're a damn good skipper. And all of a sudden somebody gives you not just the keys to the enterprise aircraft carrier, but an entire fleet. And so the scale of challenges is very unique.

But Mark also was actually overseeing the Agile transformation at a company called GSI Commerce, which he had done. His background is he's an electrical engineer. He likes to get in there and actually observe and figure out how things work. This really matched a lot of the PraxisFlow ideas around experimentation and the scientific method. Mark already had that in. I think all I've been doing over the last couple of years is providing some ideas once in a while.

Oh, we want to make sure that you tweet the crap out of this.

Mark Landy

Yeah. So #DOES16. These are our Twitter handles.

William Evans

Before we get into the real meat of things, I think Gene Kim is in the back of the room. Gene, are you there?

There he is.

The only reason I say this is I want a huge round of applause for the community, the empathy, the respect that he has been gathering together over the last three or four years. So a huge round of respect for Gene.

And one more thing I want you to do. If you could whip out your smartphone. First, set it to vibrate. Second, turn on your camera, get close to the people around you, take a selfie, post it on Twitter with the hashtag DOES16, hashtag DevOpsCulture. That way everybody knows what's going on here. That would be awesome. I'll give you one minute. It's time boxed. It's a constraint.

Perfect.

Mark Landy

All right, so while we're doing that, a little bit about Johnson & Johnson. I'm sure everybody understands or has heard of what Johnson & Johnson is.

We do a lot of IT work, and it may not have been known. Our company traditionally had kept IT in the back office. That's the convention where it was. Lots of heavy manufacturing and R&D. The science was really in the labs with the lab coats because we do things like actually cure major diseases, which for me is a pleasure because, I've got to say, in all my years I've been working in healthcare, it's usually been in the access or insurance markets. Now I'm actually in the delivery.

When you're in IT and you get a request from a scientist who says, "I need to upgrade a cluster," and you start to ask, "Why? And by the way, why do you have your own cluster?"

"Well, so I could do these algorithms."

"And what are you working on?"

"Oh, immuno-oncology."

"So, okay, you're curing cancer. Got it."

How do you monetize that stuff, right?

We're going to go through what is going to be an interesting journey on emergence and weaving DevOps and architecture together.

William Evans

One of the interesting things, and this kind of sets the context: the previous slide tells you a little bit about the size and scale. 500 terabytes of data. 450 apps released every single year.

The other interesting thing, and Johnson & Johnson has grown both organically, but also through acquisition, so inorganically. One of the key things to remember is when you have, at any given point, about 14 acquisitions or divestitures that are currently in flight, talk about limiting your A and D, right? That creates a whole lot of negative behaviors. It means that we have things like a cycle time for an acquisition until we realize business value of like four years.

So there are also a lot of different kinds of pressures that are happening. There are industry pressures that are happening. There are external market forces that are putting a great deal of pressure on how you actually deliver value.

But the other thing is just the major disruptions that have happened just over the last 15 to 20 years, as more and more IT stuff that people had been investing in, which became CapEx, which we needed to depreciate over a significant portion of time, actually becomes baggage that slows you down.

Mark Landy

Right.

William Evans

All these new startups, they don't have all that baggage. They're able to spin up instances immediately on AWS and really start to innovate, what we would call at the fuzzy front end of product innovation.

Mark Landy

Right. And so with all of these disruptions that are occurring in the healthcare ecosystem, certainly the move to better access, the move to better outcomes, this all speaks to data. It also speaks to, we really don't all know what it means yet, so we have to be able to change quickly and pivot.

You combine that with our company's natural tendency to, at this stage in its life, over 128 some odd years old, grow through acquisition. And so whatever you plan or whatever you think you can build in a... And the conversation before is all about process, right? By the time you think you've gotten something process modeled and documented and pushed out, in three years, another 20 or 30% of the company will be different.

You really have to look at what we do at J&J as serving a small market of business needs that really is an amalgam of the actual market in small to mid-size healthcare. That's the way we treat it.

We're going to cover more fancy terms. Ephemeralization, which really is at the centerpiece of how we've put together enterprise architecture and DevOps. Ephemeralization, everything from virtual or software-defined, all the way up to rapid experiments that are safe to fail, not fail-safe.

This mindset, this thinking, which actually goes back many, many years before virtualization, literally means doing more with less to the point where you can do infinite work with nothing. But that's kind of like calculus. It's almost like L'Hopital's rule.

William Evans

Right. This was a thought experiment that Mark introduced me to before we even began the journey, before we started to think about how would we design the federated enterprise architecture.

We were thinking about this ephemeralization, and originally the notion came from Buckminster Fuller. What he was doing was really critiquing this guy, an economist named Thomas Malthus, who said we have a finite number of resources. Those resources are being depleted every year. But our population growth continues to grow and grow and grow.

Buckminster Fuller's reply to that was actually that we're learning how to do significantly more with less. This is kind of informed by Moore's law, which as computing power doubles every 18 months, it requires less and less materials, less and less raw resources, to be able to get better business outcomes or more business outcomes. These things kind of informed the way we thought.

But the other thing is this leads immediately into, can we plan up front a major transformation when, at the end of the day, Johnson & Johnson is really 260 operating companies within three sectors, where each one has some degree of autonomy? Many of the local OpCo CIOs or heads of engineering have their own P&L, which means they can make a lot of decisions. Sometimes we can't necessarily, from the top down, from the executive committee down, dictate how everybody does their work.

Mark Landy

Exactly. We can't dictate it, and it never really made sense back in the days when we thought we got away with it as an IT industry. It really is about emergence.

If you've ever done major IT project work with real project plans, the planning is awesome. The planning is great. When you actually try and chunk through the program and, by gosh darn it, you're going to stick to these milestones, you got it done by the relationships and the people that you knew, and you crashed at the end.

So we've all learned, hey, let's not just do that again. DevOps and Agile and all of those things think about the flow and value creation, which is a left-to-right, not a top-down controlling.

So informing us for how I'm going to do architecture, and why architecture? Well, I was hired principally because friends of mine ended up leading parts of IT, and that's how it works in our industry. But I was hired to create, and really create, a version of enterprise architecture that drove value.

Now, if I just got some TOGAF certifications and had people trained and installed an inventory tool, I could just say, "Done."

But why, right? So why do we have enterprise architecture? To me, it was a huge opportunity to weave the DevOps spirits of flow, value creation, but at the high-order economic level. So not just getting work done faster, but why are we doing this work as opposed to other work? And what economic value does it generate for our company?

We were seeking and getting permission to ask for relaxed constraints at the business level. Conveniently, our leadership was looking for that at the time. So it started to really emerge.

Emergence, combined with fancy words like ephemeralization, really just means establish the constraints, situation, context, and coaching and care and feeding so that an outcome, ideally a beneficial outcome, will grow.

With 260 some odd operating companies distributed around the globe, changing all the time, you really have to go with emergence. The model I like to use: if you had your smartphone, when you bought that, did you get requirements docs and a change management program? No. You liked it, so you got it.

In that regard, what can we do in J&J, in IT, to create awesome value for our business partners? That had to be the rule. To do that, that also meant that we had to think about the constraints and when we can get into shaping our demand.

William Evans

The other thing, and this is really important and it kind of leads to the very next slide, is allowing for the conditions where emergence can happen, where people can run constant experimentation. They still need to understand what is the why, right?

We're kind of explicitly referencing Simon Sinek here. What is the purpose? The nice thing is J&J actually has this. One of the things, Mark will describe what the credo does and where it came from, and then I'll give kind of a philosophical spin on it.

Mark Landy

The text you see here, and please go to our website and find our credo. I read it when I joined the company. We use it as an internal rubric for decision-making. It's used at the highest level of business ethics.

It is in four paragraphs. It was written by General Robert Johnson, 1943. He was chairman from '32 to '63. It was written just before going public. The reason for that, if you might imagine, is if the fourth paragraph is finally to our shareholders, well, if that's the only consideration, we might lose our way.

So there was a sense that having a noble cause and purpose was something that we needed to institutionalize and make part of our leadership. Who we hire and how we coach and how we train them and reward them is based on our credo. That actually turns out to be a nifty set of constraints when we're thinking about what our system is.

William Evans

One of the nice things, and the credo, I've seen this. What's funny is a lot of organizations have what I would call the ninth waste of lean, which would be mission statements and value statements, right? That's the ninth unstated waste in lean.

But the credo actually acts as something that guides, informs, and provides some set of guardrails so that we could actually empower and enable the emergence of different kinds of work streams to happen, different ways of people organizing, different kinds of experiments that would actually go about.

But before we did that, we just wanted to insert one more kind of backgrounder thing. Where did this all come from when Mark started his journey a couple of years ago?

Mark Landy

Yeah. Who started it?

William Evans

Or whose fault was it?

Mark Landy

Whose fault it is, right.

Well, I could start emergently enough. People were interested in The Phoenix Project. It was a great book, and it resonated with folks that kept thinking, "There's got to be a better way. Why does this feel like an out-of-body experience, this IT work I'm in?"

The Phoenix Project made sense to a lot of people, and a lot of the folks that were starting to think about, "Why can't we at J&J use public cloud? Why do we have to buy all these canned software packages? Why can't we write our own?"

Well, we were coming out of an era of highly regulated and very staid, risk-averse thinking. Risk-averse in the conventional sense meant, "Don't touch it. Leave it alone, and it'll be okay." That's not going to handle the disruptive elements or the opportunities that we talked about before.

What started as a simple book club actually resulted in me... I think I met Gene once at a small conference in New Jersey, and then talked to Kevin, and literally had Kevin come out to Redwood City and do a book club for our IT leaders, which I think was pretty much the start of it. That's how I met you.

William Evans

Right. Then, I guess he hadn't had enough abuse, so he invited Kevin and I to come back to J&J in Raritan, New Jersey, where we did a thing.

This is one of the thinking tools that comes out of Eli Goldratt's theory of constraints. We did a current reality tree, and we tried to identify, what are all the potential undesirable effects that are happening within your system that might prevent a move to flow thinking, that might prevent the transformation of this organization?

So we didn't even necessarily say DevOps. We didn't say necessarily Agile or top-down lean implementation. But we said, just what would prevent us from getting to flow, and thinking in terms of flow? Because it really was about culture and mindset change, and these things are not something that you can necessarily product-roadmap your way towards.

After Kevin and I did that, and we ended up doing a couple of them and then talking with some of the senior executives, Mark was able to secure enough funding just for us to do some small experiments.

I'm reminded a tiny bit of Greg Grier's book, Lean Transformation, and he talks about sometimes organizations, when they're doing a DevOps transformation, they'll do it with three small teams. They'll identify all the positive patterns, and then they will immediately go to scale it out across the organization.

I hadn't read the book at the time, but I was thinking along the same thing. I think he was running those experiments to see whether or not we were full of shit or not, right? So he gave us some small teams, and we learned a bunch.

We also learned what are some of the dysfunctions, what are some of the constraints that are really preventing people from thinking differently about the way they work. One of the things that we introduced to Mark was this.

Mark Landy

Good old Melvin.

So how does this come together? DevOps, architecture. Where's the architecture part? It's coming. But you can see where our thinking is going. It's full system.

One could argue, particularly in architecture, architects are solutions negotiators, not so much architects. I think if you've ever had experience with architects who really output actual work and deliver, they are negotiating for some technology assignment to some business need in collaboration with IT professionals and business.

For me, one of the biggest challenges I was given in coming up with a manner in which to improve the economic value of IT through architecture as one of our governing bodies was, why can't we get Enterprise X? Enterprise Data, Enterprise CRM, Enterprise ERP.

We bought 260 versions of those enterprise software packages. But that made perfect sense. Conway's Law says that, in a sense, if you're a system that creates other systems, you're going to create those systems in the genetic structure and DNA of yourself.

Conventionally, IT was the capital expense, capital and expense that went into building a factory and was depreciated over some time, either 5 years or 10 or 15. And it was always for the local P&L. Believe you me, that is very hard to change.

That really is where we get to our next slide.

William Evans

What to change.

One of the things, and especially for TOC aficionados, yes, I realize that there are only three questions that Eli Goldratt came up with. We kind of added a fourth, which was the why change? What is important about it? What problems are happening?

What we really identified was that a lot of people were, from GNOs all the way down to line employees, the people that actually create the value for the organization, all the way up. They were either localizing for their goals and objectives. They were optimizing for their performance evaluations. The team was optimizing for whatever it was that would make sure that they didn't get into trouble, or to prevent a death march.

If you think about that all the way up, just within an OpCo, and then think about an operating company CIO optimizing for their organization, nobody was even thinking about, how is this impacting the rest of the org, and how are we going to achieve any kind of enterprise value?

Mark Landy

Another why change, and I can talk about some of these elements because they've been put in the press. As you can imagine, by the way, having a company like J&J show up at this kind of a tech conference, very rare. The amount of approvals that I had to go through, you have no idea.

But well worth it. That tells you something about the leaders in our executive committee. In fact, Alex Gorsky, who three years ago I first met at a meet and greet for new leaders, when I asked him about tech innovation, he said, "You know, we're really the health part. We're going to let our partners do the tech."

Now it's the other way around. If you talk with him or you see some of his analyst comments, we are a health tech company. You're going to see a lot of activity in that regard.

So one of the whys to change is, how would I take care of improving health outcomes, reducing costs, and improving access for people in developed or developing worlds? How could J&J do that differently and uniquely?

One might be that I could actually bundle outcomes from a surgical procedure for a knee or a hip implant with consumer-based coaching and behavioral analysis, the actual device itself, and then pharma-level clinical informatics to prove the effectiveness.

That means you really need to have something that cuts across all of the operating companies. So how did we get that? What system are we in to do it? It had to be our economic environment. In fact, it had to be our credo.

It couldn't just be, "This will make this IT cost much less," or, "I will get your projects done much sooner." Because that's not what the company was really asking for. That's a challenge because that really made IT step up and have to become a true business leader and partner, which we did.

Classic local versus global. When you think enterprise, in our world, enterprise is conventionally, financially the aggregate and roll-up of all of the sales and transactions of the operating companies. What it's supposed to be, though, is the interaction of the parts as a system in novel and new means.

In doing that, we have to do things differently. Building things the same way no longer fit the mold, but it was a very hard thing to go out on a campaign and tell people and ask for that participation. We had to do something better.

Back to architecture. We created a model that we call Federated Enterprise Architecture. What that basically means is rather than have a group of people that either pursue or block IT work, or hold the standards, or do the metrics, we would deputize IT and business leaders in position so that they could be our sensing agents, and they could also affect change before an IT initiative came to a shared service to be built. So it had the right enterprise plugs and ports considered.

In doing that, we had to run through, and this is where Will and Praxis really helped. We had to build a set of courses that was about a week, maybe four and a half days: theory of constraints and all of the related elements, cost of delay. We built a white paper that we gave to our IT leaders, to our CIO, Stuart McGuigan. By the way, that white paper came back with comments written throughout the entirety of it.

Enterprise architecture for J&J became essentially a focused set of training, almost in a boot camp, where we changed people who were in an architectural role to now think about the system they're in and get armed with a community and also continued coaching, so that we can do things like cost of delay and actually have people understand it.

But that still doesn't solve the problem.

William Evans

There were a couple of things that we noticed along the way as we were training these people.

One, while we saw a huge amount of value at the team tactical level, implementing Kanban boards so that we could limit WIP and improve the flow of value, the other thing is that a lot of the enterprise architects that we had deputized and sent out, they don't actually contribute to the production of any particular IT system. The flow in their systems is actually decisions. And usually, it was bad decisions.

One of the things we use, and it's kind of a joke...

Mark Landy

Local decisions.

William Evans

Right, local decisions.

One of the things that we used to train them was simple things like if anybody's seen the Lucille Ball chocolate video of the factory. Just teaching people to understand that if you don't see what is coming from upstream, and if you don't understand or have empathy for the people downstream, it's very difficult for you to understand the why of what you're actually doing.

So we thought about, okay, we want to have a period of time where people are actually putting all the decisions into a Kanban or a personal Kanban, tracking the flow of those decisions and just tagging them. Is this more value creation? Is this value demand, or is this failure demand? And if it's failure demand, how do we root cause that so that we get more decisions flowing through the system?

The biggest root cause that resonated across all of them...

Mark Landy

Was Mr. Conway again. Or at least our version of him. But applied to something that we all hold dear and we never challenged, which was projects.

Projects in our world were, considering the system had to be enterprise, the project was actually a sub-scale constraint laterally for different silos of data and function, and temporally because you had to do everything within a year.

We relaxed those. We didn't relax it for every single thing that we've got in the shop. That would be nearly impossible. We have to cover some ground. We relaxed it for data, and that's currently what we're doing.

Enterprise architecture applied at Johnson & Johnson is not so much the practice of it, but it's the meaningful creation of value along Goldratt's theory of constraints. Subordinate all else and move on.

William Evans

Which gave us at least a cover, and this is where we're moving to, is this notion of embedding this continuous improvement all the way down at the team level, but then all the way up at the OpCo and then the senior executive level. We say, for flow of IT capabilities that deliver economic value.

So we have our standard explore, optimize, create a new position. One of the other things that we did was we said, okay, for all of these enterprise architects, if you're going to be sensing business intent and getting ahead of that so that bad things don't end up at the doorstep of procurement and all of a sudden you have a capital expense request, we need to teach them how to talk to people and talk to the business.

We're going to need to figure out what is our current standard work, and then teach them the basic double diamond of design thinking. So you can see that it's kind of a mix-match and with a heavy toolbox, of course, you've got to lug it around, but provide the coaching as well so that people would know if I really need to understand what is the business intent, I need to know who's the customer.

Well, that means these people need to actually learn the skills of talking to human beings. The nice thing, at least, Mark was able to get us a meeting with the global CIO, and he said, "Normally, IT thinks about their customer as the business unit IT or the business itself. I want you, and I'm giving you the sponsorship and the freedom to say the patient itself is the customer."

Mark Landy

So our IT leader gave us that permission, and I'll wrap up because I know I'm over. This is hard to put in 25 minutes, though.

Technology is a benefit if it diminishes a limitation. Our limitation is considered the larger economic good. In fact, what's getting in the way of J&J and its noble cause?

And here's proof of life that this model is working. This was one of the first deliverables that was under it, which is our software-defined and hybrid cloud environment that literally used some of these toolkits and gave us reason and hope for there's going to be a lot more.

William Evans

So to conclude, these are some of the principles, at least, that we've discovered. These are going to change. Some of them are riffing on other principles, like lean principles.

But visualize your work, but first visualize the system. One thing we did was had people value stream their work, then fit that into the value stream of their upstream and downstream customers, and keep going up two levels so that people understand the relationship of the value they're creating to the larger systems and the subsystems within larger systems.

And then the other thing was really, what are the biggest constraints? And do one at a time. Do not create a massive backlog of constraints that you have to design countermeasures for. Pick the biggest one. It might be your capital expense request. It might be the project funding. Whatever it is, identify that one thing, focus on that, design the countermeasure, move on.

Mark Landy

Yeah. And go end to end as opposed to broad. The IT industry experts are, we're great at starting. We have to start stopping.

William Evans

Then we have a few more challenges. Luckily, the project office reports to Mark, but the thing that we're tackling now is the idea that normally projects come in, resources are identified, resources are assigned to projects.

If we're moving from resource efficiency to flow efficiency, we have to turn that entire model around. Which means that we're going to be pretty busy for the next year negotiating, identifying constraints, running experiments to find if it's possible to create a pull model so that people aren't being assigned to projects, but projects are being pulled by cross-functional dedicated teams.

We're doing this at small scale now, and we're going to continue to spin this up.

One last thing that we wanted to share with you. We've documented all of our references, everything that inspired us for this talk. It's all up on SlideShare. It's going to be available in the app, so you can get it.

We're here for the entire conference, so questions, small tactical, or really big strategy Hoshin Kanri kind of questions, we're available to talk. We love exploring and just kind of riffing on these things.

Mark Landy

And learning too.

William Evans

So please walk up to us, talk to us about it. We've had a great journey. Thank you.

Mark Landy

Yes. Thank you.