DevOps Enterprise Forum Report Outs
The DevOps Enterprise Forum's mission is to bring together technology leaders and facilitate a dialogue that solves problems and overcomes obstacles in the DevOps movement. Our goal is to create guidance on the best-known methods for overcoming these obstacles and to share that guidance with the technology community at large.
Read more: http://itrevolution.com/devops-research
Chapters
Full transcript
The complete talk, organized by section.
Gene Kim
We're going to have what we're calling a forum report-out.
We've talked a lot about the top problems that technology leaders are facing in the DevOps Enterprise community. Do we have slides up? Yes.
I mentioned earlier today that this is being informed by something that almost every speaker has shared, which is, "Here's what we don't know how to do, and here's what we're looking for help with."
Specifically, there are four teams that were created. This year was all about leading change. How do we start? How do we finish? How do we create the coalition to be able to elevate the state of the practice?
Second is, what are the modern architectural practices that every technology leader should know? Because it is not enough for us to know them. If we're trying to influence the state of the practice of thousands of engineers, how do we get them on board as well?
How do we design our organization so that we can actually get the right outcomes? And how do we bridge the information security and compliance gap, not just to prove to them that we're secure, but so we can actually prove to ourselves that it's secure?
I want to just give you a brief peek behind how this guidance was generated. I had mentioned that we invited many of the DevOps Enterprise speakers and subject matter experts in these areas, and we spent two days basically writing guidance.
The goal, I love this phrase from The Phoenix Project: "Messiahs are great, but scripture is better." There's no better, more viral mechanism to spread messages and elevate the state of the practice than written materials.
All of their work is now being put into written guidance that, if you go to this URL, you can actually download. I think two of them are actually ready to be published. The next will be coming soon.
What I'd like in the next 12 minutes is to have the leader of each one of these four teams come out and discuss really two things, three things: what problem did they set out to solve, what did they do, and each of them will be competing for your attention to attend their specific DevOps workshop either today or tomorrow.
First up, we have organizational design, and I think that is Mark Schwartz or Jason. Mark Schwartz.
Mark Schwartz
It is, in fact.
Gene Kim
Mark Schwartz wins.
Mark Schwartz
Thank you, Gene.
Gene Kim
Perfect.
Mark Schwartz
It's interesting. I think we were told that we were all going to come out, occupy our chairs, and then start.
Gene Kim
Oh, is that true? All right. You want to do it that way? Come on out, folks. I don't want to hog the stage. No problem. Courtney, Jim.
So, Mark, you first.
Mark Schwartz
I didn't want to have to do it all myself.
Yes. The conference has just gotten off to a start, a great start. We've just been here for the morning, and already the subject of organizational design has come up a number of times.
That's no wonder, really, because I think we all realize that this is a big challenge we face as we're rolling out DevOps in our organizations.
The traditional way that we structure an IT organization is around functional silos. We might call it something different in each company or each agency. Maybe there's an application development group. Maybe there's an operations group. Maybe there's a QA group, requirements.
One way or another, the traditional way structures us based on functional specialties. With DevOps, of course, we're trying to put together teams that go across the different functional specialties. Clearly, there's an issue for how you do that.
But I think what we found is that it goes much deeper than that. It's not just a matter of, logistically, how do you assemble people into a full-stack team when you're in that sort of organization?
Organizational design embeds a lot of assumptions about the organization and about the relationships between people. The organizational structure determines how authority flows, how communication flows, how work flows, how budget dollars flow, all of these other things. It's not just a matter of the logistics of putting people together.
As DevOps transformational folks, we need to think about whether the organizational structures we have facilitate not just the logistics, but the culture and the appropriate way of thinking about the work and the relationships between people.
What our group did is we talked to people from a number of different companies. We got a feel for how different organizations are thinking about these challenges of organizational structure.
One thing that we noticed is that there are patterns, really, that conceptually you can break this down into four different ways of thinking about how to organize in order to facilitate DevOps.
We set up these four different models that are in an order of increasing, let's say, change from the status quo. But we didn't just show how the models are set up and what their practical implications are on the surface of it. We also tried to think about and research what the implications were for the culture of the organization, the relationships between people, and how they think about what they're doing.
One of our conclusions really is that everybody's still in experimental mode. It's kind of hard to say, "There is a best practice here. We know all the answers."
So we're going to have this Lean Coffee. We're going to have a discussion around it, and we really need your participation, because this is still an evolving area. We need to surface ideas, bounce them around, really beat up on each of the ideas to see if it's really going to work, and try to emerge with better ideas about how to structure the organization.
Please come to our Lean Coffee. Thank you.
Gene Kim
Thank you, Mark Schwartz. The level of thoughtfulness that went into this, in terms of cataloging different archetypal organizational patterns, I thought was just amazing. If you care about organizational design, this might be a good way to spend your time, 1:15 to 2:15.
All right, next up we have information security and compliance, Dr. Topo Pal from Capital One.
Topo Pal
Thank you. This is the second year that the forum met at Portland.
Gene Kim
Because we didn't get it done last year.
Topo Pal
Yes. We presented some high-level approach to how to handle security and audit and compliance into DevOps, because after org design, this is one of the biggest challenges for us.
I'm not a security expert, but in the team, we did have a lot of security experts from different organizations, including audit. I was there because I'm a developer, and I know that for sure the security and audit folks don't like me. So that's the main reason.
Over the last two years, we looked at the high-level approaches, how to handle security audit into DevOps. This year, we actually took one step deeper. We went one step deeper to find out if we can actually provide some solid guidance for the teams to follow in terms of DevOps and continuous delivery and all that.
We basically looked at the DevOps value stream, and we found at a high level, there are four things that we need to look at.
One is secure pipeline. How do you actually take that code commit from the commit stage all the way up to production and meet all the security needs and do not scare off auditors and compliance risk officers? That's number one.
Number two is your classic change control process, where every step has to be manually approved, reviewed, and then blessed by so many people. So that's out of scope for our discussion.
Gene Kim
And the reason it's out of scope is that that's kind of the old way. That's the orthodoxy. You don't really need to meddle too much there, right?
Topo Pal
Right. Exactly.
The third one was the actual production environment, or any environment for that matter. We looked at many models in that production environment, because depending upon what organization structure you have, in terms of your infrastructure, the model will vary.
It could be an IaaS, you host it or the vendor hosts it for you, or a PaaS solution, or just bare metal. So we are looking at all those different models to see how we can actually meet the security and compliance and audit requirements in DevOps, where you have this kind of infrastructure model in-house.
The last part is your production telemetry. We debated back and forth on how much we should cover in that area, and we went from very sophisticated network monitoring to threat monitoring and threat modeling and all that.
But then we figured that irrespective of whether you do DevOps or not, you've got to do those things anyway, because if you don't do those, you're in trouble anyway. So what we concentrated on is how do we detect the threat in terms of configuration changes in the production environment, and how do you actually bring those together in delivering the whole pipeline in just one single pipeline view.
Those are the things we'll be discussing today in our Lean Coffee session. I think it will be interesting, and I really want you to come to that Lean Coffee session to share your thought process, your experience, and your expertise so that we all can learn in a collective way.
Gene Kim
By the way, I learned so much in this session just because I think we created some simplifying mental models in terms of really how to think about the value stream differently than the typical SDLC, and then specifically how do we get to the control objectives, evidence, and so forth. I think this year's the year. We'll have some great guidance.
If you care about information security and compliance, 2:25 might be a great way to spend your time.
By the way, any auditors here? If you're in audit and compliance, I think we would really appreciate your presence in this room. For once. A little joke there.
All right. Next slide. Leading change. Courtney, take it away.
Courtney Kissler
All right. Our forum group focused on leading change, and hopefully this will resonate with everyone.
We decided to really focus in on leadership, because what we found is that usually with DevOps adoption in an enterprise, or really anywhere, it kind of starts as grassroots. So you get a lot of change agents that are really close to the work, and often in order to scale, you really need that executive leadership support and really all of leadership to be supportive of the change.
What we focused on was how can you empathize with those audiences? What do they care about, and how can you do a "what's in it for them"? And then really focused in on some tactics that can be leveraged.
Even though the audience was really leadership, it really could apply to anyone. So don't not come if you think that it wouldn't be targeted at basically anyone in your organization.
Some of the tactics that we'll talk about have already been mentioned in some of the talks today. How do you make work visible? How do you bring in outside perspective to help with the momentum of the change?
So if you are interested in that, and that seems to be the theme across the whole summit, join us. We were all nervous about only having three minutes, and I have two minutes left, and I don't really have much else to say.
Anyways, join us, and we'll go deep on leading change. Thanks.
Gene Kim
Thank you.
By the way, I realize I've done a disservice to you and to the leaders here. Courtney Kissler is the VP of Development at Starbucks. Topo Pal is a technical fellow at Capital One. Mark Schwartz is famous for his 2014 DevOps Enterprise presentation for the work done at U.S. Citizenship and Immigration Services.
All right. Jim Stoneham from New Relic, a longtime friend. You are leading up the last team. We're on modern technical practices.
Jim Stoneham
Thanks, Gene.
So you're staring a big monolithic app in the face, or there's some new mobile thing that's launched that's killing your internal services. What the heck do you do for your modern technology practices to get to the answer?
I had a chance to work with a really impressive group of people at the workshop, or at the forum, excuse me. We first stepped back and said, "Okay, what are the core technical practices that people should be thinking about?"
There's many of them, over a dozen, I believe. Alongside those, what core architectural best practices are there? And then on top of or next to, what are some of the cultural norms you want to aim for?
It became a pretty big, complex problem, as we know. It's not just about your technical practices. It's all these other things around it. We focused on technical practices, but thought about core architectural best practices as well as how do you get better cultural norms, because they're all interrelated.
Then we basically started doing case studies over and over again. Looked at, I think, about eight or nine of them total. How had other successful companies navigated through this set of different practices?
As we laid them out, these practices became like little continents that you'd visit. You'd visit the CI/CD continent, and then you might go over to the test-driven development continent. You might go to the enabling teams to iterate quickly continent.
It started to feel very much like a family vacation. How do you plan this family vacation, this trip and this itinerary, so that everybody's happy, it doesn't explode, and there's no crying kids on the ground, basically? You'll see that theme come up in the workshop tomorrow.
Really what we found when we looked at all these different organizations and their successful traversal of these different continents is there's no one right answer. Big surprise.
Do you start with a new API to enable a team? Do you work on a new kind of rapid deployment pipeline first? Where do you start? The best answer is what's best for you.
But we did try to profile, in the end, five organizations and how they went through that process. The goal is to have those be examples to learn from and see which one might fit best for your environment.
The goal of the workshop actually is to talk with more of you about it. What have you experienced? What kind of successes have you had? What kind of learnings have you had along the way?
If you care about this progression toward more modern technical practices and how to make that a stepwise success for your organization, please come to the workshop. Thank you.
Gene Kim
Awesome.
These are the four workshops that we have scheduled for today and tomorrow. This is the first year we've done it. It's actually competing against other great talks. But if you have specific problems in this area, and I can't imagine that you don't, then I think this is a great way to spend your time, and I'll be looking forward to seeing how that goes.
With that, I think we are now ready to adjourn for a break. Thank you so much for the team leads and the forum teams. I genuinely appreciate it, and catch you in the sessions.
Jim Stoneham
Thanks, Gene.