Log in to watch

Log in or create a free account to watch this video.

Log in
London 2020
Share
Download slides

In Crisis: Danger and Opportunity

With 30 years’ industry experience in leading software teams, Sam acts as the chief customer advocate, responsible for strategy of the next releases of these products, focusing on DevOps, Agile and CI/CD Pipelines.


Sam curates the website, DevOps at Microsoft. He is a regular keynote speaker at many conferences including DevOps Enterprise Summit and Agile. He is the author of four books on software engineering. Prior to joining Microsoft in 2003, Sam was Director of Product Line Strategy at Rational Software Corporation, subsequently IBM.

Chapters

Full transcript

The complete talk, organized by section.

Host Intro (Gene Kim)

I met Sam Guckenheimer when he introduced himself to me at an Agile conference in 2013, and it was one of the most astounding and memorable introductions ever. He basically said in 45 seconds, "I read The Phoenix Project. I really liked it, and I think I know exactly how you put it together. Am I right?"

And he was so right, it was actually a little uncomfortable. Ever since then, he has been one of the biggest influences on me about how I think about DevOps. When he told me how he has been studying how engineering workforces have been impacted by COVID-19, and the incredible data sets that he has been able to analyze, I knew this was a talk that this community needs to see.

In fact, we changed the entire schedule to create time to accommodate these type of mini lectures just to make this talk happen. I'm so excited that Sam can share this data with you because it is relevant to every technology leader. Welcome, Sam.

Sam Guckenheimer

Thank you, Gene. I wanted to also say thank you for this conference and the years you've been running it. It's one of the things I really look forward to every year.

This year is our first year remote, and I wanted to look at this crazy time we're living in through the lens of the Chinese word for crisis, which is written as danger and opportunity. I've used traditional Chinese as a shout-out to Taiwan, a country of 25 million. Tsai Ing-wen, their president, has done a remarkable job holding her death count from COVID down to seven. We should all be watching that.

Now, in this crisis, what are the dangers that we are facing? Well, you know what quarantine feels like now. Back in February, Lancet put together a review study of 400 medical papers around the effects of quarantine. What they found was fairly consistent: long-lasting PTSD, confusion, anger, "Why me?" All of this feeling of disruption that didn't go away.

On top of that, we are seeing from the pandemic a depression such as we haven't seen since the 1930s. We're arguing about how much global GDP will fall this year. The economists are having a field day, and meanwhile, most people are worrying about their jobs. The New York Times cited a recent study from MIT that 43% of the jobs that have been furloughed during the pandemic are not coming back. That has everyone stressed.

Then we see on top of that a great movement for social reform and civic justice. That movement has led to worldwide protests and counter-protests and false flag operations and hooliganism that we've all been coping with. London's been a hotspot, for example, of the distress that may or may not have anything to do with the social issues, but the social issues are here to stay.

History's had us here before. This is the fifth macro cycle of technological revolutions. Mik Kersten at the last summit borrowed from the work of Carlota Perez, she'll be here to talk with us tomorrow, about the progress of technological revolutions. They go through these inflection points. They go from periods of invention that are fueled by financial capital, to deployment that are fueled by production capital. In the middle, there is a disruption, consistently, where we go from rising income inequality to rising income equality, where we see great social change and suitable regulation to allow us to enter the new era.

I don't think anyone in this audience would dispute that we right now see a sewer in social media that needs suddenly to be treated more like broadcast regulation when it came on under the FCC in the '30s. I don't think anyone would dispute that we need to see many of the reforms similar to what we saw in the 1930s under the New Deal.

So we're in this historic turning point, and not surprisingly, in the middle of this historic turning point, we have a leadership crisis. We've been talking about what's transformational leadership, what makes a great CEO, talking about all these things for a decade. What we see is that that transformation, that digital transformation, it wasn't the great CEO. It was the health crisis. That's what's brought us here.

Now let's talk about the opportunity that we have in the middle of this crisis. The first good news is that the internet and cloud have scaled. Microsoft Teams, for example, in three weeks of March, saw its video meeting minutes increase 10,000% with hardly a blip. All of the services have been scaling like crazy. The cloud has been scaling like crazy, and it's basically worked. That is, in itself, astonishing.

We've also seen the financial markets recognize that this change we're seeing, with all of us remote, is very friendly to the technological change we've seen toward DevOps, and it's showing up in the market valuation of companies. This is from Citigroup's June Industry Monthly Review for software. This is an investor-targeted newsletter, and they break out sectors. You can see that DevOps over here in green has just run away from the pack.

If we look at the data from the other side, from us practitioners, we see that the volume of code being written by developers is increasing. This is data from GitHub. We've looked at the year-over-year comparison of code pushes, that is 2020 compared to 2019, and we see that since COVID lockdown in March, it's gone up dramatically, and these results, beyond the published ones, have been continuing.

We see that there's no COVID effect in the amount of changes that are coming up through pull requests. So pull request activity has not been adversely affected by COVID. We see, however, a fascinating thing, that the responses to pull requests, the approval time to actually merge those changes, is going down. So cycle time has gotten shorter as we've gone remote. Fascinatingly, we see that open source contributions are climbing radically during this same period of time. I'll dive deeper into all of these statistics in my breakout tomorrow.

Meanwhile, we're starting to read in places like The New York Times that this working from home stuff might be here to stay. The Times Magazine earlier this month put together this great survey talking to folks like Erik Brynjolfsson at MIT and Nick Bloom at Stanford, and many, many companies and U.S. government agencies like the patent office, who've been finding really positive results on working from home.

The interest has been fascinating. If we look at what has happened in the press, we had two tiny blips around work from home over the last 10 years. We had one when Marissa Mayer told all the Yahoo employees, "No, no, no, no, no, no, no, you can't work from home anymore because we don't trust you," which was perhaps a reflection of the employees saying, "We don't trust you, Marissa." Then we saw a little bit of a blip when two of our senators, McConnell and Rand Paul, had to work from home because of physical conditions. That was down here. Then came COVID, and all of a sudden, we see 120x growth in interest in working from home.

The Federal Reserve Bank of Atlanta did a study in May about how employees and employers feel about this. What they found is that the employees, post-pandemic, want to keep working from home 40% more. Employers, they're thinking about half of that. So they're leaning that direction. I think the employees are going to win, and the pendulum's going to swing strongly toward much more work from home for them.

Even more fascinating, earlier in June, MIT published a research paper in which they looked at financial performance of companies relative to the extent to which they had embraced working from home. What's interesting about this is that they found higher stock returns, lower volatility, better financial performance, primarily among non-high tech. This is totally surprising to me. I thought, well, we'll see a few of those high tech unicorns, like I work for Microsoft, we'll see companies like us do much better, but a lot of people are really going to have a hard time. Turns out that the benefits are quite widely spread.

So how do you be successful in this world? First thing you need to do as a DevOps leader is to kill the zombies that are trying to hold you back to the status quo. You need to look at your portfolio and ruthlessly prioritize. Invest in the products that are going to make you succeed. Divest or kill the ones that are holding you back.

That means let go of the old infrastructure. If it's costing you too much to maintain, if it's taking too much effort, if it requires too many people like Brent in The Phoenix Project, get rid of it. Take it out of the data center and give it away, because it's holding you back. The public cloud is here. It's available. It's been working.

Then take a look at your technical debt. If you are spending your time coping with technical debt, and this is keeping you from releasing frequently, strangle the technical debt before it strangles you.

Fourth, take a look at your zombie processes. How many meetings and approvals and reviews do you have that you don't need? And why do you still have them? Kill them before they kill you.

Then look at all the customization you've done to support those processes over the years. You took this great packaged software, and then you customized it 20 ways to breakfast. Do you need those? If not, get rid of them. Best of all, if you can replace on-prem versions of that with SaaS from the cloud, modern services that update themselves, do it.

A great way of thinking about this was expressed pre-pandemic by our icon, Warren Buffett. The difference between successful people and really successful people is that really successful people say no to almost everything. Say no more. This is your chance.

Then take the habits of highly successful product teams and make them your own. Optimize for the asynchronous workflows. You don't need to meet for everything. You can use pull requests for changes. You can use chat. You can use the technology to let you work without being disturbed.

At the same time, connect with your community continuously so you have a two-way dialogue that's going on with your stakeholders, with your users. Focus on the outcomes and focus the team on the outcomes, not the outputs, so everyone knows why they're doing what they are.

If you're running a service, practice mature incident response. I hope you went to Erica Morrison's talk. I hope you've heard Scott Prugh from CSG. Take a look at their experiences moving their incident response forward. That's exactly what we need to have.

And then for that technical debt, get rid of it. Get clean, and then put in automation to stay clean so it doesn't come back. With that automation, shift your quality left and shift it right so you have observability in production, so that you're not dependent on humans to tell you what machines are doing.

A great performance metric that you can think about improving is how many manual approvals do you need to get from an idea to deploy, and how much time between those manual approvals? I heard a great story of a bank that went from 22 months for a normal approval cycle to a couple days when it came time to implement the Paycheck Protection Program under COVID.

After the crisis, there's going to be a new normal. There will be laggards who will blame the pandemic, who will say, "We couldn't rock the boat." They'll want to restore the status quo ante. They'll be held back by all the zombies. There will be thrivers who will hit refresh. They're going to stop doing what they don't need to do. They're going to focus on what they do need to do. They'll value learning over prediction. They'll get stronger under stress. They'll invest in their core that differentiates them. They'll divest in the context and use SaaS so that it's not in the way. In order to make that core more successful, they're going to embrace DevOps for agility.

They will be working in remote and hybrid settings like we are now, and they will be adapting continuously.

I've got a breakout tomorrow, which I'm going to dive deeper into the data. Please come.

If you follow my Twitter account, I'll be announcing a remote engineering summit that we from Microsoft will be hosting. We'll also have an academic get-together at the beginning of August to collect more of the best research from the community.

So thank you, Gene, for the opportunity to share this with everyone.