Host Intro (Damon Edwards and John Willis)

Damon Edwards: John, it's day three. How's it going?

John Willis: Yeah, good. Great conference, right?

Damon Edwards: I feel like we're in the home stretch here.

John Willis: Good.

Damon Edwards: So if you all have enjoyed the show thus far, as I said, John and I are reprising our, people tell us it's quite popular, DevOps Cafe podcast. So if you go to devopscafe.org, you can sign up for notifications about the next series, and I'm sure you'll see a lot of these DevOps Enterprise Summit speakers going in-depth with us. So John, our last pair of lightning talks, who have we got lined up today?

John Willis: Oh, we got the great James Wickett, a great community member just all around DevOps, Austin DevOps, runs the Austin DevOps program down there. And this Botchagaloop character, I don't know.

Damon Edwards: We don't know much about this guy, but we'll see how it goes.

John Willis: Botcha who? Botchagaloop.

Damon Edwards: Botchagaloop, yeah.

John Willis: I don't know who that guy is, but you know what? We'll give anybody a chance. So, hopefully he comes through.

Damon Edwards: See how it goes. Here's the last pair, and thank you all very much for watching all of our lightning talks. Thank you.

John Willis: Yeah. Bye bye, everybody.

Damon Edwards: Bye.

James Wickett

The lessons we learn in complex systems today are not too far removed from the lessons that we can learn in the past, and my hope is that in this brief talk, we uncover four lessons of chaos.

Hi, my name is James Wickett. I'm the head of research over at Verica. I'm an instructor at LinkedIn Learning and an organizer at DevOpsDays Austin and DevSecOpsDays Austin.

Verica, where I work, we're building chaos engineering and continuous verification tools that are built out of our experiences at places like Netflix and other large enterprises.

Well, let me tell you about the 1890s and this fast-growing, cutting-edge technology company, the Missouri-Kansas-Texas Railway, pronounced the Katy. The railway had a lock on transportation in the region, but they needed more sales. Well, enter William Crush. He was a ticket sales agent and a member of the digital transformation team, I'm sure.

Crush had this idea on how to get more people to ride the train: simply create an event where everybody wanted to go to. Well, train demolition events where you crash two trains together had started to pop up around the country, but they weren't exactly successful yet. Crush's idea was to make the event free; just make it in a spot where you had to buy a train ticket to get to it.

Well, it turns out it was a success. Crush crushed it, if you will. Sorry for the pun. This freemium model worked, and really the ticket sales started to soar. In fact, they sold 40,000 -- 40,000 people showed up for the event. They had to build a town for a day called Crush, Texas. It was on the outskirts of Waco, where the hills make a natural amphitheater. The town had 200 law officers, a jail. Let's see. Crush built water wells. There was entertainment while waiting. There was a midway. Everybody showed up.

Well, because of this, Crush also started unlocking Google AdWords back in the 1890s and sold advertisements on the side of the trains, and here's a photo op that they took of the trains before the big crash.

Well, one major problem of all of this was that boilers on trains of the day would just randomly explode. Here's a picture from the 1850s of a train that did just that while just in normal operations, much less crashing them together.

Crush was concerned. He laid extra track. He kept the crowds back. He took every precaution. Well, the day and the time came, and the rumble of the two trains started, faint and far off at first, but growing nearer and more distinct with each fleeting second. It was like the gathering force of a cyclone, said people watching.

The trains collided at a combined speed of 90 to 120 miles an hour, and this incredible picture captures the moment of impact. Think about the physics going on in this exact moment.

But you might have guessed that there was an unmitigated vulnerability in the system. The boilers, they had been certified to meet PCI compliance. Oh, wait, that's the wrong talk. They were validated by engineers through visual inspection. But unlike our software systems, they could not really be verified. There was no experimentation or verification, and approximately one second after impact, both boilers exploded.

Steam, iron, wood filled the sky. People compared it to battlefield experiences. As you might have guessed, there were a lot of sustained injuries throughout the crowd. Four people died, and even after the event, people swarmed the explosion to get a souvenir, a memento. That added more injuries due to the burns.

So what do we learn from this story? Well, one is that breaking things in production sounds good, but it isn't. Sometimes we do learn from production, but it's much better to learn about how our systems will fail much earlier before production.

The next thing we learn is investigation can't stop at human error, and Crush was immediately fired after this happened. But a few days later, the Katy realized that he had taken all necessary precautions, and the accident wasn't limited to just his fault. Our modern-day prophets like Allspaw and Dekker would be proud.

Another learning here is that we need to know our system safety margin. All of our systems have a breaking point. We need to know when we're close to that boundary.

The last thing that we can learn here is that verification beats validation every time. This means actually experimenting with what we have and verifying it versus just validating a configuration or passing audit or somehow visually inspecting something.

So those are my four lessons learned: breaking things in production sounds great but isn't; investigation can't stop at human error; know your system safety margins; and verification beats validation every time.

If you like this story, or you're more interested in chaos engineering, I would like to send you a free copy of this book that Casey Rosenthal and Nora Jones just wrote. Simply go to verica.io/book, and we'll get that taken care of for you.

Hope you have a good day. Thanks.

John Willis

Hello, I'm John Willis. This is called "Deming to DevOps, a very short history." I've been obsessed with Deming from way back to 2012, DevOpsDays. We had open spaces where Ben Rockwood suggested that everything went back to Deming, and I've been sort of obsessed with that ever since.

Last October, I went to work for Red Hat with Andrew Clay Shafer, Kevin Behr, Jay Bloom. Like Andrew likes to say is we wrote some books, but we really are looking at transformation over the next 10 years to see how we can apply this.

There's a head fake here that it really goes back to Darwin, not Deming, because Darwin really puts a stake in the ground in the heart of determinism versus non-determinism in "Origin of Species," really sets the stage for 19th-century thinking.

And then Ludwig Boltzmann is interested in Darwin's work of biology, so he decides to apply it to properties in gases. He becomes the father of statistical mechanics. Statistics and physics now are melded.

And then Max Planck then takes the work from Boltzmann, and he sort of accidentally discovers quantum physics. He helps to measure the unmeasurable, something called the Planck length or the Planck constant.

And then you have Albert Einstein, who uses Planck constant to solve a problem. Einstein wins a Nobel Prize for photoelectric effect. Quantum physics is afoot. The 19th century has this non-determinism, not exact, but probable way of thinking.

And then at the same time, there's a gentleman over at Western Electric who's worried about the quality of manufacturing of phones. He applies statistics to manufacturing, non-determinism way to create. He basically becomes the father of operations management.

And then we have our protagonist, Dr. Deming. Deming is sort of a study or a student of Shewhart, and he promotes Shewhart's ideas. Deming has a long story about going over World War II, becoming really the Shakespeare of quality in Japan. Long story.

And one of those people, Taiichi Ohno, is indirectly, some people say influenced by Deming's work, is the founder of Toyota Production System, which we know becomes lean. Interesting enough, the just-in-time was actually discovered at a Piggly Wiggly by Taiichi Ohno.

And then we have Eli Goldratt, for those of you who know, "The Phoenix Project" is a modern-day rewrite of "The Goal." This is the theory of constraints, bottlenecks, and heavily influences, certainly "The Phoenix Project" and everything that's come in IT Revolution, of course, DOES.

And then we have John, which is, he actually is the one who coined the term lean. His first job was actually at NUMMI, the GM and Toyota joint venture. He's actually now the CEO of Google's self-driving car project, interestingly enough.

And then we come back to Deming again, because actually, Deming, starting around '83, writes a book called "Out of Crisis," his 14 points, and over a 10-year run into his second book, "The New Economics," he has this major influence in America thinking in quality.

And then I can't stress enough the importance of Mary and Tom Poppendieck's "Lean Software Development." They're the first people to apply lean manufacturing to a software thing. And I think in a lot of ways, it's the genesis of sort of lean IT, Agile.

And then David J. Anderson also, he's reading "The Goal" on a plane ride. He's managing a large development team, and he comes up with this idea for Kanban for software based on reading "The Goal." It's in one of his books and his stories there. So again, that's really, really interesting.

And then this is another incredible thing. 2006, Jez Humble, Chris Read, and Dan North are showing at an Agile conference, 2006, a pipeline. It's five years later before people even start getting on the radar, 10 years before it's table stakes. Just amazing.

And then, of course, we have the godfather. In 2008, him and Andrew Clay Shafer have a conversation about Agile. I get invited in 2009 by Patrick to be the only American for his DevOpsDays. He coined the term. He's been an excellent guardian and advisor.

And then we have Eric Ries. Eric Ries, interestingly enough, is working in Silicon Valley on a startup. He writes a blog, eventually becomes "The Lean Startup" book. And in the book, he actually does a tribute to Deming and a lot of advisory for a lot of the startups in Silicon Valley.

And then, of course, John Allspaw. It would be a risk not to mention his 2009 O'Reilly, "10 Deploys a Day" at Flickr. I was there, and I remember it seemed like people were like, "You can't do this. That's impossible. You can't deploy."

And then, of course, Gene Kim. "The Phoenix Project" has been a stake in the ground for enterprise and DevOps. Everything we have here, DOES, his leadership and his tenacity to create what we have now, it's just been incredible.

And then ending up with Nicole. Nicole has done an incredible job putting the science behind DevOps: the first DevOps Shingo Award winner, her work with DORA, the book "Accelerate," and then academically peer review of DevOps.

Closing Announcements

Hey, not a surprise, another great set of keynote talks to get us started for day three of the DevOps Enterprise Summit.

We're heading into the break, so just a few things to know before you go. The breakout talks are starting at 11:20 a.m. London time. We have the networking time and the networking opportunities starting at 2:50 p.m. London time. Remember, we've got birds of a feather and chat roulette, no lean coffee today.

And then we're back here for the keynote talks again at 4:30 p.m. London time. This is your last opportunity to engage with our sponsors in the virtual expo hall. Go talk to them and find out how they can help you on your DevOps journey. This is also the last opportunity to play the games. They're fun, and you can win stuff.

So, with that, have fun, learn lots, and we'll see you back here a little later in the day.