Host Intro (Gene Kim)

Good afternoon. Welcome to the last talks here at DevOps Enterprise.

And heads up to the backstage crew: Damon Edwards is next. You might have the wrong ones up. Okay.

So to introduce the next speaker, Damon Edwards, I would like to introduce him like this. Over the years, many people ask me, "Gene, how did you get into DevOps?" And so you can imagine I'll often give an answer that sounds like it was just a natural continuation of my 20-year journey studying high-performing technology organizations, which drew me into the DevOps movement.

But if you kept asking me, "No, no, exactly how exactly did you stumble into the DevOps movement?" eventually you would hear me say, "Well, in 2010, I was at Tripwire at the time. I got this email out of the blue from someone I never heard of, inviting me to be on a panel on a topic that I'd never heard of."

But I did go to it, and I was blown away by what I saw. And what I saw was a bunch of amazing mavericks that were at the epicenter of the DevOps movement, including Patrick Debois, John Allspaw, John Willis, Andrew Shafer, and many more.

That event was DevOpsDays 2010, the first DevOpsDays in the U.S. And it was really the inflection point that changed DevOps from an event to a movement.

The person who emailed me was Damon Edwards, who was one of the conference organizers along with John Willis and Andrew Shafer.

Damon Edwards has been on the program committee of this conference ever since. And like Patrick Debois, it should be no surprise that Damon is out on the frontier again, trying to find all these genuine problems as we try to do new things.

So, in 2010, Damon was very early to the DevOps party, and now he's very much at the heart of the generative AI party, trying to figure out what does it take to use generative AI to achieve better organizational outcomes.

Here's Damon.

Damon Edwards

Come together.

Hello, everybody. Thank you for being here. I'm happy to be here.

If you drive through the southern part of San Francisco, there's this stretch of highway. It's pretty much a crossroads, we can call it, where it connects Silicon Valley to San Francisco and then out to the Bay Bridge, which I guess takes you to the rest of the country. And this short stretch of highway is famous for having all these tech billboards. It's where the famous Yahoo billboard was and whatnot.

And I felt like this was a good metaphor, as the crossroads of my life are sort of represented by this billboard.

Going way back, I used to be the managing director of a DevOps consulting firm. Many of you are our customers. Thank you very much.

And then with Alex Honor and Greg Schueler, I founded an automation tooling company called Rundeck. Some of you were big supporters of that. Thank you very much again.

And we ended up selling that nice business to PagerDuty, the owners of this billboard here. And after the integration period, I got to, I was asked by the Chief Product Officer and also head of engineering, Sean Scott, said, "Hey," he wanted me to do some cross-portfolio strategy work.

The first thing I worked on was, I think, kind of a lot of the culmination of the DevOps knowledge that I've gained from all of you over the years. And that was the Operations Cloud.

There's tons of stuff that you probably don't even know PagerDuty does, from AIOps event management to automation to customer service operations tie-ins, and bring that all together into one platform.

And in the course of that, about a year ago, generative AI burst onto the scene. So that kind of fell under my purview as well. Now we do it from the teams, but trying to tie all that together and further operationalizing it.

So when Gene asked me to speak today, it was about that journey that I've been on. What have I learned about it? And also, doing that, one thing I learned from our industry here is get out and talk to people. And people are way more helpful than you'd ever expect.

So, a ton of conversations with other folks in the industry like me who were working on generative AI. And this talk is kind of just, a year later, what I see is going on.

So if you ask colleagues around the industry, "Why is this big bet on AI?" it kind of always came back to me with this same answer, which is, "Because we can't afford to be wrong."

And it's like, "Well, you know, it's like you're just missing a trend."

They're like, "No, we literally cannot afford to be wrong."

And so when you kind of dig in why, you sort of realize there is, what is a good meta way to describe what people's both excitement and fear is? And it kind of brought it down to this, which is, if you think about this gray circle here, it's a representation of all the compute power and all of the, call it, kinetic potential of our data that exists in the world.

How much of this are we actually using?

And then you can argue that we're using a very small sliver of it. And why? Because how can you access it? The computers actually are amazing gatekeepers. They tell us we've got to learn these esoteric programming languages. We have to figure out these finicky APIs. We have to think in different abstractions.

And there's very few people that can do that. It's hard work. It's really, really expensive. So we're only using a small part of what we could.

But suddenly, what if there's this linguistic layer that appears? It has these surprising abilities of inference and seems to follow directions. Well, now we're in charge of the computers. We can actually use our natural language to tell them what we want them to do.

So this represents a huge shift in the power dynamics of work. So that's one of the first, oh, wow moments that really hit: how big can this possibly be?

But that's kind of pie-in-the-sky thinking. Let's bring it back down to reality.

I divide technology work in two buckets. There's the external product focus, like what are we doing for customers? And then there's the internal efficiency, operational side of technology.

So first, let's start with what can we do with this? What can we do with the external product focus?

Everyone will give you all kinds of use cases, but there's one recurrent theme that I found over and over again, was that it allows us to revisit all the ideas that previously we thought were too hard or too costly. These are the things that had the inputs or the user intentions were just too variable, too unpredictable, or the underlying data that we're working on had too much variability or lack of structure: a lot of natural languages and messages, and these things.

We looked at them and we're like, "Well, that's just too hard." So it just gets crossed off the list automatically.

But suddenly we're realizing, hey, by applying this new technology, and by the way, I'm going to use generative AI and AI interchangeably here. But by applying this new AI technology, we're now able to start to attack those problems at a fraction of the cost of what we previously expected.

In fact, even at PagerDuty, there were things across the whole kind of event. An event comes into driving resolution. There were things across there that just seemed too hard, too expensive. And now even looking at internally in our labs, we've got it working. And this is talking about months, not projecting years that it would've taken to do some of these things.

And so, from a product focus, if we're able to do the things that previously we thought cost too much or were too expensive to do, if we can do it right, we're now gaining a structural advantage over our competitors who can't, or won't, or haven't gotten around to it.

So that's a common theme among the external product focus folks.

Now, the internal focus: this is what kind of surprising was that I found the fear/excitement was far greater from folks who were thinking about internal operations, more so than my product colleagues.

And really, it all came down to one thing, which was margins, especially at public companies. It's business 101. Margins being, take the revenue that you generate versus how much it costs you to generate it. Put that together, you've got your margins.

And when your margins are big, you've got all kinds of options. You can increase cash flow. You can reinvest in parts of the business. You can buy back stock. You can hire talent. There's all kinds of things you can do, and you'll be rewarded handsomely by the markets.

But if that margin shrinks, your options shrink, and your valuation is going to get a beating.

So bringing it back to why margins, why generative AI? Because they saw it as an opportunity to change the game for the biggest cost in their business, which is their people, and the most strategic cost of their business.

So they're looking at their people, saying, "Look at all the inefficiency in what they have to do: the repetitive toil, the communication ceremonies they go through, the status reports, the TPS reports, everything on there." They're like, "Well, how much of that is not adding value to our business?"

Previously, this was just too hard to automate. It was missing that linguistic layer that understood what people are saying, because so much of business is documents with data and narrative combined. And it's very hard to automate.

So looking at today's tasks, they're saying, "Well, hey, you hear these quotes already of 30% improvement to 60% improvement, depending on what somebody's job is doing." But looking at, we can use these technologies to improve tasks that improve our people.

But they look down the road, they're saying, "But tomorrow we can replace whole roles, or give everybody their own scheduling assistant, their own army of coders, a financial analyst, whatever subject matter expert you need." You can have dozens or hundreds of these roles that will help you drive.

So now we're talking orders of magnitude. Is it 300% improvement? 3,000% improvement?

Suddenly the realization happens that we can have this massive impact on the margins of our business and hire better people, and through all things we're talking about. And that gives us a structural advantage.

And so, if we have this large enough structural advantage, it turns into the classic unfair business advantage. And us being here in Vegas, it's like the idea we're sitting at the poker table. You've got all these chips. You can lean on everybody else until they fall over.

So the possibility of being on one side or the other side of this unfair business advantage, if this happens, you've got to move, and you've got to move now.

But if you want to think seriously, why the urgency?

Well, if you think about the web, it was cost additive. We added a bunch of people, added a bunch of computers, and there was some future strategic benefit that we would get.

Mobile, same thing: future strategic benefit.

This, the problem is, it's massively cost deflationary. So that means there's an immediate strategic benefit that you get out of this.

So when one company in a sector figures it out, everybody's going to have to do it, because you'll be at the structural disadvantage to everyone else.

So I think the big message is someone's going to lead in your industry, your sector. Why not your company?

So the big imperative is get your company moving. Everybody I talk to, it's like that's the thing that they have to figure out how to do.

So the rest is just some observations that I've had.

You'll hear these things out on the web, the Twittersphere, that AI committees need an AI committee. The problem is, I haven't talked to anybody that's actually worked for. Why? It's a committee. We all know how that goes. People have a day job. They show up for the meetings. And you get to the good part, and suddenly you have to adjourn the meeting.

Who do you put on the committee? We found out that the old experts are not the new experts. People we thought would be great at this, actually turns out other people are.

Weak execution ownership: that's another issue. Everybody has another chain of reporting and budgets and V2MOMs and OKRs or whatever you use. So how do we get from the committee back to that?

And so I find, how do we get companies into this discovery mode, where we can find those unknowns, find the people who are closest to the problems, empower them and get the most out of this, and make it everyone's work, everyone's job?

And it really helps challenge that assumption that, oh, somebody else is going to figure this out for us.

So two modes I see people in. One is we're going to anoint specific teams. There's pros of that. You pick hopefully the easy wins. You can focus and swarm resources. You can keep the rest of the organization on plan.

The cons: only a few teams get moving. Maybe the new stars and best use cases are somewhere else. Now you're just delaying uncovering them. And that osmosis is really slow because you're telegraphing, "Don't worry, this special team's going to figure this out for the rest of us."

Option B, I've seen work very well, is the company-wide prototyping sprints or hackathons. It gets everybody going.

Now, I get it. I realize in a large enterprise, you can't get the whole company going at once, but you get the idea. New ideas and new stars emerge from within. It always surprises, and sends a bold message.

The cons: finance and your operations folks, in the COO sense, are going to freak out. Like, how are we going to do this? How are we going to delay our plans? And we've got to get people over that fear of getting their hands dirty.

So if you're going to go with option B, a few things I've noticed. Don't let it devolve into a tangent of free-for-alls. It helps to pick a stack for people. Encourage people to learn that stack in advance.

For example, you can say here, you can read this. This is stuff that Patrick and Joseph and John Willis and other folks have been talking about. But you pick a stack of things, narrow down people's choices.

Also be aware of the vendor trap, which is kind of this paradox: you want to go faster, so let's bring in the experts. But then you're kind of avoiding getting your people to learn. And in this new world, they need to learn and see where the opportunities are.

And very key: you're operationalizing AI, not building AI. This is kind of a current idea that people want to fall into. "Oh, let's train a model." At that point, I'd say just stop the meeting, because it's ridiculous, the amount of effort and cost into that.

Fine-tuning a model seems exciting. Be very, very skeptical.

Get people focusing on prompt engineering, chaining APIs. That's where you prove it. And you'd be amazed at what you can get out of this stuff.

And there's a surprise consensus, right? This is actually, it's not super easy, but it's easier than everybody expected.

Joseph, here on the stage before, he did this, where he asked Perplexity AI to go through and analyze the "We Have No Moat" paper or memo. And then in a day and a half, he took some Python code. He used ChatGPT to help him with the Python code. He used LangChain and the GPT API and basically created the same thing.

So on the left is a company with tens of millions in funding and dozens or hundreds of employees, and Joseph, a non-developer, hacked his way through it, figured it out, and came within shooting distance.

So not to say vendors don't have their place, but it's the idea of, let's let your organization learn. Don't hire a Razorfish, for those of you who remember that, to build your website for you. Do it yourself. And you'd be surprised how easy it is.

So what to expect. There's table stakes today. Do it now. Coding copilots. I didn't want to talk about that. This should be everyone using that.

Summarization, synthesizing, manipulating a bunch of documents or content to create new output, generating code. The whole chatbot, chat with your documents/data. That's easy to do today, and everybody should be doing it.

Still emerging is the multimodal. That's getting to be a lot harder.

And really not ready for prime time is the idea of true open-ended, goal-oriented agents. I think this will come. There's a lot of interesting initial things about it. Some people think LLMs aren't really going to be the full piece of it.

But what's important to know is this stuff you can do today. This is where you're in that performing tasks for people. Good dollar savings there.

When we can unlock that agents, which is where the intensity and heat is going, that's where it can perform whole roles. And that's where we really get the best margin improvements.

Don't put unreal expectations on your data science folks. Most teams, they've never shipped and operated products before. Don't suddenly expect them to. DevOps is as foreign to them as AI is to us. They're as much as scared of you as you are of them. And they're way outnumbered. Everyone wants their time.

A promising marriage is now we know what to do with platform engineering. Marry them up with data science. They are the AI experts plus the delivery experts. You can build that dial tone that the rest of the organization can focus on, that higher level, and keep everybody from falling into the model level.

Shadow AI is going to happen. It's going to happen everywhere. So really try to harness it. My colleague and friend John Willis has talked a lot about this, saying, look that up.

One more key thing here: who's going to be your enthusiasts and detractors?

J. Paul Reed had this famous thing back in the DevOps world and said, hey, dev always sees the world as very deterministic, and non-determinism is a bug. We're going to root it out. It either compiles or it doesn't. It runs or it doesn't.

Where operations, non-determinism is a reality.

And I've kind of noticed that the same mindset, coming on where someone's coming from, is a major issue here. Because generative AI, it's a lot of fiddling.

Here's that retrieval-augmented generation pattern people are talking about. You fiddle with the queries. You fiddle with the embeddings. You fiddle with the system prompts. Then you fiddle with the settings of the model. And then success: nobody touch anything.

It's operations work. It's a lot of fiddling.

So quality's going to look a lot different. Seventy years of trying to root out non-determinism. We've got to rethink that now. It's deterministic by design.

Thumbs up, down testing. Thumbs up and down monitoring is not going to cut it. Development's going to be a lot of, well, got it good enough. And then we've got to get into more of a process control evaluation type scenario.

And that, I think, is where companies are needing to spend a lot of their time thinking about how are we going to do this, and a lot of the investment. Because you can't outsource another black box to another company, because now you're adding a black box to manage a black box. It's never really going to work.

UX is going to be totally different. How do we handle conversations?

And beware of prematurely optimizing for cost. I've seen companies fall into this because you can measure it. You're going to need a lot of tokens for these conversations. You need a lot of tokens for the control and evaluation of those outputs. And prematurely worrying about token usage is going to stunt your organization's discovery and adoption.

And you just can't afford to do that right now. And the trends are showing that there's going to be super hyper-competitiveness between the different foundational models. You've got to believe the price is going to be dropping next year, and the year after from there.

So how I can use your help: love to talk.

Together with John Willis and Patrick Debois and Joseph and some other folks, we're starting this new community effort. It's not a commercial thing. It's a community thing to document and share these patterns, to work with folks that want to share and bring what we're learning to them and vice versa.

We'll call it like a little research group. We already have a domain name, so we're halfway there. But you can email anytime: damon@operationalizing.ai.

Love to talk about this stuff, and let's learn together.

So thank you very much for your time. Appreciate it.