Hey, everybody. Uh, welcome back. We're, uh, still here at the, uh, DevOps Enterprise Summit. I am, uh, Damon Edwards from, uh, PagerDuty.

And I'm Jason Cox. And I'm Topo Pal from Fidelity Investments. So, uh, the topic at hand that we're gonna talk about is, uh, generative ai, right? How can anybody talk about any other topic?

It's a topic of the, uh, it's the, uh, topic of the decade, right? Um, so, you know, I think why we were getting together to talk about is, you know, looking at sort of your perspectives from, you know, operationalizing this kind of technology, right? And I think, you know, for years we've talked about the promise of ai, and then suddenly like, oh, my, holy moly, this stuff actually, actually useful, right? Something's actually actually happening here.

And, um, you know, we've gone for, I think you see the handwriting on the wall, right? Like we've, you know, the idea that, you know, human beings are the ones that write the code. Human beings are the ones that debug and, and, and, and, you know, operate the code. And I'm saying code there very, you know, very purposefully, right?

Um, you know, that we were the tools that did that, but now we realize, oh, wait, you know, these tools will probably be doing that, right? Whether that's, you know, one year, two years, 10 years, I think it's pretty clear that that's gonna be happening. So that means we're gonna be operating the robots that write the code and run the code, right? So, you know, I'm just kinda curious to, I think, start with, with that.

Um, and, uh, you know, maybe start with what was the first kind of, uh, mind expanding, eye-opening moment where you're like, oh, this is different. Like, this is gonna change how we do this work of writing and running software? Yeah. Yeah.

I, I'll say this, right? As you start to look at, um, these, um, generative AI models and you see the large language models actually starting to get adopted, it feels like the tectonic shift, uh, that we saw during the internet right there. It's something that's gonna disrupt so many different areas. And our space in software we can only predict.

I think what we're doing is saying this is what it's gonna look like to be running these, uh, software systems in the future, but we don't know, right? I, I think there's enough unknown here that five years from now could be dramatically different. I think what we're seeing today in 48 hours changes dramatically. And I think that's part of the excitement.

And also the challenge before is, is that the acceleration curve years is beyond anything I've ever seen, right? In terms of technology. And so, when you see the capability of today's AI able to today code, and it could be infrastructure code, like your, to your point, using code very specifically, it could be infrastructure, terraform, you can paste it in there, have it develop on that. You can have it, review it, it will tell you exactly what it's gonna do.

You can have it even ride it from scratch by giving it some prompt, right? Same type of thing for, um, any of the application code. Doesn't matter what you're coding in Python, no, doesn't matter. It will help generate that.

Give you the, um, what's going there, perfect it, summarize it, whatever you're looking for, right? As a developer tool or as a, in that, and developer in the broad sense, you could be managing infrastructure, you could be doing all this. You have the ability now, like we had at the very beginning when search came out to be able to access internet and co copy paste. Now you have this tool to 10 x your workload by using these tools to, to get to where you want to go.

Now the question is what comes next? And I think that's, that's really the challenge before us, is how do we stay ahead of this thing? How do we stay ahead of this curve? Not just because we need to be, uh, you know, relevant.

I'm not saying that, I'm saying that it could become the mountain that buries us in the sense that there's gonna be more work for us than we can possibly do if we're not in front of it to help, uh, guide it along the way. So I think it's gonna be a challenge for us. Topo, you think? Yeah.

Was there like a moment something you, for you that, that went off in your head, or, or what Jason said? Is there like a way you're seeing this? Uh, kind of, I agree with you and, and, and in my mind, I'm thinking that, uh, I think what we are going to see is, is an increase in speed and volume, speed and volume of code that are getting into the so-called pipeline. Pipeline as in feature ideas all the way to, uh, production in customer's hand, uh, and devs is all about increasing speed and volume.

Now, if you look at how can you know better than anybody else in the, in the, in the world how DevOps came about, it was because there was a constraint put on the system, by the way, of delivery software. And this is another set of new pressure coming into the system that will put constraints on our pipeline. And when I talk about pipeline, it includes human being also. Yeah.

So it'll put more constraints on the system to be able to deliver the amount of code and the features that are being developed by these generative ai. So that's a very interesting point, is like, you know, in the DevOps now, you know, I, I kind of, I think we always sort of didn't like the word, but I understand it's, it's a label. It's useful that, you know, DevOps tools, right? It's like all these tools are DevOps tools and DevOps languages, and, you know, it's like everything.

It's like, but it's all, they're all written for the old way, which is the humans are the ones that produce the code. The humans are the ones that find the problems in the code. So it's all made for humans, right? So, I know we have no answers to this.

This is not a question you have an answer for, but any thoughts around, like, h how do you like, or can even, can we even get our minds around, like how the landscape is gonna change, how our view of the, like the example I heard today was like functions, right? This is a great one. Mm-hmm. It's like if computers are always writing the code with, you've ever developed functions as a programming construct mm-hmm.

Like functions there, because humans can't hold stuff in their head enough, and you can't type everything out. So you just gotta remember the name of something, how to call it, right? Write it once, perfect. It call it a lot of times.

But if you were, you know, like a machine writing this, you have per perfectly scalable memory and access to whatever, like, would you ever wasted time coming up with functions? Or would you just sort of do copy and paste a thousand times? If you didn't care about the size of your code base, would you break it up into different files? Would you have the same type of build tools?

You know, what's an I D E? What's a, you know, what's monitored? It's just curious. Like this, I don't know.

Can you even, we even get our minds, or either of you could even get our minds around, like how it's gonna change if the more we lean, the more we lean on the tools to do the work. So something clicked in my mind, and I'll start and let's see how it goes. So if I think about why programming languages were created mm-hmm. Right?

As opposed to coding in assembly language or machine language, whatever called that is to allow human being to get abstracted from the way the computer works, right? My question is, those programming languages, the IDs, the tools, and terraform, all these are human interface constructs. Mm-hmm. I'm thinking, do you even need them anymore?

Right? Right. Can the generative AI actually generate the machine language and cut all these things in, in the middle? Right?

So in, in terms of today's generative ai, uh, terminologies, all you need is prompt engineers and directly to machine language. I, I, I think you have something there, right? When we talk about constraints, current state, right? Constraints on delivering a new feature function opportunity for businesses, a lot of that constraints sits on what the technology team and which part, the development team that needs to go and build it.

So you start to optimize that. And when we begin to optimize for that, and now we have like, what, what is before them is whatever the, the, the title, the business analyst, the one that's sort of the product management, right? That owns that product is translating, here's the business need into the, what an engineer could consume to translate that into the machine's, uh, abstracted language. Then the machine translates that into the actual what.

So you come upstream from that. What are we left with, we're left with defining the business needs in English, let's say becomes the new programming language. It's actually English, right? So maybe that's what we're developing, is you become experts in that first layer of like how you, and that's the prompt engineer.

How do you, how do you code in English, right? How do you represent what the business needs in a way that the AI model can produce the machine language, right? Or whatever it is that's going to actually run and deliver that. Y you know, it's interesting, I saw this, uh, interview, uh, that, um, Sam Altman, he was the, uh, c e O of OpenAI, the ones that are doing chatt PT and all that.

And the interview asked him, um, what, uh, what do you know the least about? And without it hesitation, he said, he called it the, the, the, uh, not the art, the science of human guidance, right? Like, how do humans guide these systems? He's like, we know that's the part we know the least about.

How does it actually, like, and he was talking about how does it actually work, right? Like, what's the engine? The mechanisms under the covers, it actually allow the human guidance to keep things aligned and together and doing it what you want it to do. Um, and, but now when you kind of translate that to what we're talking about, it's like what's the, you know, the engineering practices, right?

What do those become? Because it's not just as simple as like, you can't just say like, oh, if you write me an s if you write me a four page essay and I pass it to the, you know, to the, to describing a user experience and I pass it to the machines, it's gonna create for me fidelity.com, or it's gonna create for me like Disney plus, right? It's like, it's like that part, it's like, it seems like that's a, but there's so much nuance, there's so much ideas. And you know how, and that's why we have so many people working on building these systems.

You just wonder like, you know, what is that science of human guidance gonna translate into the, the art or the, you know, the engineering of the human guidance and like, you know, yeah. That's, that that part is the part that I, I don't know what that's gonna look like, but it makes me the most excited about this kind of advances, right? Yeah, that's correct. And going back to what you started with the function construct, do you need the function construct?

No, because we are not having human interface actually coding the functions, because humans are restricted in those constructs, like function and all that, right? So if you cut short the thing, then not only your speed increases, but efficiency goes higher. Yeah. Because you need lesser amount of those constructs to do the same thing.

I mean, I saw some, uh, uh, LinkedIn posts or Twitter that actually had somebody drawing up an user interface Oh, yes. On a piece of paper, right? And it produced the code. Now, the code that it produced was JavaScript.

My question is, do you even need that JavaScript code? Right? Right. Now we do, because we need a human being to put that into ID to check it into a source control to build that thing, and then deploy that thing.

I'm like, do I even need any of these anymore? Right? Right. That's a good point.

And I think it, it probably will be a walk, right? The walk is the, the, the models are trained on the languages that we use. So I think the first iterations are probably going to be producing the JavaScript. So you create the thesis over this is what a, uh, you know, uh, a Disney plus, this is what a website, this is what a marketing website, this is what a, what a Fidelity website would look like if you, here's the user interface.

Here's what I want to happen. Provide it to it in that English format. If it's producing that in JavaScript, here's the next level that you could get. You have the JavaScript produced, your node, maybe Java, all the components of it.

You could produce those, right? But what prevents it from also creating the infrastructure code and actually launching the infrastructure and actually managing that, start taking away, 'cause we've been talking up to this point, like developing it, but it could go all the way back to some of the value of functions, is you move to less and less operations, ongoing maintenance of what if these, uh, AI systems are actually creating the infrastructure of the things run on. So now your businesses are defining that. We're actually creating that.

That's, That's a, you know, just brought something in my head. It's interesting. It's like, you know, is there gonna be, what would be a ne next like step function thing to expect? Could we come up with a, a full stack solution?

Like right now you're like, say it's, it's like we're putting AI on top of the human derived stack where all the assumptions and biases, and it's Conway's, a lot of the max, like everything's been created around human beings and our work and our, our limitations. You know, would we see another big leap forward at some point where the full stack is like, you know, AI friendly, right? It's like, it, it's like, how would a robot, it's like you would say s r e is, is how a developer would, would do I design operations. How would a, like, we, we could have something in the future, we'll call it the, you know, the, the Topo J, right?

It's like, is is it this, like, is Topo j this like this, like from the metal infrastructure that's all built on? How would, how would robots build technology? They'd use Jason, right? So, no, again, so Jason is Jason.

So, so he talked about full stack, who created those stacks, or the notion of the stacks human being, because we cannot think beyond that. That, hey, we need an infrastructure stack, we need a network stack, we need the UI stack, we need the middle, all those kind of things, right? And now we are struggling to find one person knowing all the stacks. And if we do not need all the stacks to start with, in our definition or our language, we do not need one person to know the whole thing.

It's just like going back 50 years in, in, in, in the way we used to develop application, right? We had only cobalt programmers, and that's it. That's funny, right? Yeah.

So same thing, it's like AI generator, and then interfaces English language. Yeah. And then generates the whole thing. Nobody cares about stacks anymore, right?

Yeah. That's interesting, right? Yeah. It is mind blowing.

I, I think that and the progression towards that, right? Because I, I think it isn't the first stop. I think maybe it's a fast follow, right? But the first stop is because, again, what it's trained on, right?

What we have to train it on is gonna look like the way that we do it, including, like you say, right? All our badness, right? So it's going to see some of that. And, but here's the thing.

We, we know it's difficult cognitively to have full stack engineers for humans. Yeah. Not so much for an AI 220 iq, whatever it is. Like it's ability with infinite memory, with it ability to scale.

Maybe that's not an issue. And so short term, the full stack that it's managing could realistically still be doable. Then you start to ask it optimize the full stack, right? So why not let it start to solve some of that too?

Yeah. The other thing is, uh, Uh, I will ask you, you know, if, if you, if you think about it, like production incident, uh, the detection mechanism. Yeah. Like we see today, as of today, like in radiology, and if you take medical example, the radiology, the AI or the image analyzer can detect much more efficiently and correctly than a human being.

Yeah. How is that going in, in the, in, in, in the very near future? Is AI going to change the kind of the production incident detection much better than a human being? Uh, I think it definitely, it definitely is, right?

I mean, it's, it's, you know, the ability for machines to spot anomalies, spot patterns, it's that infinitely scalable vision, right? It, it's, you know, it's, the tension can kind of go everywhere at, at, at once that, you know, humans don't have. And I think in the past, technologies were sort of very, the machine learning we did was very, like, not even domain specific, very like task specific, right? So de-duping and quieting noise and looking for, you know, related incidents, that sort of stuff.

So, and, and I think what we're now adding a layer on top of that is more of like a general brain, right? That can look for lots of different patterns and very easy to, to program, to look for, to learn new patterns based on information that either the human guidance guides give that or it gleams from other sources, or from reading your a p i documentation or, you know, like, it, it, I think the, the learning feedback loop in the general nature of the language models makes it so, you know, I think you're seeing an explosion in, in, uh, in capabilities there. But also it's important to remember that, you know, operations is not just the sort of software lifecycle, which at that point is write code, you know, you know, test the code, deploy the code, uh, you know, the code's there running, either it's a code error or it's a, you know, um, uh, a configuration problem. And we need to change that to achieve, to go back to the state that we want it to do.

But, you know, often in operations, there's so much more involved, right? Like, kind of why did it do that? And is that what we really wanted to do? Is it, you know, maybe different, but is it, is it okay, who are the human stakeholders involved?

They need to make a decision. Who are the people that need to be notified about it? There's a lot of human dance and coordination that is really the most expensive part of operations. And so I think also there's a huge advance there to say, Hey, how do we apply these models to beats where it's like, you can have the best, you know, enterprise.

Like, you know, like if you had a colleague that had the most, the, that had the, the greatest domain expertise across every bit of technology and had all the tacit enterprise knowledge of everything that got gone on in your company and was sitting right next to you. That's right. Anytime you had to think about something and they're like, Hey, what'd you think of this? Right?

And they can do it for you. I think that's where you're gonna see, um, a lot of, uh, innovation, innovation going is, you know, helping the hu human coordination side side of things, which is where the so much of the expense goes to right? Now, going back to something that you, you mentioned about the tools, right? So, so if these, uh, generative A p i, uh, AI and all the, all the, uh, AI models can detect anally starting from, you know, get go.

Do you think that when generative AI writes code, it'll be much more accurate and hence may not generate that muchly or failure in the system because most of our production incidents are caused by human beings? Yeah. Well, it's the unexpected. I, you know, I, I think yes and no.

I mean, it kinda reminds me of the, uh, the old Netflix, uh, folks, and they have to, you know, figured out they spent like a billion or two or whatever it is on reliability, right? They realized they don't have fewer incidents. They just had weirder incidents, right? Like the Swiss cheese holes just lined up in some crazy way in the, like, so they, they thought it'd be getting, they thought they'd see the incident volume drop, and it didn't, it was just weirder stuff was happening, right?

So I, I, I think, you know, a a lot of that is, is, um, yeah, so a lot of like the, the individual bugs. 'cause if you can, if you can stop your, if you don't have the human fallibility of like, getting confused and missing something, right? But that, I think that's, there's a certain class of bugs like that. But I think a lot of, in complex systems, a lot of the major incidents, or even just the, the not major incidents are like, is this right?

Is this wrong? We didn't expect this. The user behavior changed. Some third party changed something underneath the covers.

And lo and behold, it causes some weird interaction that we didn't expect, or the type of data we're getting suddenly changed, right? Mm-hmm. Um, you know, there's all these kinds of weird things that, that happen that is not as simple as, did the code run or not run, right? So while did the code run or not run, I think that class of error gets solved a lot quicker, right?

Mm-hmm. Either quicker on the, on the, on the production end, or just quicker, you know, in the, the pre-prod never happened in the first place, but I think then you're, you're left with just all the other stuff, which is, you know, say you just get weirder mm-hmm. Weirder, uh, problems. So, so we've been kind of going on the deep end of different crazy ideas here.

I I want kind of bring it down, you know, few minutes left, talk about, um, like, let's get practical about it, right? So where do you, where do you see in, you know, you guys are in, you know, large household names, right? You know, and, and, uh, uh, your companies, you guys too. But the companies, the companies that you're, that you're in and you know, you've had these roles and positions and you know, like where do you see this technology comes into an org?

You know, you don't have to talk specifically about your organizations, but for, as one would say, if organizations like you, you, you deal with, as this kind of technology comes in, where do you look to apply it? Is it like, is this like a, in the developer productivity land, is this in the sort of core infrastructure platform, you know, uh, changes? Um, you know, yeah. I just kind of curious, like if, if you had formulated thoughts on sort of like, or maybe it's all still trying in one of everything, you know, how do you start to apply something this much of a sea change, this much of a, of, of a, you know, shock?

How, how do you start to figure out where to where to put it in to, you know, to get rolling on it? I think it's a great question. Um, my, my first thought on it is that I think that our companies are going to at first struggle to figure out how we so approach this in a couple of different reasons. One, what are, what, you know, what's the liability here?

What's the protection here for the companies, depending on the tool? And let's say that we get past that, I really want to kinda move past that. 'cause I believe that's gonna happen. I, I, I think there's, there's probably an in some intentionality on blocking to protect things like ip, right?

And I think rightly so, but at some point there's going to be mechanisms and ways to protect that and still have these models available to the businesses. But I just wanna highlight that. 'cause I do think that's real. I think that's something that, that, that businesses are gonna have to grapple with.

And I don't know how long that takes if legal's involved we're about a year, right? You know what I mean? So it's gonna be a bit of time, right? For them to sort of sort that.

But once we get past that, I think the first step is that it really doesn't matter where you are as a technologist. This is going to be a force multiplier, like a 10 x. Like we talk about the 10 x engineer, and I joked about, uh, it, the 10 x engineer suddenly became the person that could use search and, you know, yeah. Stack overflow and suddenly you were a 10 x engineer.

Well, well, here's a tool that if you're a developer that will help you start out of the gate, help you take your existing projects or your new projects and bootstrap and get you faster in delivering them, um, as that begins to mature. And I think that doesn't end there on the operations right front, especially in ss r e space where we can have a lot of, of training data that maybe we augment based on our environments that could help, uh, not just the infrastructure as code aspects of it, but also to your point, even the incident remediation, understanding what could be some of these, uh, failures in our design. Could we at some point upload these designs? It's identifying constraints, playing through simulations, if you will, of where we would fail doing Chaos Monkey with actually, I actually have to execute Chaos Monkey, right?

Why not? Right? And that could be a 10 x thing for, for us to be able to see, um, further time will be gained back because I think we're gonna be handed it. And I think that that vacuum will pull more work into it.

What I'm saying is I think that, that even when we get to that further state we were talking about, uh, later, I think it's just going to increase our ability, and I know this is optimistic, so William am wired, but it will increase our ability to experiment and develop new things faster, do more experiments that we validate from a business standpoint in market. And I think that it will be a tool for businesses to do just that. And I think that's why we're going to see some of these initial problems, uh, uh, overcome. Because I think the businesses are going to see the value in using something like this.

That's across the board. And we've been talking more in the technologists, the engineering front, but it doesn't, it isn't limited just to that, like you were talking about radiology. This goes into business, right? This goes into marketing, this goes into anybody that's working in across the business that could leverage a tool like this, Uh, just like any other developer in any other company, right?

Uh, there are two aspects to it right now as you speak. One is, Hey, this thing is super cool. I can do whatever I wanted to do without knowing much detail about that thing. For example, I generated Selenium cucumber test cases of unknown websites without knowing anything about the website or knowing about Selenium either.

All I needed to do is follow the instructions given to me by Chad, G p d, and I just stuck it into the code, and I just ran it the way it wanted me to. So that's super cool. So I'm experimenting with stuff that I have no idea about. So that's that.

The other side of it, the fear or unknowns, right? Can I put this in production? So I think every company is thinking about that. What are the legal boundaries about this?

What are the security boundaries about these while using chat G P t, am I exposing information that I should not be? So those kind of unknowns need to be figured out. And I think all companies across the board are kind of focusing on these two things, which is experimenting new things, which are, and then at the same time being a little guarded around That. So you're seeing the, the velocity of, you know, a developer productivity sidekick kind of tool that seems like much more smooth sailing or there, there's a clear path, right?

Versus like, Hey, how do we start to actually build this into our infrastructure? Much more murky, much more kind of, you know? Now here's an interesting thought you brought up. You know, and I, I, I hear this too, that's like the, hey, there's such a big backlog of digital desire, right?

That, you know, the more productive we get, it's just gonna suck more work into that, right? But here's this weird thing I'm, I'm thinking about is that, you know, the web, mobile, like, you know, probably, you know, all these things were, they were like, uh, cost additive. Like you had to go build a web team, you had to go build, you know, this thing. And it was also kind of like, you know, if you're not a technology company or dependence company, which everybody is now, but you can be like, Hey, we could be a little slow to that.

You know, if you make great movies in great parks, you can be a little slow to, you know, making a, making a better website, right? Um, you know, and, you know, fidelity, I'm sure they started, you know, there's more, you know, like there was other places to sell funds, and it's just kinda like, eh, we could be, we could, as long as we're not too far behind, we can catch up, right? And so, but it was a cost, or even moving to the cloud, right? It was like, oh, we have this sunk cost of data centers.

We gotta buy stuff in the cloud, right? So it's like a cost additive thing. But this, you mentioned the business, right? And customer service and marketing and all the, you know, sales, B D R jobs, all that kind of, you know, all the entry level positions, right?

This is a massive cost, like cutter deflationary, right? So it's like the first C C E O in a business or in, in a category, who realizes they can use this to cut cost, which boosts earnings, right? Right. Then what's that?

I think the pressure, what's the pressure gonna be like then, right? And it's something to think about is like, now it's gonna be, Hey, I know you think that we're gonna cut, you know, we're gonna cut the time it takes to do things, and we're gonna use all that extra human capacity to do great new things. But is that also gonna hit that corporate reality of like, if we have a structural cost as advantage to our competitors, we're dead in the water, right? Yeah.

So then is that gonna, is that gonna add a different dynamic that's that's, that we haven't felt in our other technology transformations? Is that the big cliffhanger? Yeah, I think, I think these are all good questions that we can answer maybe next year. Yeah.

So think about, right? I mean, where's this, I guess that's the point, right? The point is like, this is like the web and that, like, it's a, it's a brave new world, right? What's gonna, what's gonna, oh, sorry, I wasn't talking on microphone.

It's a brave new world, right? And I guess, you know, the best we could do is just keep thinking about what the next couple steps are and not get caught in like transitional thinking. The transition's the end state, right? Yes.

Yes. Not think too much. Just go with the flow and see where it lands. Well, I'm good.

I'm good at not thinking too much. So that's, that's, uh, that's important. All right. Uh, any fi any parting words or should we just, uh, wrap it up?

No, I think this is exciting time for all of us. Uh, you know, and, uh, hopefully next time when we sit down and chat together, then we'll have more data behind our, our thought processes. Yeah. I, you know, I just as a, as a technologist, you see this, right?

This is, we got so many of us, right? Got into technology, computer science because we saw the potential tools amplifying human ability, helping us go further, faster, at greater scale. That's what we're seeing. It's unfolding.

It's exciting. Yes. It's a little terrifying. I think we we'll agree, right?

Is it, where does this go? We don't, how do we answer this, but yes, I agree. We'll have more to say. Right?

And I, I definitely, it'll be exciting to see where we go on this journey. We Can reconvene next. Maybe we can reconvene next year and talk about this. Yeah.

Alright, well, hey, for all of you who are still listening, thank you so much and, uh, we'll talk to you next time.