Host Intro (Gene Kim)

Our first speaker of the day is John Rauser, senior director of software engineering at Cisco. Among other things, he owns their zero trust offering in the cloud security group. In 2024, he presented their early work creating common AI capabilities that span multiple products, and how one decides to balance whether you have the individual teams build those capabilities or whether you want to centralize.

Since then, he's continued to work on amazing things, and his product portfolio recently doubled in size — which I'm so happy about because I know this will give him an even bigger impact on his organization. Today he'll be sharing how AI has been reshaping his technology teams, how it's changed how he leads, and he'll also share this amazing story about how he convinced his SVP to require one hundred of the top leaders in the organization to vibe code something and put it into production in the quarter that ended in October.

And what he learned — what we learned — I think is relevant to all of us. Here's John.

---

John Rauser

1969. The first digital network is created: ARPANET.

Within a year of ARPANET, one of the most impactful communications technologies of all time is created when Roy Tomlinson sends the first email — with the @ symbol that we all know and love today.

But for decades, email adoption in large enterprise was slim, and it wasn't moving. In fact, IBM did a study, and they found that the problem was at the leadership level. If a leader was using email, their org might use email, their group might use email. But when they weren't, there was no email. And in fact, what they found was most executives were having their email printed off in the morning, on a stack on their desk. They'd note them up, hand them back, and somebody would type their emails up for them.

In fact, there are still people doing this today — famously. Not going to name names.

But now here we are, and we're trying to get everybody in our organization to pair with agents. And it turns out that it's not as easy as we thought.

So today I'm going to talk about our journey to what I'm calling agentification. We're allowed to make up words here, I found out. Right, Gene?

Agentification of our enterprise. How do we get agents in our software development — but in everything that we're doing? I'm going to talk about that journey here at Cisco.

My name is John Rauser. I'm a senior director of software engineering at Cisco. I lead a group of security products that we deliver to the largest companies in the world — banks, insurance, airlines — helping to secure businesses and organizations all around the world.

So, like I said, the challenge in front of us was real. We're not getting the kind of adoption that we're looking for amongst the people that are supposed to be getting the most value from agents: engineers.

And the problem, it turns out, was not a technical one. The problem — and this shouldn't come as that big of a surprise to anybody that has looked at these problems in the past — the problem is a psychological one. It's a cultural one. And in technology adoption, this is not new. We come across this a lot in this community. Everything that's old is new again. Somehow, when the new things come out, we forget that.

So as we started on our journey to agentification, we were measuring productivity of our developers. We were looking at the tokens that our developers were using. But it didn't occur to anyone that we should be looking at maybe another level.

We have all the tools. Cisco is, by the way, one of the most AI-bullish companies in the world. They give us every single tool that we want. If we don't have a tool, we can ask for it. There are no credit limits, no VPs sitting on getting more tokens. Everybody has a budget, and you're expected to use it.

But like I said, it's a psychological problem, and we needed what I thought was a structured approach.

So last year, I was at our annual conference, Cisco Live, and got a chance to sit down with the SVP of our organization. We were noodling a little bit on how to do this. And it occurred to me in that moment: why don't we look at how many tokens the leaders are using? Maybe that'll be an indication of something. And one thing led to another, and next thing you know, we were talking about — well, how do we actually get our leaders using those tokens?

One thing led to another. We had a challenge put out to the entire organization: every leader, every people leader in this organization — if you lead a team, you are going to build something into our product and ship it over the next quarter. You have three months to do this.

It wasn't an ask, it was a tell. It was a directive. And it had to be, because we needed to just get that initial spark going. And let me tell you, folks, it worked.

There are a lot of run-on effects from this, and I'm going to talk about them here. The first thing I'm going to talk about is that challenge — and Gene's going to present some of the data, because we surveyed everybody afterwards and collected data on how effective this experiment was. Then I'm going to talk about how we developed a passion in our leaders by getting them to do their own personal projects to enhance their own leadership experience. And finally, how we're kicking off right now at an organizational level and truly building out that agentic SDLC. Those are three things.

01Leader Experience

We issued that challenge to the organization. We had 100 people, all enabled with the tools of their choice, unlimited tokens. Ship something into the product — it can be anything. A small UI enhancement, an automation, some new monitors, some new observability. Doesn't matter. The point is to get that hands-on experience and also develop an opinion, develop a perspective on where the agentic SDLC is today. What are the bottlenecks? What are the roadblocks?

And truly, not just listening to people when they say, "Oh, that's not going to work," or "It'll never solve that problem," or "That'll never work here" — we've heard that before too, right? Not just listening to that feedback, but actually validating it and understanding.

Like I said, ship anything. And here's what happened. Gene's going to come up and tell us a little bit about the results of our survey.

---

Gene Kim

Yeah. So when John told me about what he was doing, I was like, "Holy cow, I need in on this." It was for the quarter that ended in October, and so I reached out to him in November: "How about if we do a survey that goes out to all 100 people?" The research question was: all right, who finished, who didn't finish? And of the people who finished, how did it change how they lead? The independent variable was: did they have more fun? What did they do? And then just get a survey of practice — what they did differently. Think about a vector: direction and magnitude.

So it was just an amazing thing. In Claude Code, I developed a survey in about an hour — which, having done the DORA work with Jess Humble and Dr. Nicole Forsgren, would have taken weeks. I text John, "What do you think?" He didn't respond right away, I got a little bored, and I'm like, "Okay, let's start writing the survey questions." That was done in about 90 minutes. I'm like, "Who would ever go back to the old way?"

So I'm going to share the top five findings, somewhat quickly, because I don't want to detract from John's time.

It was 100 engineering leaders spanning managers, senior managers, directors, and senior directors. Managers are first-line managers; senior managers are managers' managers — and they might have a team of their own. Directors manage senior managers; senior directors manage directors.

We only got 19 responses, even though John promised 100% compliance. But it's actually a great sample set. The response rates were quite high — one out of three senior directors and senior managers responded — so it was enough to get some interesting signals.

Here are the top five surprises.

One: The higher the leader is in the organization, the more fun they had. Across every dimension of FAFO — which is this vibe coding book — excitement, net promoter score, trust, the senior leaders outscored the more junior leaders.

Two: 85% of those who responded reported how they changed how they lead, across four really interesting things that would be familiar to anyone in leadership. They got better at setting direction. They viewed themselves as raising standards for the entire organization. They went from individual problem-solving — where "I make all the decisions" — to empowering others to make better decisions. And modeling and multiplying: showing their team, "These are the behaviors that I am going to model for you, with the expectation that you are going to show me." It was just amazing to see the vivid descriptions of what they did differently.

Finding two — and oh my gosh — former coders, people who haven't coded in 10-plus years, wiped everyone else off the map. They did better than non-coders, which was second place. The ones who performed the worst, who had the least difference in leadership behaviors — in fact, almost none — were people who are still coding every day. Isn't that interesting? Former coders aim bigger, have more fun, influence more people.

Finding three: Fun is one of the highest correlations with everything. The more fun the leader had, the bigger the changes, the more things they did as a result.

Finding four: Active coders almost did not change at all. On average, they had half an idea of something they did differently. They had the least fun, dreamed the smallest, and had nearly zero concrete leadership actions that they reported.

And the last one is something really curious: senior managers were having the least fun. Across almost every measure, they were doing the worst. The theory here is that it's kind of a tough job — they don't have a team big enough to delegate to, and they're also the busiest. So they had the hardest time carving out time to do this.

Really, really interesting.

I'm hoping this is of value to you. Our plan is to publish the dataset, and if this sparks some questions or resonates with you, come find John. Our goal is to publish an anonymized dataset soon. So with that —

---

John Rauser

Fantastic. Thank you.

So just to summarize why this works: I'm a very strong believer in leading through experience, leading through example. I think it was a very powerful thing for the people leaders of the organization to be showing up with their teams week over week — showing up at our group demos with the things they had built using agentic coding systems.

The feedback from the engineers — and that's actually a population we didn't survey, and we could — the feedback from the engineers was extremely positive. A lot of people just saying, "If there's one thing that's going to get me to change, it's my boss showing up at the end of the week with what they did that week." Incredible results.

And the other thing that happened was the leaders gained a shared perspective — not just on the work, but on the language and the patterns and how to discuss these things in practice. Extremely powerful.

But it's not quite enough, in my opinion. If we want to really get to the root of the problem, we can't just have people doing the work. We have to have them doing work that they truly feel passionate about and get excited about.

So that's the next part of this presentation: how do we ignite the passion within these people leaders?

02AMOS: Agentic Management Operating System

This effort was more focused on my own organization — building an agentic system for managing my organization, which is a multi-level organization, a couple hundred people: managers, senior managers, directors. I want to have that whole organization connected and operating using agentic practices.

So we developed what we are calling AMOS: the Agentic Management Operating System. A suite of tools and capabilities that managers are using to essentially act like a chief of staff. Now, I've never had a chief of staff, but if I did, what I think they would do is help me with all these tasks — getting ready for all the things I have to do. Attending meetings, doing one-on-ones, looking at various documents. I think that person would be helping me get all those things together.

Does anybody have a chief of staff? Okay, not many people. Good — we're all on the same page with that one.

In order to do that, you have to have knowledge. You have to have information. You have to have all this stuff available to your agents. So at its core, AMOS is that: a knowledge management system powered by agents, enriched, and producing what you need for your day every day.

I think we're actually onto something with this one. If you saw this recent tweet by Andrej Karpathy, it's pretty interesting — his coding agents aren't coding anymore. They're doing knowledge management. And that is what we have been doing in my organization now for a couple of months: building systems that pull in all the information that we're looking at and then connect it together.

It's almost like an enterprise Open Claw — we're not allowed to run Open Claw in the enterprise, but we can build it ourselves, it turns out. And it's a lot of fun. It is the F part of FAFO. It is the thing that gets you excited to get out of bed in the morning, jump on your computer, and just see what it did last night and think of one more idea.

In this age we're in — it's the age of ideas. Ideas are the currency. If you have an idea, you can execute on it. As a manager, I have a lot of problems, and I have a lot of ideas on how to solve them. So AMOS helps me do that, in this loop where it's pulling information in, showing it to me, and raising the bar on everything I do.

Here's the thing. I spend the same amount of time getting ready for my one-on-ones. I spend the same amount of time on the dreaded weekly report. But I spend that time thinking. I don't spend it finding and remembering and searching — which was a lot of what I had to do every day. Where's that document? What did I do this week? Checking my calendar, trying to get all this stuff together. I don't have to do that anymore.

It just comes together with AMOS, and the quality is incredible. The difference between somebody using an agentic system to produce their weekly status report and somebody who's doing it manually — it's night and day. So much better.

There's a lot of fun to be had with all these capabilities. On the top, you've got the things that you're going to work with: you're giving agents instructions; you give them a memory so they know what they did — kind of like a feed-style thing, but not that complex. You create skills — for example, examining change requests, a skill for examining an incident that happened and seeing the output of that. Then on the second row, you've got all these things that are gathering information. Hooks are happening: any time I chat with my agent, it's firing off, remembering things, maybe creating documents. Any time I open a document in my browser, I have a trigger where my agent goes and summarizes that document and puts it in the right place. Nightly automations are enriching my docs all the time, and then connecting them into all the different things.

The result is something like this — and this is where the fun is. I've got my knowledge system on the left, which is just a folder of Markdown files that's constantly being populated. I've got tools that I built, like a clipboard manager that's MCP-enabled so that my agent can actually interact with my clipboard and see the kinds of things that I'm clipping. In fact, Control+C is now an input into my knowledge management system. And then the thing I love the most is my browser extension that's watching all the sites I'm going to, recording them, allowing me to note them up. Here, I was just noting up the agenda for today's program — which is a very fluid agenda and changes sometimes. So I need to be able to go back there and check it.

These are all just fun things that I build, and I'm constantly adding to them. The organizational piece is getting everybody in my group to do that too. When we have to create a weekly update at the end of the week, they're all coming in to me with Markdown files. I have automations watching that, generating my weekly status report, and then I go — like I said — spending my time doing deep analysis, doing deep thinking about what's in there, rather than having to search and remember and find things.

I think the key enablers here are getting off those old note systems. If you're using OneNote, I'm sorry — I was a OneNote guy for 10 years, it's over. OneNote's over. Even stuff like Notion, Evernote — it's not agentic-friendly. I just need Markdown files that are local and in shared folders to pass them around.

Shared reference maps are important, because our organization is very complex. There are a lot of components, a lot of acronyms, a lot of shorthand. There are Jira projects — hundreds of Jira projects — there are GitHub repos. The agent needs to be able to find this stuff when I say, "Hey, I need a report on SWG incidents for the last week." It needs to know what that means, who the leaders of that are, where their Jira projects are, what GitHub repos they're using, and it will just go and find those things. So these shared reference maps are important. We create those together, we pass them around.

And then finally: community of practice. Like I said, ideas are this amazing currency now. And it shocks me, the stuff that our leaders are coming up with. Just last week, a manager on my team showed me what he calls the predictive dashboard — the agent looking at all the transcripts from the meetings he's had, looking at all the work that his people are doing, and telling him what projects are at risk. Are things off track? Are we going to slip these dates? Is this engineer overloaded? All that kind of stuff — and it's a nice dashboard in a web page.

A really neat idea that came up even earlier this week in a different context: how do we get to that predictive intelligence using agents? Like I said, it all starts with just ingesting that information. The key thing for a frontline manager is getting those daily stand-up transcripts into a folder so your agent can see what happened this week. The agent then has the same view as you do about what's going on in your team.

A lot of fun, and it truly ignited a passion in my leaders. We're constantly getting excited every time we meet about the things that we've done. So not only did they have the experience of building systems using agents into our product, but now they're building systems for themselves that are making their job easier. And that's fantastic.

03Team Adoption: AI Squads

So how do we diffuse this into our teams? How do we get our teams thinking the same way as us about building, generating ideas, and executing on them quickly?

Here we come to the third layer: our team adoption layer.

We have this idea that in the future — maybe even in the now — software is going to be built by much smaller, nimbler teams. Maybe not fewer people, but teams that are very focused on achieving single goals. We call those things squads. We call them AI squads.

The goal of an AI squad is simply to develop a feature using mostly agentic approaches — as much as it can — and move beyond this idea of... here's a common question: "What did you do with AI last week?" "Oh, well, I used Codex and I built a feature or capability that would have taken me a week and I did it in an hour." Well, that's fantastic. But how did the system change? How did that contribute to the overall growth and evolution of our entire agentic SDLC?

That's the question we have to ask everybody today. People need to be thinking not just "I'm building features using tools" anymore, but "I'm building the systems that are going to use those tools to build the features for us." The system is the product. The features in the product are the outcome of the system. That's what we're trying to do with the agentic SDLC.

We think about it kind of like this: teams are like EC2. Teams are like VMs — long-lived, always running, persistent. Whereas a squad is something that forms quickly. It's a Lambda. It's going to form quickly, get its job done, and then disappear.

But there are a lot of problems with that. The people running the VMs are like, "You're going to do what to my environment? You're going to come in and write all this code, and then I'm going to go on call for it? I'll be responsible for it? No way." That objection is rooted in a number of assumptions — assumptions about the quality of that code, the testing that can be done with it, the ability to go on call for it, and the changes in the system. So what we need to do is change those assumptions. We need to attack those bottlenecks.

So the first squads we have running right now — we've just stood up a group of eight squads, very cross-functional, touching a lot of different parts of the system. Their target is to build a feature. But just like the leadership challenge I talked about earlier, the goal is not to build a feature. The goal is to affect change. The goal is actually to build the agentic SDLC. In building features with this challenge of 100% agentic feature development, what bottlenecks are you encountering? What roadblocks are coming up for you? Attack those.

Every step of the way — is it integration testing? Is it performance testing? Is it deployment to production? Go after it. Figure that piece out. A big one was requirements — clear requirements. Agentify that too. We'll work with the design team, we'll work with the product team. We'll create agentic-driven specs, and we'll create them really well so that the coding agents can go execute on them.

The point is that every objection that people have to this agentic SDLC, we can figure it out. It just takes that focused effort, and it takes a belief — a conviction — that we can do it. And that has to come from leadership.

04DCAF: Declarative Agentic Framework

We're having a lot of interesting things come out of this. One we call DCAF — the Declarative Agentic Framework. It's a system that you point at your product area and it will generate all the things that agents need to work with it: the skills, the bootstrap information, and then a set of activities for your teams to pull out all that tribal knowledge that an agent might need. How we test, what our monitors look like, what our performance thresholds are, what the customer issues are that we care about. Where is this product going? What does the future of the product look like — the scaling, the globalization, that kind of thing. All that needs to come out of people's heads and go into a set of written documents that agents can work with to understand what they're building.

So this is sort of just an automatic thing. You point it at the repos for that product area, and it will generate that for you and initiate the activities.

This is just a grassroots idea that somebody had as they were going through the squads. They said, "Here's the problem: I need to work with this code base over there and it's not agentified. So what I'm going to do is build this solution that goes and agentifies it for you, and then you — owner of the team — can go and review it." That kind of thing is just coming up all the time, and it's just fascinating to see all these sparks flying amongst our engineers and our leaders as we've gone through this journey.

Of course, there are a lot of activities that go into supporting and diffusing the knowledge that we're gaining. But I'll tell you, folks, it's really working. We have got the beginnings of a true agentified organization, and we're seeing the results, and we're measuring the results.

Perhaps next time I can come back and talk to you about what that's looked like. So thank you so much for having me here today. Gene, amazing to get this group of people together.

---

Q&A

John Rauser: This is important — this is actually what I'm looking for, right, Gene? I do want to understand how people are thinking about the agentic SDLC. What does that look like for you? What does that look like for your organization, and how are people reacting to that? I think that's the biggest thing I'm looking for. You can find me on LinkedIn or on the group Slack. Thank you. Thanks, Gene.