Host Intro (Gene Kim)

[00:00:21.255] All right. I met the next speaker, Dr. Tapabrata "Topo" Pal, in 2014, and boy did he make an impression on me. He was the first distinguished engineer at Capital One, and everyone points to him as the person who led the technology modernization movement there.

[00:00:36.855] This was shortly after the era where 80% of their software engineers were outsourced, which he was trying to bring to an end. Rumor had it that he was probably the first enterprise architect back in those days who had an IDE installed on his laptop, because that simply just wasn't done.

[00:00:52.415] In 2021, he joined Fidelity Investments as their VP of Architecture. Fidelity was founded in 1946, recently celebrated its 75th birthday, and has over $16 trillion of assets under management.

[00:01:05.735] Topo is one of those people who was initially skeptical of vibe coding, especially in the enterprise context, especially with large code bases and contexts where there were lots of security requirements. But a couple weeks after a bunch of us hung out in April at our forum event, he texted me about something he was incredibly excited about, and I'm so excited that he can share the story with us. Here's Topo.

Dr. Tapabrata "Topo" Pal

[00:01:32.705] Good morning. How is it going so far? So many good talks this morning that I almost forgot what I'm going to talk about, so we'll see how that goes.

[00:01:42.815] This is, I think, the ninth or tenth time on this stage talking about DevOps. But this time it's a different topic. The reason is not that I'm done with DevOps. It's a different reason, and I probably will be able to talk about that in the next 20 minutes.

[00:02:03.535] I have had many hats in my career, from academia to developer. I wrote a lot of code in my life. I logged into production. I had pager. I blew up production, and I fixed it also. That's why they promoted me to different levels. The idea is you cannot fix something if it's not broken. I don't know if that makes sense or not, but from DevOps, I'm kind of shifting toward AI, GenAI coding assistants. That's my talk today.

[00:02:44.895] But before I say anything about my talk, these are certain disclaimers. I'll let you read it for 10 seconds or so.

01Fidelity and Topo's role

[00:03:01.225] You all know about Fidelity Investments? It's one of the largest financial institutions in the world, headquartered in Boston, about 77,000 associates all over the world, 11 countries, and total assets under administration is $16.4 trillion. Total discretionary assets is $6.4 trillion as of June 30th, 2025, and 4.3 million daily average trades by customers.

[00:03:36.905] Why I like Fidelity is because of these: our priorities are about improving financial well-being of our customers, leading technology landscape, career vitality for our people, and we have world-class benefits. Then we are deeply engaged in sustainability and diversity and inclusion.

[00:04:01.345] A bit about me, as Gene mentioned. Currently I'm one of the vice presidents in Fidelity Architecture and Engineering, which has many domains for the whole enterprise. My domains are those two things: software delivery domain, which is essentially my background for the last decade or so, and then I also have the open source program office, which is related to my use case that I'm going to talk about here.

[00:04:27.765] Previously, I've been with Capital One, Home Depot for a while, Quadramed, bought by Nuance, now Microsoft, Circuit City. Before that I was in academia for some time. This is the book that I'm very proud of, Investments Unlimited. I co-authored that with many of the people that are also here in this conference, so that was a great experience for me.

02Hypothesis and use case

[00:04:53.315] Here's Fidelity and my hypothesis: GenAI coding assistants can accelerate development, streamline code reviews, enhance test coverage, and support faster delivery cycles. This is the hypothesis that we have in our enterprise. My talk is going to show you whether that hypothesis is provable or not.

[00:05:18.465] My use case. I'll pause here for a while, because as I said, I'm accountable for open source usage, open source governance across the enterprise. Software bill of material means a lot to me. In terms of software bill of material, I thank Josh Corman, who introduced me to software bill of material back in 2015. I'm still kind of in there, seeing things happening that I wanted to see in my mind.

[00:05:49.085] Software bill of material is not about security vulnerability. It is there. It is one of the use cases, but there are many other use cases. To me, if we are building software, we need to know what's in our components, period, and security is part of that.

[00:06:06.385] So what do I have? I have a very good team of people, amazing backend developers, that over the years developed a complete graph of every software that we build and their components, their metadata, where they're deployed. Essentially it's software bill of material, including a part of infrastructure bill of material, if you think about it. Why do I kind of combine them together? Because infrastructure is nothing but software. So it's basically software bill of material. Where are the things that we have produced? What do they contain? What kind of metadata is in there? Where are they running? If something goes wrong, where can I point to and figure out how to fix that? That's the use case.

03Existing architecture and bottleneck

[00:06:53.275] High-level architecture of what I have is essentially from code repository to artifact to container running, and in runtime I collect a lot of data and put them in a graph, in Neo4j graph database. Now, we have been building this for about two and a half years, three years now. The main use case of this is not realized fully because it's just a graph.

[00:07:22.705] Now the team becomes the bottleneck, because there are so many use cases of that and so many requests that come into the team to answer, like: who is using this? Who is using this commercial product? Where is this license being used? Which repos are mostly using this particular library? Think about the Log4Shell scenario. Right now, in five seconds, I could tell you exactly where Log4j is used across the whole enterprise. That was not there before.

[00:07:54.155] The problem is that I don't have something to give to the user in terms of an API or a nice dashboard. My team stood up a NeoDash dashboard, which is built on top of Neo4j. It comes with Neo4j. It's open source. It is kind of okay, but not very good from a user perspective. This is what I want in my mind, and I've been kind of struggling for it for a while.

[00:08:27.705] This is out of the box in NeoDash. I grayed out certain things just so that you don't see all the details. Essentially, for any application you can actually see all the licenses used, all the libraries used, how many repositories are there, whether they are healthy, and some details that I grayed out. Imagine some Commons Collections. If there's a vulnerability, I can actually see all the repositories where it's used. Again, this is out of the box, NeoDash. That's what I have today. It works. It is slow, it's clunky, and it doesn't meet all the needs.

04First experiment: Cypher queries

[00:09:08.705] So I started experimenting with this. At this point, you need to understand my background. I'm a Java developer. I can work with Python. I can hack around with JavaScript. I know SQL and all the good old stuff, but Neo4j graph database, I have no idea.

[00:09:28.585] I started asking GenAI coding assistant: hey, how do I get my schema from the Neo4j database? It says, here you go. I said, okay. I pasted back there in the coding assistant and said, with this, can you help me with Cypher queries? Neo4j uses Cypher queries. It basically said, yes, I definitely can. What do you want to know?

[00:09:54.285] I said, I want a query to find out which repositories in my enterprise have used Log4j as a dependency in their POM or Gradle build. It says, here's the query. I took the query and I pasted it on the Neo4j browser, which is essentially, think about it as a SQL development platform. I pasted there and I got the result.

[00:10:20.785] Now you are thinking here, Topo, don't do that. Don't take a query produced by a GenAI coding assistant and paste it in a browser to see what's going on. I checked. I was on the development platform, and I checked the query didn't have any of the D words: delete, detach, drop. I'm safe. So I put them and I got the result.

[00:10:46.545] Then I started asking complex things, and the coding assistant started giving me complex queries. Then I said, well, these queries started running for a long time. Can you optimize it? And it did. So this is my first experiment.

05Second experiment: product requirements to build

[00:11:02.045] Then I started thinking about my original wish, which is that API and that nice user interface. Can I build this? So I started my second experiment. I closed my laptop. I wrote down everything I need, that I wish for, given the Neo4j database. I need an API layer that I can give to the users. I need a nice dashboard.

[00:11:28.905] I wrote everything and I asked Copilot and GenAI coding assistant: hey, can I get a complete list of functional and non-functional requirements formally written up using my story? And it did. Then I said, okay, I need a detailed task produced by these functional and non-functional requirements, and write me a Markdown document. And it did.

[00:11:53.105] I took that and went to the team. I said, I want this. They looked at it with big eyes and said, that's a lot of work. I said, I know this is a lot of work, but this is what you need to do. The team gave me an estimate, but I'm going to hold back on that estimate because I'm going to ask you what you think the estimate should have been after seeing the product.

[00:12:18.185] I took that detailed task and then I took the estimate from the team. Then I started asking my GenAI coding assistant: can you build this? He said, sure, I can build this. It'll take a while, but I can build this. I said, all right, let's go.

06Results after five days

[00:12:35.785] So next five days, including many nights, maybe three nights and five days, I came up with this completely written up: React frontend, Node/TypeScript backend APIs; code for SSO integration, because that's our enterprise standard. Everything that you need, every UI or dashboard or anything that you have needs to have SSO integration.

[00:13:02.065] It had feature toggles, enterprise standard logging. In fact, I asked GenAI coding assistant to look up the document that describes our enterprise logging standard, and it followed it. Then end-to-end request tracing, correlation, performance monitor hooks in it, end-to-end testing, greater than 80% coverage. It took a while for me to get there. In the beginning it was not 80%, of course.

[00:13:31.235] Zero linting and code quality issues. That was one of my hard requirements in my task list, that you will not have any linting or code scanning error. Dockerfile, CI pipeline, architecture diagrams in a Markdown format, and of course a few functional bugs that we all know about.

[00:13:51.945] Here's the result from that. It came to this with beautiful, colorful, interesting charts and numbers. Essentially it showed me at a high level how many total applications I have, how many high-risk libraries I have, license compliance, vulnerabilities, technology stack usage, and a few other things, license risk distribution. It basically drew all this chart for me.

[00:14:28.875] Bill of material registry. This particular application that I'm building, the whole bill of material registry's bill of material is shown here. It is actually showing all the repositories this particular application has, when it was last scanned, scan status, and all the details. All this data is already in my Neo4j database, but to see this and give it to the user, it was all custom Cypher query. Now users can actually help themselves.

[00:14:59.405] Commons Collections: if I do a library search, it comes up with the hits. If I select any of these, it basically will tell me all the repositories that are using this and all the other metadata along with that usage. Then license risk assessment: what kind of different licenses am I using across the whole enterprise and their risk profiles?

[00:15:22.865] This is a beautiful thing. It actually is building the graph. The red box, if you look at the red box, is the application. All the blue round circles are repositories for that particular application. The triangular-shaped things are direct dependencies or components that each of these repositories, or the whole application, is using. Then the square kind of things or diamond kind of things are the indirect dependencies. If you move your mouse, you actually see the dependencies. This is awesome visualization that even I couldn't think about, that can be done so fast in five days.

[00:16:11.505] Here's another thing. If I actually touch the diamonds, it basically shows what dependency it is and some more metadata. So I started wondering myself: okay, five days it coded. How large is this project? If you think about it, it's not that big, but for five days I think that's huge: 270 source code files across the entire project. The structure of it: React frontend, Node.js backend. Those are the two major components.

[00:16:50.225] Now think about all the things that it has built in five days. What do you think a normal agile team would take to build this product? Any guess? Six months. My team actually gave a similar estimate around eight to 12 months to build the whole thing, and I got it in five days. I actually handed over this whole product to my team 15 days ago. Last Friday, it went to production.

[00:17:33.065] A few things my team needed to do to get to that finish line, which is of course we know our enterprise processes that take a while, and I want to talk about that for a little bit later. All these processes that we have actually caused 15 days to get to production. The whole thing, 12-month thing, was written in five days and 15 days to get to production.

07Confidence and supportability

[00:17:59.505] I asked a GenAI coding assistant again, a different model: so how good is this project? It basically said exactly the same thing that I was thinking: it's senior-plus developer skill sets, full-stack expertise, enterprise software experience needed, and bottom line, this is the kind of project that gets you interviews at FAANG companies and demonstrates you can architect and build production-grade enterprise software. Definitely brag about it.

[00:18:31.385] Here I am bragging about it, even though I have not written a single line of code in that whole thing. Now the question is, do I feel confident that without knowing anything about Neo4j, Cypher query, TypeScript, do I feel comfortable to actually send this code to production and be supported? Yes, I do. Because you know why? After five days of coding, I now know Cypher query. I actually know TypeScript now, because GenAI coding assistant, where I was working with it, helped me learn those things.

[00:19:10.225] Now I think, and my team also ran through the same process, they also feel confident because now they understand the technology that they're going to support. The whole concept that if GenAI coding writes you code, can you support it? I think in my mind I'm fully confident that yes, we can, because we learn through that whole process.

08Learnings as a developer

[00:19:33.065] My learning as a developer: I do not trust 100% any AI coding agent, period. They lie. You need to know when they lie. You need to understand that they're lying. So you need to kind of go back to them.

[00:19:46.985] Use all the scanners, linters, analysis tools that you might have right onto your IDE or wherever you are using this coding agent, because without that you'll not get the quality that you're looking for.

[00:20:00.225] Use multiple models all the time. I now use multiple models all the time because I know one model may lie and the other model can catch it. Human must be the decision maker. Because I know how to code in different languages, I have some experience that I can make decisions using those experiences.

[00:20:23.335] Never use the same model to generate code and test cases. Now, I'm a big proponent of developers writing test cases, but not with these agents. No, I will use model A to generate code and model B to write test cases. There's a validation that happens there. That's my experience.

[00:20:40.865] Use version control and commit very, very frequently. I lost code many times during this whole five days.

09Learnings as a technology leader

[00:20:48.065] What I learned as a technology leader: if you think about the whole thing, the whole Agile process created so much back pressure on ops that they started hating devs, and that's why DevOps was founded. Then security people started hating both dev and ops because they did not take care of security, so we got DevSecOps. Then the risk, compliance, auditor started hating all of these people because they thought that we didn't take care of the audit compliance requirements. Now we got them.

[00:21:21.285] Each of these things created back pressure on the next thing. This thing is going to create back pressure of all back pressure on all the things that we just talked about for the last one decade here on this stage.

[00:21:34.465] AI coding agents are here. It's not about can we use it? It's about how can I use it? Monitor the usage of coding agents all across your enterprise. Double down on your core DevOps capabilities, because you are going to create back pressure that is going to challenge all these capabilities that we have created over the last 10 years, or five years, or one year.

[00:21:59.185] Double down on your security, risk, and compliance capabilities. Embrace the change. It's going to happen. Caution, right? Coding is addictive. Ask my family. They know it.

10Help requested

[00:22:16.265] Help. I'm looking for your POV, or point of view, on GenAI coding assistants. What controls do you have or plan to have? What are the major use cases that you're trying to address? Because I think this community helped bring DevOps to enterprises through sharing, through conferring. Let's do it again on GenAI adoption in enterprise. Thank you.