Stephen Magill

That was great. So after Dr. Kirsten's talk on what this future might look like, where AI is doing so much work for us, and then Patrick Debois's talk this morning on all the amazing things that AI is already doing, I get to kick off the ChatGPT Ignites by talking about what's in it for us. Is this really going to lead to a future where we have to worry about: will there be a DevOps conference in two years, or will dev and ops and everything that we and millions of developers do just be automated?

So this is how to stop worrying and start adapting to the rise of AI in software development. That completion was generated by ChatGPT. Could be a little shorter, could be a little catchier, I think. So maybe don't fire your marketing department yet, but hey, I didn't have to do it. It saved me some time. So yay, AI efficiency.

I want to start with everyone's perception. We're seeing the vision of AI and what it could do, and it looks a lot like this: you can just hand a laptop or API access to an AI agent, maybe you give it access to a bank account, whatever resources it needs, and it will generate software and technology that humans are currently doing. The reality is much more like this right now: the human is still in charge. They're setting the agenda. The AI is maybe this robotic doggy here. It's being pulled along, providing assistance and companionship and helping out, but it's not in the lead. This art was also all generated by DALL-E, and I love what it did with the eyes here, this dead-behind-the-eyes look. You can really feel the existential dread this programmer is experiencing.

I want to start with some of the limitations. Where are we now in terms of what AI can do? One of the limitations of these large language models is their ability to reason. They can fill in the gaps. They can do this generation of small pieces of code. But when it comes to solving larger problems, that's where we run into issues.

Here I posed a task. I've said, I lost a ball under the sofa. I need help getting it out. I have some items available: hockey stick, jar of peanut butter, some string, this cute little unicorn stuffy. How can you solve this problem for me? ChatGPT will provide a solution. It says one solution is to tie the string to the hockey stick. All right, that's a good start. Then smear some peanut butter on the stuffed animal. Interesting choice. Let's see where it's going. Next, place the stuffed animal with the peanut butter under the sofa close to the ball. The scent of the peanut butter should attract the ball, causing it to roll towards the stuffed animal.

So it's a novel solution. It's probably not going to help here, but it's a great visual. In fact, I asked DALL-E to generate a visualization of this, and it came up with this, which I think is great. I think that's a great summary.

Of course, that's not the end of the story. These are interactive systems, and so you can provide feedback and say, actually, balls are not attracted to the scent of peanut butter. The AI will say, you're right, I apologize for the error. Balls are not attracted to the scent of peanut butter. Then it generates a reasonable solution. We're seeing the AI can do certain things, but it requires someone to check and guide it.

Even with these limitations, AI is having a big impact already in software development. I want to present a report from the field. This is from some folks I know at Fossa, which is a company that produces software for automated governance. They've been using AI in their business for several months now, and they report a 5x improvement in productivity, which is big. That's substantial.

What's interesting is it's not just from code generation. Yes, you can use these systems to generate code, but they also find them useful in customer service, learning about technologies that a customer might have deployed in their environment that they haven't encountered before, or ideation and collaboration, exploring architectural ideas, sort of a brainstorming process with a generative AI, or learning about new tools and frameworks. Maybe you need a library to generate PDFs and it needs to be able to handle text and images. What are the options and how do they work?

What's interesting about these use cases is they also benefit junior developers. That architectural discussion is usually something that a developer would have with a more senior developer. Now maybe they can have that conversation with an AI agent. I've talked to people who are worried: in this future, yes, developers are still needed, but do we just need the senior developers and architects, and they break down problems, and instead of handing them off to a junior dev, they hand it off to the AI? I think this shows there's a role for developers at all stages of their career here, which is good because you can't get senior developers unless you start with junior developers.

The end report is that their biggest challenge right now due to using this AI is a shortage of critical thinkers that can guide GPT to the right decisions. They need more engineers, not necessarily to write more code. The tasks of these engineers are changing, but because GPT can only scale productivity as long as it has adequate guidance, there's still demand for developers in this world.

But five times increase in productivity: does that mean we're going to just need like 20% of the developers that we have now? To answer that question, I want to talk about something called Jevons paradox. Jevons was an economist in Britain in the 1800s, and he was looking at what happened during the steam revolution. Steam technology had just come about, and there was a huge advance early on in the steam engine technology when James Watt introduced his version of the steam engine. Steam power requires coal to generate the heat. That's the primary input. The Watt steam engine was five times more efficient in its use of coal. Suddenly you could do the same amount of work with a fifth the amount of coal. The expectation was: coal usage will now decrease.

This was good because coal was viewed as a fairly limited resource at the time. They discovered more deposits after that. But what actually happened was coal usage increased. The efficiency gains meant that steam power became more cost effective. It was now cheaper to run a steam engine, and so that opened up the technology, made it cost viable in more industries. It went from usage in mining to then usage in textile mills, smelting, and other industries.

The reason this happened was the demand curve, going back to freshman economics, if any of you took those courses. The demand curve for steam power is elastic, which means that if there's a small decrease in price, there's enough demand out there at that new price to more than make up for the efficiency gains. The question in terms of what will happen with software developers and demand for developers boils down to a question of: is there elastic or inelastic demand for developers?

I think the answer is we have elastic demand in this industry. I'll give a few points here. One is looking at the startup ecosystem and venture capital. Andreessen Horowitz is one of the top venture capital firms in Silicon Valley. They invest in only 0.7% of startups today. If you look at all the startups that apply, 0.7% get funded. If we suddenly become five times more efficient at producing software, you could imagine now maybe they can fund 3.5% of the applications they get. I'm sure there are 3.5% of those applications that are worth funding to see where that idea goes. Similar story for Y Combinator. They have a very low selection rate. You can imagine increasing that selection rate and getting a lot of value from society being able to explore more new ideas.

The average backlog for planned IT projects is three to 12 months in most companies. That's another place where they can soak up new capacity. Another thing to point out is this is not the first large increase in productivity that we've had as developers. Open source is 70 to 80%, or sorry, 80 to 90% of modern applications consist of open source. If you think about the world before open source and the world now, developers are writing a fifth of the code that they used to to get an application out the door due to the availability of open source. That's a 5x productivity boost. That's what we're seeing, at least in Fossa's reports, from the usage of AI.

We didn't have a bunch of out-of-work developers after open source took over. Demand only increased. I think the thing to think about at the end of the day is: could your company benefit from being able to write more software? I think for most of us, the answer is yes. Developers are a resource we would love to have more of, and we could make productive use of that. So think about that, and hopefully that helps you sleep better at night and believe that we'll all be here again next year and for years after. Thank you.

Mitun Zavery

All right, let's get started. Hi, my name is Mitun Zavery. I'm a Sonatype solution engineer. Today I'm going to talk to you about Prompt Kitties and ChatGPT. We all know how good ChatGPT is. In fact, ChatGPT got me into this Ignite talk right now, so the abstract was actually ChatGPT generated. Let's look at that.

What is the modern face of security? If this would work, that would give me time. Let's try again. The clicker is not working. Do I get a restart here? Does someone want to come and fix my clicker? Thank you.

All right. What does the modern face of security look like? Back in the day when I started this, that's what I thought it was: Swordfish, fantastic hacking, Matrix-style terminals. That's not the reality. The reality is bad actors prefer simplicity. They prefer effectiveness, user-centered design. In fact, script kiddies, as we know that term in the industry today, have malware as a service. You can see some of the examples there. You can actually just buy this online using Discord tokens. It also has its own support channel as well. You can obviously tell by that, though, and the screenshot, that infinite support and lifetime support doesn't exist in the malware community.

So what are script kiddies? With easy access to code and solutions out there in the world, kiddies or young individuals are actually using the readily available code to create these malware attacks and use that for bad purpose. They are not always young, though, and they are not always inexperienced. Some of these groups are actually led by very senior technical individuals.

I represent a business that makes a living actually detecting malware in the software supply chain in the open source community. This is what we've done for a while. Our researchers are known for this. Let me tell you a little story. I call this the cat-and-mouse story.

What you're seeing on screen is actually a vulnerable library. It's called Reverse Shell. It was created by a group called Syntax Theo, and our research teams and our system detected potential use of this code inside of the open source community. Really what it did is ran a setup.py, ran a bypass.py script that then created persistence in your Windows registry and stole things like credit card information, took a screenshot of your desktop, those sort of things. Developers were also using this.

Now, we detected within the system that there was another package that was there. It actually has a different name, and it was completely obfuscated. The interesting thing is, with a little bit of work and a little bit of smarts by our researcher, the few lines of code that you see highlighted, the plaintext, "What are you doing reading this?" were actually the keys to de-obfuscate this code. We were able to see that it was exactly the same package as the previous Reverse Shell package, just slightly different.

Interestingly enough, obviously the GitHub repo was taken down, and so this package was removed. The reality is actually a few days later, this reappeared as something completely different on another GitHub repo, and it does exactly the same thing. It downloads Windows Defender.py.

At this point, our researchers thought, there's got to be something fishy going on here, and actually, I reckon within a few days we're going to see something similar up here. Guess what? There was. It was actually a clone of the Discord remote access Trojan, and it was exactly the same code. We realized immediately, just by that moment, that the lights went on in our research team. This was just the same attack. It was being cloned, and we had a feeling it was probably using generative AI.

Guess what? It was evolving. It actually had new capabilities. It even had a menu function as well. So it was really generative AI that was doing this. With a little bit of detection, a little bit of research, we actually go and have a look at ChatGPT. Our researchers ran a prompt in ChatGPT telling it to generate the code, and it was exactly the same code that existed in the old Windows Defender.py file. You can see it was easily generated.

What we have found is live malware in the open source ecosystem that existed and continues to be replicated inside of the ecosystem. This is being done by, ladies and gentlemen, prompt kiddies. That is a new attack vector that your DevOps teams have to deal with. It is a new angle of attack by inexperienced developers that have access to ChatGPT. If you want to learn a little bit more, read the entire blog on that link there. You also have access to a software supply chain report. Also, prompt kiddies do exist, ladies and gentlemen. It is there. It is a new attack vector. If you want to learn more on how to protect yourself against this, come over to the Sonatype booth at 100. Thank you very much.

Ivan Krnić

Hello everyone. My name is Ivan Krnić from CROZ, and I'm here to inspire you to throw a company-wide hackathon based on generative AI.

Every once in a while, a new technology comes along with the capacity to change the world, and generative AI is definitely one of those technologies. With generative AI, it's like with any other technology. In the beginning, there is a lot of uncertainty and our confidence to use the technology is very, very low. But at some point, these two lines will cross, and when these lines cross, then it's time to rock and roll.

We feel like hackathons are a very good way to build this knowledge about technology. How do you make these two lines cross? You make it by testing the technology. You make it by experimenting, by learning about the technology. The best way to learn about technology is actually to do a hackathon. This is what will make you truly masters of the technology.

Hackathons shouldn't actually be serving their own purpose. They should be part of a wider strategy. This is the approach that worked for us. We first defined the strategic goals and objectives: what we want to achieve with this generative AI hackathon. Then we threw a company-wide internal hackathon. Step three, we gathered all the ideas, evaluated them, and prioritized them. Step four, we developed the roadmap in terms of what shall we do further with these ideas, but also with the technology itself.

Let me demystify step two, hosting an internal hackathon, how that looked for us. We first put together an organization team, and they were in charge of everything that was going on around the hackathon. They did an introductory workshop. They set the stage. They set the date. They prepared all the logistics. They also encouraged cross-functional teams to apply. They prepared cloud environments. They set up our API keys. They also put in place some API budgets because we wanted things not to go out of control. When everything was done, the teams documented all those results in a wiki page. They defined new policies. In the end, we started with new initiatives.

So how did this all look? The energy was amazing, and there were many more people than we expected that would show up. In terms of numbers, we had 125 people in 36 teams. They were in four locations. They ate at least 70 pizzas. We learned today the importance of eating pizza. After God knows how much beer, they came up with 36, 37 ideas that we were pursuing further.

We won't pursue all of them further, and I won't talk about all of them, but I wanted to briefly mention three of them. The first one is from podcast MP3 to social media post. This was an attempt to help our marketing team put together content for social media posts. They took one episode of my 0800-DEVOPS podcast. They threw it into Whisper. Whisper created a transcript. The transcript was fed into GPT to create a blog post. This blog post was put into Midjourney, which created a visual, and basically the marketing team had everything they needed to put out a social media post.

Another example was an AI support agent. How cool would it be if a normal person could pick up the phone and ask a question, and then an automated support agent would answer in the same human language? This is what one of the teams tried to achieve. They tackled the problem from the middle. They put a customer frequently asked questions database into GPT, and they asked various questions. This turned out also very nice, so they almost circled the whole loop.

The third attempt, the third team, what they did was a Q&A sales bot. This is basically what the sales team did. They took anonymous procurement documents, put them in ChatGPT, and tried to ask different questions to see what kind of value could they get out of it.

In terms of key takeaways, this is not a New Radicals song: you get what you give. In terms of the results, the quality of results that you get from ChatGPT is basically the same as the quality of questions that you ask. We understand that this is an evolution, and this internal hackathon was only one small step. We want it to feel natural to everybody in the organization. If you have some doubts, please don't. Start as soon as possible. Uncertainty will almost certainly arise, and whatever you do, be sure that you'll be positively surprised with the results, because we have been. Thank you so much for this. If you have any more questions, please feel free to contact me. Thank you.

Johannes Nicolai

All right, welcome everybody. I thought this is a DevOps conference, a real DevOps conference, so let's do some live coding on stage and show how to automate ChatGPT and all the other APIs that come with OpenAI. Doing a live demo on stage, what could possibly go wrong? Fingers crossed and praying to the demo bots.

My name is Johannes Nicolai. I used to work most of my career for GitHub, where I supported developers in their workflows. I was also closely working together with the Copilot team there. Nowadays I work for Postman, which is kind of the GitHub for APIs. There, I'm one of the main contributors to the OpenAI collection that does not only contain the ChatGPT API, but many other fun APIs like DALL-E, which you have seen, where you can generate pictures based on textual descriptions, or Whisper, where you can turn sound into text.

Without any further ado, I would like to use these five minutes for a live demo. If you wanted to follow along, all you would need to do is to get an API key from OpenAI. That is pretty straightforward. You also get $18 to test out the APIs, and all the slides you will have access to as well. If you wanted to do that demo by your own, searching for the specific examples we go for, we would just go into Postman and search for fun, AI, and Lego, and then find our workspace, which is called Fun with Lego for a reason.

The very first API I wanted to show you is the DALL-E API. We are asking it to create an Amsterdam train station made of Lego. While this is sending the request to OpenAI, you can also see that whatever programming... Wow. I'm not seeing anything on the screen other than, I guess, the first page. That's interesting. Let me see whether I can change that. Did anything else show up? Doesn't look like this. Now it's... That's the beauty with live demos, right?

Okay, that looks great. From here, we see Postman now, right? For all of those examples, you could just use the programming language of your choice and get the code generated for you on how to interact with those APIs. The first one was DALL-E to create pictures based on descriptions. That's probably something most of you have already seen.

So let's do something more advanced. This is the Postman astronaut, or mascot. We actually want to replace its face. We just upload a second picture, which does not have the face anymore. Then if we go back to our collection, we would say, let's do a new request where we are actually editing that image. We have the first picture, the second picture, and then we want to replace the face with a smiling Neil Armstrong. Let's give this a try.

Postman also has the ability to visualize the example. Any kind of API response you get, you can turn into an HTML preview, which is nice if you want to visualize something like pictures. Let's see how Neil Armstrong looks like as a postman now. Here we go. Pretty impressive. This is about DALL-E as an API.

Another API OpenAI provides is actually Whisper, which we have heard in some of the talks before as well, which is turning audio into text and also does language translations. I'm trying to have Linus Torvalds say how to properly pronounce Linux and see whether the sound works. Okay, I'm trying this. "Hello, this is Linus Torvalds, and I pronounce Linux as Linux." So let's use that MP3 and actually send this to the OpenAI API. Here we go: "Hello, this is Linus Torvalds and I pronounce Linux as Linux."

This wouldn't be a complete talk if we don't actually show how to automate the ChatGPT API. We have an example there as well. Often in the DevOps world, you find some kind of Terraform script or SQL code, which you don't know at all what it's actually doing. This is the example here. We want to have ChatGPT act like GitHub Copilot and tell us what this very complicated SQL statement is actually doing here. We are sending this request and wait for ChatGPT to actually show us in human visible language, understandable language, what this will be about.

While this is running, again, you can turn any Postman request into any programming language example of your choice so that you can also have it as part of your DevOps pipeline. Here we go. This is a long explanation what it's doing, but it's still pretty much clear in comparison to the original one.

If we go back to the slides, sorry for the technical presentation problems, but most of the live demo worked, which is great. We also implement OpenAI as part of Postman itself. If you see any kind of API that gives you a response and you want to have all the test cases automatically generated for you, we just announced Postbot AI. If you want to learn more about that one or how you can use Postman in the enterprise, just stop by at our booth in the exposition hall. Thank you.