Host Intro (Gene Kim)

So it is now time to introduce our first speaker, so give me a second to get my windows to where they should be.

I had mentioned the amazing talk from Parloa, and this is just one of the many experience reports that we've had from technology leaders sharing their experience of what works and what doesn't work as we're all trying to figure out this crazy new capability that's called GenAI.

Up next is Stefan Ostwald. He is Chief Product Officer and co-founder of Parloa, and he'll be co-presenting with Peter Petrovics, strategic advisor at Equal Experts.

Parloa was created in 2018 with the vision of making every customer interaction as easy as talking to a friend. They spoke last year about launching Europe's largest GenAI-driven conversational FAQ service for a multinational enterprise that has over 15 million customers in 30 countries.

I am so excited to have them back to share their journey of their GenAI aha moment that happened in 2022, before the ChatGPT moment, and how it resulted in an incubation effort that resulted in this incredible offering alongside their flagship product. I think there's so much to learn from in this journey, and I think it sounds very familiar to people in the DevOps community. Stefan, Peter, I'm so glad you're here.

Stefan Ostwald

It's good to be here. Hi, everyone.

Q&A

01Gene Kim

I introduced you, so maybe can you introduce yourself in your own words and tell us about Parloa and what customers love about you?

02Stefan Ostwald

Yes. Let me give it a brief start. My name is Stefan, and as Gene already mentioned, I love what we are able to do now. I think the world is changing right now in a completely different way. What the AI capabilities can bring is tremendous, and I think the impact on us as an individual is unique.

I think really we can make people's lives so much easier by helping, giving them a hand, providing companies personalized AI agents which can help you and resolve your topics, but really be on your side throughout your entire journey.

So I think the relationship between companies and customers can completely be changed, leading for individuals to really have a much easier life and more help along that journey. That's me. I love that topic. I love to collaborate also related with you on Slack. And this is Peter.

03Peter Petrovics

Hi, everyone. I'm Peter Petrovics. I work with Equal Experts. I joined Parloa more than a year and a half ago from Equal Experts. I never felt external from the beginning, so I'm very embedded in working together long hours on a very exciting topic.

What I really love about working with Stefan and the whole Parloa team is that we are actually working on the cutting edge, and we are bringing actual cutting-edge technology in front of customers with all the challenges, but all the fun of being a pioneer.

04Gene Kim

When we were talking before, you had mentioned cutting edge. You had described this moment in early 2022, before the ChatGPT moment. You're reading the papers and your conclusion is that there are some new extended capabilities that were likely around the corner. My heartiest congratulations to you on being right about that.

Can you share the story about what you did about it: the innovation effort that you started, how you organized it, especially as you thought about this in conjunction with your flagship product? How did you put together a team? What were you looking for? What sort of skills and personalities were crucial for this type of initiative?

05Stefan Ostwald

That's a really good question. To briefly embed that moment: all this innovation always comes where feasibility meets customer needs. We've been in that industry for quite a while, and how you structured conversations before was that you mapped out the entire conversation up front. You try to estimate how the conversation can go, but conversations are messy. They go all over the place. They go left, right. It's not that linear. Just imagine a conversation like that, mapping this out up front. This is hard. You really don't succeed.

When I was looking at the conversations which were actually happening, there was that need: we need more flexibility in that conversation. How do you get more flexibility instead of rules? You need AI to really go into that job. But for quite some years, AI just wasn't there yet. We looked left and right and tried different things. In theory it looked nice, but the capabilities were not strong enough.

Then InstructGPT came out, which was a bit before ChatGPT, which made it so popular as we all know. InstructGPT had a different approach, thinking not only creating text, but hey, you have an AI and you can give it instructions. You can make it do things. You can make it call external APIs, extract additional knowledge, which you can then leverage in that conversation.

There was like, ah, yes, that sounds like the approach to go for, which was a bad bet then because the AI models were nowhere capable to really deliver any service. We are just now actually at a state where AI is really able to handle the complexity we have in real-life situations.

But it was: yes, this is the path forward. AI is accelerating. We cannot start when AI is there; we need to start before AI is there. So when AI is there, we are there as well with the tooling all around it.

So we started to think, how can we do this? We already had a really successful product, but it was in that old rule-based world. You have your flagship product and then you think, well, there's a completely different approach which makes it so much better. How do you do that?

What we noticed is you do need a different skill set and different kinds of personalities, process, and structures when you want to start something completely new where there's no blueprint on the market at all, where you really need to be open-minded, customer-centered, and explore quickly how you can achieve this.

We thought, all right, let's get this one started. We added additional people, and Peter also joined, but really with a focus on people who don't just need to execute, but people who can observe, who are customer-centered, and iterate quickly and fast.

Also the process we had back then was way different than we have it now, because now of course we are out of production, it's much clearer what we know and how to progress. But it was a big change. Peter, you experienced that entire journey. How was it at the beginning? How was it later? What's your take on that?

06Peter Petrovics

It was really cool. I immediately felt that when we joined, it wasn't only me; it was a team of people already there and some more people from the beginning, but it was a small team. What I already felt was that we are pioneers and we got the freedom to experiment.

It was understood: this is a new technology, we don't know. We feel, we had the hunch, we had some evidence obviously, but mostly hunch that it helps. But we got the freedom to experiment and the understanding that this is a journey we are taking together, learning how we apply this technology to Parloa's needs and ultimately the customer needs.

What I would emphasize is, especially at the start of the stages, when you start to implement a new product like this, being exploratory, being open to explore, and being open to pivot very early and very quickly. I think every week we had some discovery which was like, oh, we have to think about this. How do we do this?

Mostly I think the freedom to discover is the main thing I would love to emphasize at the beginning of a new product.

07Gene Kim

When you talked, you described some things that really just jumped out at me because they seemed so familiar. You said it was a small team, they were cross-functional, they were good at working with the organization, they were well respected within the organization, they had the ability to reach out. Can you talk a little bit about that, confirm that was actually the case, and talk about why that was so important?

08Stefan Ostwald

First of all, yes, it was a cross-functional team, composed of product managers like Peter, but also engineers, data scientists, AI experts. It was really cross-functional.

Not to forget: in the end, the problems of the customers are the same. It's just a different way how we solve them. So we could build up on all the expertise, all the insights we had built up over the years. Of course, we reached out to everybody in the commercial org and the partners, really understanding what are their needs we've discovered so far and what are the additional fears and expectations they have when they put an AI in such a central position. Because in the end, if your AI talks to a customer, you do have certain expectations.

This was a lot of discovery also from a user perspective, but of course also from a technical perspective. This is the context you have. This is the mission.

In order to really build up on that mission, imagine the contrary, which we definitely did not do: start in the beginning with, well, this is our two-year plan, let's just build it. We'll come out in a year and then everything will be fine. It's never like that. You fail. You fail all the time. It is about failing, but failing early.

This is really what we set ourselves to do: get into prototypes quickly, bring them in front of people, understand what's working, what's not working, get real challenges, what people really have, and then have really fast iteration. There were so many insights, and we came up really different than we initially started. The direction was there, but we needed to learn, we needed to be open.

It's really that kind of attitude you need to bring in, especially in the beginning where there's so much unclarity around it: that spirit to the team, the freedom for the team. It's not failing. It's nothing bad. It's about de-risking your business model. It's understanding what you have. It's about validating hypotheses. A failed hypothesis is progress. This is how you measure progress initially, not the lines of code you build or whatever feature you deliver, but progress toward how certain you can be that you can really solve these customer problems.

09Gene Kim

Peter, you were telling me a story that I just loved because it was so concrete. My memory of it was that you're doing this early prototyping effort, and there's some Python notebooks, and you're trying to see what these AI models can do back in 2022. You described this holy-cow aha moment: oh my gosh, might that actually work? Can you describe what prototype really meant in a Python notebook and what exactly you saw that maybe changed your life?

10Peter Petrovics

Just for context, by that time everyone was playing with chatbots: okay, I can ask questions, it can help, can use tools. But actually we tried to apply it using Parloa experience with real customers. What is needed here?

We built a demo, basically a flight-booking agent. Then you started to discuss: okay, I want to book a flight. Then the agent asks, okay, when do you want to fly? Where do you want to fly from? What class do you want to fly? How many people? That's a normal thing. You can do this with a deterministic flow, rule-based as we say.

But when you start to stretch it, and when you start to say things like, oh, sorry, I changed my mind, I didn't mean that. Or you can say in one sentence everything you want, and the agent books you the flight immediately because it understood: okay, it's so many people, this is the destination, this is the origin, and so on. It goes to search for the flights immediately.

That's when it's like, oh God, this is actually unbelievable. That's what Stefan referred to: conversations are messy. People are not going through a logic flow when they ask for something. They ask you, telling what they want, and the LLM can understand.

That was for me the aha moment, or aha moments, actually for months. Even these days, I am sometimes positively surprised how well it can react to dynamic input and how well it can handle edge cases by itself.

11Stefan Ostwald

Maybe let me add another story. This experience was from a call perspective, but I do remember one situation where we just got into the space that we actually got a first UI. It was actually Peter who said, I won't join for lunch today. I got that thing I want to try out. Okay, let's see.

We came back after lunch, and he's like, all right, I added now a text message service, an SMS service, which was really just adding the API to it and giving it the instruction: if you want to send an SMS, this is how you do it. You give it to the AI agent.

All of a sudden, it was able to use this in all kinds of scenarios. What's my luggage information? Okay, can you send it to me via SMS? Or a confirmation. It was possible to use it all over, but the implementation was a matter of minutes.

That this capability all of a sudden was ingrained in this AI agent with so little effort is wonderful, and you can generalize this to so many things. You just give it these capabilities and how to embed this into a conversation, how to handle errors which might arise, how to handle wrong phone numbers, or ask this. All this was just done by the AI.

The builder experience is on such a different level, and that also makes me so confident that you can really get into significant automation because you need such rich company knowledge, processes, but also backend infrastructure. But the speed, how quickly you can really integrate them, is just tremendous.

12Gene Kim

So great. I think so many of us have had those moments that literally change your life. It sounds like absolutely these are stories for you. Congratulations on your bets paying off.

One of the things that jumped out at me when we talked was that these experiences are showing how the entire value proposition of customer support could be changing. You said customer support is primarily focused on maximizing deflection rates, minimizing call times. But with AI, when the cost of these customer interactions go potentially to zero, something truly exciting can happen from a business perspective.

Can you describe how entirely different we can view customer support as not just a cost, but as an opportunity?

13Stefan Ostwald

You nailed a really important point. Right now, customer service is mostly seen as a cost center. How do you get costs down? You have fewer people calling. So what do you do? You optimize for deflection, that people don't reach out to you, they don't have a connection with you. You bring in these weird FAQ pages which never really help you. You might hide your phone number on your website so people don't call you. It's because it's seen as a cost center.

AI now brings, as it has such a different cost structure, all of a sudden the costs are not so much connected to these conversations. You can really embrace the power of this connection because in the end, what you want is a brand. You want brand loyalty, you want brand success, you want to be there with the customer, you want to understand the customer and help them, you want to advise them along your portfolio or how to connect with other things. You want to be there with them, and you want to build up that connection. That connection you can build up through these conversations.

The more you can actually provide your personal AI agents who know your context, who are there for you right away, you don't have to wait in any lines, are there in all the channels as you want, and really can help you with your problem, your understanding, also with quick questions, it becomes a different relationship. It becomes a different relationship to your brand.

With that brand, of course, they are not only more loyal, but that translates into more revenue. So from a cost center, actually transforming into a relationship center which leads to also more revenue. That shift is what we see, especially in upper management who have a more holistic view of the company, that there is really something fundamentally changing about customer relationships here.

14Gene Kim

That is awesome. By the way, I think that's one of those surprises where it sounds so contrarian, so crazy, that I think it will change so many people's minds.

One of the things that we talked about before was that AI is moving so quickly, models are changing quickly. Can you talk a little bit about your efforts to preserve optionality so you can take advantage of these advances, potentially regardless of what vendor they're coming from? Talk about those challenges and opportunities. I think this is something that every organization is facing who is in this field.

15Peter Petrovics

In general, we are always joking: do a three-year plan in the AI world, when the whole area is less than a year. When we started, it was less than a year in the LLM or language model world.

When we started, we were always looking for all different models. There wasn't anything really better, anything close to feasible compared to GPT-4. So it was a very easy decision at the beginning: we keep going with this. That's what we need. Obviously we have a big latency, a real-time latency challenge, but we had to overcome those challenges in multiple ways.

But we always kept an eye on what's going on. These are the days when we are getting real alternatives for OpenAI. It's getting more commoditized, and we are having open source models actually getting even overperforming and also other commercial models.

In regards to connecting to these models, at the beginning it was chaos. There are different ways how you actually interact with LLMs. I think now nowadays it's getting almost a de facto standard how you interface with LLMs. There are different dialects which obviously can be handled by different ways, but it's much easier than in the beginning.

What brings our biggest thinking and our biggest current in-house innovation work is obviously multimodality. It's already there. It's still not there to be productionized, but we are on the cutting edge as usual, so we are already working on these advancements. That's probably our current biggest challenge. That's quite a big shift of the whole thinking: how you think about interacting with the LLM when you're suddenly not text in, text out, but voice in, voice out. Then it brings a lot of questions which we probably don't have time to go into, but again, there are always good challenges because of the developments.

16Gene Kim

You said something that I thought was so provocative: ultimately maybe you let the customers choose which model they want to use, which I thought was so provocative. What are you most proud of in this journey so far, and what help are you looking for from this community?

17Stefan Ostwald

Overall, what we're most proud of is when we read customer feedback, when we have quality assurance along with our customers and they show us, look, these are the awesome calls and this is the great feedback we are getting, which comes from our customer directly, but also from the call center reps.

To me, that's the most exciting thing. They say it's so much better when people get forwarded, because if our AI agents cannot solve the problem, then it's forwarding that call to a conversation, so you can incrementally, smoothly extend into that area.

The call center agents are saying, ah, it's so nice; people arrive much calmer because the agent is nicer. The NPS went up 180% of the customer. This is really touching to me because of course it's nice to have a vision, but seeing that in reality and getting real customer feedback around it, this is of course really what touches at least my heart. That's definitely what I'm most proud of.

Overall, what's going to help with this? I think it's a wonderful time. It's a massive shift in all industries. We believe the people who are adopting AI early are able to accelerate along with AI. This is really what in each industry will define who will be the leader of the next era: who will adopt AI early.

We love to work with winners, existing winners, but also with future winners. We believe that AI is a big milestone, and we love to collaborate with people who want to bring their customer relationship to the next level. That's something great we like to partner up with.

18Gene Kim

Super. Thank you for sharing so much of your learnings. I'm hoping that you'll be able to share your story when we all get together in September in Las Vegas. Fantastic. Thank you for teaching us things that I think are relevant to all of us. Looking forward to actually hopefully meeting you in person soon.

19Stefan Ostwald

Thank you. Thank you, everybody.

20Peter Petrovics

Thank you. Bye-bye.

21Gene Kim

Bye. Thank you, Stefan and Peter.