Please ensure Javascript is enabled for purposes of website accessibility

Raj Koneru Podcast Transcript

Headshot of Raj Koneru

Raj Koneru Podcast Transcript

Raj Koneru joins host Brian Thomas on The Digital Executive Podcast.

Brian Thomas: Welcome to The Digital Executive. Today’s guest is Raj Koneru. Raj Koneru is the founder and CEO of Kore.ai, a leading enterprise AI platform helping organizations build, deploy, and manage AI agents at scale. A serial entrepreneur and technology executive, Raj has extensive experience building and scaling enterprise software companies. 

Well, good afternoon, Raj. Welcome to the show.  

Raj Koneru: Well, thank you very much.  

Brian Thomas: You’re very, very welcome, my friend. I appreciate it, and I appreciate you juggling calendars, time zones. I know you travel a lot, New York, Orlando. I’m in Kansas City. So thank you again, my friend. I appreciate it. Let’s just jump right into your first question, Raj, if you don’t mind. 

You founded and led Kony, Intella Group, Serenova, and iTouchPoint before founding Kore.ai in twenty fourteen. So what lessons from those companies shaped your bet on enterprise AI?  

Raj Koneru: Well, I mean, there’s a lot of lessons, but I suppose a founder’s journey is you just jump in. Every wa– every set of waters are different, and you figure out how to swim and, float and then swim, and then swim really fast, basically. 

So, there’s no playbook for this. Obviously, experience in hiring, experience in building a business plan, executing towards the business plan comes in handy. But every business plan is different, every team is different, every company is different. So, there’s some experience that goes with it, but mostly it’s, you figure it out as you go  

Brian Thomas: Completely understand. Entrepreneurship is not for the faint of heart. I’ve had hundreds of entrepreneurs on the show, and they talk about just really finding their path, finding their way, and sometimes you learn along the way. So I appreciate your insights. You’ve done quite a bit in the space of entrepreneurship, so thank you. 

Raj, with different parts of an enterprise adopting AI independently, how should leaders handle AI sprawl without making every team switch to one framework?  

Raj Koneru: I don’t think that’s realistic. I think when you think about enterprise AI, you obviously have the infrastructure, you have the models, you have the frameworks and the platforms, and then you have the actual use cases that enterprises are trying to achieve. 

So almost all enterprises, I wouldn’t say a hundred percent of them, but majority of them have sort of democratized how these agents are built and deployed. So they wanna control what’s called the control plane, the governance, the observability, the security access to the models maybe some analytics and things like that, and set some standards. 

But they want their teams and their departments to conceive of AI agents because they’re the domain experts, and they’re the ones with the problems can create the solutions. So very few large enterprise customers have said, “Hey, we’re only gonna build it with this one framework,” basically. So, so I think it’s gonna be, a pretty heterogeneous environment. 

And overall orchestration is a challenge because now you have these agents built by different people all over the place. Permissioning is a, is a challenge. So overall orchestration to be able to execute a strategic business process is something that like, people are thinking about. But right now the sprawl is al– has already happened and is happening in real time. 

Brian Thomas: Yeah it’s a fine balance. Obviously I’ve been in there in the middle of that as well. You’ve got, people that really adopt the technology really fast, and they’re out there on their own, and they’re like the pioneers of the organization. At the same time, you’ve got your CISOs, security trying to, watch things. 

You’ve got legal and compliance trying to watch things, and it’s a difficult balance for sure. But it’s something that we’re learning along the way, and I appreciate your insights. Raj, Artemis is designed to help enterprises scale agentic systems from a few agents to hundreds. So what breaks first when organizations try to scale like that? 

Raj Koneru: Well, there are many things, right? Obviously, security is key to anything. So, there are guardrails, there’s encryption, there’s data that, like you sent to a, a model, a frontier model, let’s say, where you want some of the personal information to be, to be sort of, masked, if you will. 

But yet be able to get the responses that you want from it. But overall, beyond the security element of it it is the design of these systems that matter the most. This is the new paradigm. Multi-agent systems are not something that we are used to. So you have an orchestrator, you have different patterns for the orchestrator. 

You give an instruction to the orchestrator and then you give instruction to each of the child agents, and the handoff from one agent to another with context is where all of the design comes into play, essentially. Now some of these agents may be outside of your system. You may be using A2A to be able to communicate with some agents that are outside, so those agents have to be registered with orchestrator as well, with descriptions on what those agents do. 

Obviously, context handling is the most important. Filling the context is key with the– with not too much context, but the relevant context for the agent that’s receiving receiving the input to be able to process it and provide the right output. So I think overall, all enterprises, all vendors, all partners, SI partners or otherwise, are learning how to do this the right way. 

I don’t think there are best practices yet. There, there are some systems that get done really well and some don’t, essentially. So what we are learning is you build a multi-agent system however you build it- The ability to catch failures earlier in the use of the system so that they don’t propagate is a key thing. 

So for that, in Artemis, we’ve created a language called ABL. It’s a DSL where it’s a set of instructions where there’s both deterministic execution as well as reasoning-based execution coexisting, essentially. So you use the best of both worlds. You enforce business rules with deterministic flows, but you use reasoning to be able to generate the experience, make decisions where the models can assist in making, doing analysis or making decisions or producing a response. 

That’s another key element. But what we’ve learned is even though you get the design right, even though you have something like ABL, even though you use the models to generate the ABL for the agents, models drift over a period of time. Your input, which is usually an SOP, is never comprehensive, so you need an optimization capability. 

And we created something called Auto Loop. Auto Loop basically from the evals that are generated from our agent called Arch, Auto Loop simulates the kind of users that will use that system and find the failure points. And then based on the failure points, it regenerates the multi-agent system automatically. 

A human can review it before the regeneration happens. But also, most importantly, after production, in production, Auto Loop reads the s- the state traces, the traces of exactly what happened at every point in time, and then it identifies the failures, and then again, regenerates the system. So think about a situation where you’re expecting a certain outcome, let’s say thirty percent or forty percent automation, and then Auto Loop automatically brings that up to forty percent, fifty percent, sixty percent over a period of time. 

So it does the hill climbing basically by recursive learning, essentially. And that I think is gonna be more important going forward than just the ability to create an agent  

Brian Thomas: That’s amazing. Thank you. And you’ve unpacked quite a bit there. Obviously, you talked about the importance of in order to scale, obviously there’s design, context, orchestration. 

There’s a lot that goes into that. You talked a little bit about ABL, of course and what I really took away from here is that recursive learning where obviously there are things that are gonna go wrong within a system, and it’s good that we do keep the humans in the loop but having an agent that’s able to self-correct and learn through this recursive learning process is important. 

So thank you. Raj, the last question of the day, if you could share with our audience, if we look ahead to the future, right, three, five years, how do you expect multi-agent systems to change how enterprises operate and serve customers, and what should leaders prepare for now? 

Raj Koneru: To serve customers, I mean, there’s multiple things, right? There’s product development or service development. Multi-agent systems would be very valuable in the design of that product or design of the service. Then there’s the marketing function. Obviously, AI is being used heavily now in generating content, developing demand generation plans, developing the promotions and the ads and everything else. 

And then there’s the sales piece of it, to be able to actually, transact with the customer. And very importantly, I think where all of the things that I just said, basically, adoption is still early in those functions. Where adoption is heavy is in the post-sale customer service function because there’s a lot of data. 

You know why your customers are calling you. You know how to solve that problem already. How do I do this more efficiently is the question, and get the highest amount of automation that I can get the best CSAT rating or the NPS score I can get. So I think therein, there’s a lot of competition in that space in our industry, but we have been doing that for ten years. 

The largest deployments of customer service AI on the planet are on our platform. And we have seen it all. We’ve seen all– I mean, we run like a billion customer service interactions for a large pharmacy. Basically, we run four hundred million interactions for a large bank, for example. And the issues that come up, are varied. 

It’s, it goes from security to scale to addressing the eighty/twenty rule. The– Getting the eighty percent right is easy. Getting the re-remaining twenty percent right is hard, essentially, and that takes time. So something like Autoloop makes that much faster, basically. So I believe that over time, enterprises will learn a lot more because of things like Autoloop. 

Because it surfaces what’s not working and why it’s not working, and it also surfaces how to fix it. So you don’t have to be a Comfortable or, content, I should say, with just 30%, 40%, 50% or even 70%, automation. It can also capture… I mean, the agent can also capture customer sentiment, and the agent can improve itself based on the sentiment. 

So the improvisation loop, the optimization loop, is the key, I think, to automatically learn how to service customers better and do that at warp speed because the system is doing it for you, and it’s not humans having to figure it out.  

Brian Thomas: Thank you. Appreciate that. There’s a lot that goes into this, obviously, and we wanna get better. 

You talked a little bit about the efficiency and how we could be more proactive, and we’re getting better, but you highlighted the fact that you’ve been doing this for 10 years. You know what you’ve seen. You’ve done this stuff at scale. And of course, over time, this stuff will only get better, but you did talk about Autoloop, and it’s pretty cool knowing that those problems that will, because of this process and the technology, the problems will rise to the surface, not only, highlight them, but also provide that solution as well to resolve. 

So I really appreciate that. And Raj, it was such a pleasure having you on today, and I look forward to speaking with you real soon.  

Raj Koneru: No, thank you so much, Ryan. Have a good day.  

Brian Thomas: Bye for now.

Raj Koneru Podcast Transcript. Listen to the audio on the guest’s Podcast Page.

Subscribe

* indicates required