Dev Rishi: [00:00:00] I've spoken to about 180 different organizations. 12% of organizations said we haven't even started, but the vast majority, 88% had at least started. Definitely a lot of heavy experimentation.

Ashish rajan: A lot of people still plug agent behind a chatbot and call it agentic as well.

Caleb Sima: I have 3000 startups in stealth mode right now that I have been pitch.

Every single one of these guys all comes down to, oh, we'll just do some sort of anomalous behavior analysis, which is complete failure right from the start. You want to give a non-deterministic

Dev Rishi: model with a non-human identity, right? Access to our process, system system, right?

Ashish rajan: I don't know how many m ps are running at any given point in time to my organization,

Dev Rishi: but the agent makes the wrong mistake.

It's just really too early to know all the different ways agents can operate. You're gonna need a remediation layer. Again, the assumed breach mentality. The concern I have is these agents are getting built all over the place. How do I actually have active real time enforcement of guardrails that I want?

And what do I do if something goes wrong?

Ashish rajan: If you have been trying to unpack why is it that AI is not being able to made into production well possible. One reason is the fact that people cannot rewind, and I [00:01:00] say that in no pun intended, the changes that are made by LLM, and obviously there's a lot more to this conversation.

So fortunately I had a chance to speak to Dev Rishi, who is from Predibase, which was acquired by a Rubrik, and we spoke about some of the reasons why there are concerns about trust. What kind of organizations are actually going into production with GenAI. What was the difference between deep learning and GenAI use cases?

Why is it that it is a problem today that we cannot stop a AI bot or AI agent? Be able to just go away, drift away from what we are putting as a policy and how to solve that as well. All that and more in this conversation with Dev, myself and Caleb. If you know someone who's trying to unpack how they can go into production with ai, what are some of the challenges they would come across and some of the things they could do?

From a work procedure and paper perspective and what could be slowing them down, definitely share this with them. And if you are here for a second or third time and have been enjoying episodes of AI Security podcast, I would really appreciate if you take a quick second to drop that swallow, subscribe button, whichever platform you're listening or watching this on, whether it's Apple, Spotify, LinkedIn, YouTube, we Are Everywhere.

Really appreciate you taking that. Second, it [00:02:00] helps us grow and get more amazing guests as well. So I hope you enjoy this conversation. I'll talk to you soon. Hello and welcome to another episode of AI Security Podcast. What a guest here, Dev Rishi Heman, thanks for coming on the show. Of course. Yeah. Thanks for having me.

Maybe to kick things off, if you can share a bit about yourself, where you are and what you've been up to.

Dev Rishi: Yeah, definitely. So, I'm, uh, Dev Rishi You know, my, uh, official title thing is a GM of AI at Rubrik. Prior to this, I was co-founder and CEO of my own generative AI infrastructure startup called Predibase

Uh, we helped organizations really tune models for high accuracy, lower cost, and efficiency. And deploy them in large way, uh, largely for high volume use cases, so a large number of both leading Silicon Valley tech companies as well as enterprise customers. And we joined forces with Rubrik this summer.

Really excited to be leading a lot of our AI initiatives here.

Ashish rajan: When you say building L, so is this like a non foundational model, LLM.

It's a good question.

Dev Rishi: So we actually started the company in 2021, which was, you know, about a year and a half before [00:03:00] the ChatGPT wave. And we started with the previous generation of models, which were deep learning models, still transformer based architectures in most cases, but the most popular use case in 20 21, 20 22 of our platform.

Was fine tuning and deploying a 300 million parameter model. Now, you know, what ended up changing was really the same types of primitives but scaling that up one, two, or three orders of magnitude. And so it shifted from not, uh, from deep learning models to being pretty heavily involved with foundation models and small language.

Caleb Sima: Uh, so it was fairly common, obviously before sort of, the, uh, generative ai, big foundational models for I think these corporations and enterprises to take these smaller, unique use cases, fine tune neural networks or machine learning models to do this. Ever since the sort of, you know, open ais and Anthropics and Geminis of the world have come [00:04:00] out, do you see, like, what do you see as the trend in enterprises?

And I'm just gonna sort of like take a stab, but I'd love to hear your have you seen a lot of these guys a abandon, some of these models moving more towards the large language models? Do you see even enterprises? Thinking about fine tuning these models themselves, like even the open weight models that these guys have released how often what, just give us sort of like both velocity, volume and how you're sort of seeing that happen.

Dev Rishi: Sure. Yeah. You know, I think that the first thing is exactly what you said, which is it used to be that you had to train a model before you could start to get any value out of it. I think the large generative pre-train trade former models really changed that on its head. You could prompt it and it'd start to give you responses back.

We see a pretty defined trend, which is, especially in the enterprise organizations, you first start just prompting the model and then you prompt engineer, you eventually hook up something for evals and evaluation so you can get a numeric sense of how those models are doing. And then [00:05:00] it becomes very use case dependent.

For some set of use cases, especially as organizations are trying out, especially let's say lower volume inference, if you're only running one query per second or below, you're generally okay pinging the foundation model uh, directly, especially if you're getting the accuracy want. But then we see a set of users that need the next level of accuracy.

They were able to get the 70 to 80% prompt engineering, but they need 95% plus for where you need to be able to get. Or they need to do more with less. They want that, you know, accuracy. The large model can give you, but at a 10th of the latency or at a fraction of the cost because the use case only really makes sense With those dynamics in place, that's where you, we see organizations start to fine tune typically some of these smaller models and smaller open weight models so that they can run really high volume inference.

At a much more cost effective, latency effective or accuracy range. Now, fine tuning is still something that requires a lot of effort from organizations in particular in doing the data prep side. And so it [00:06:00] tends to be something, you know, if you were asking me to approximate, I'd say like probably like 80% of use cases plus that I see people are rolling out with fin uh, you know, foundation models that aren't necessarily tuned.

But for the really high value, really high volume use cases, that's where they end up starting to tune and go into more efficient models.

Caleb Sima: And if you were to just think of, gut, finger in the air, what are the use cases? Are they still stuck in sort of chatbot land or are you seeing real.

What are they? Sort of give us some. Yeah,

Dev Rishi: no, I think, uh, look, in 2023 when we and the LLM wave really hit, we spent a long time determining how we wanted to adapt our platform for training and serving deep learning models now to large language models. So when a lot of customer conversations and asked people their use cases, the first half of 2023, the most common use case I heard was, I don't know, my use case.

Which, like building a startup in that environment can be a little bit, uh, stressful. Uh, but by the end of that year in 2024, a lot more use cases came out and there were rag plus chat, like retrieval, augmented generation plus [00:07:00] chatbot use cases.

Ashish rajan: Yeah.

Dev Rishi: One way you can think about it is it was like really search on steroids with my own corpus.

Uh, it was really kind of where people were geared towards. I don't think that's where people are realizing the most value from generative AI today, though I think the most value from generative AI comes when you start to plug in the models in something that's more of like an automation oriented use case.

Something where we take like a manual repetitive task and you start to actually have the model be able to go run through that. So, uh, one of the examples of this is we worked with an organization that does background check for employees, um, all over. And so they used to go through documents and document processing, criminal record charges and others.

You need to be able to extract out these fields exactly the type of thing that models are really good at reading comprehension, extraction, uh, and being able to operate on that. So they shifted those towards models that were tuned with Predibase, uh, you know, another example. Are, uh, companies that need to be able to process call center and call center analytics.

Um, not just for being able to go and give you like the [00:08:00] suggested response, but how's the tone of this customer? You know what, there's 64 different indicators this company actually assesses, um, from like tone likely to follow up or competitors mentioned, being able to pull all of that out, I think that better chats and better search are just the first things that come to mind because that's really the modality we saw with ChatGPT. But when we think about enterprise ROI, I think there's two things that end up driving that. The first is more automation oriented, and then the second is hooking up these models to tools, which is really where I think the rise of agents and agentic AI is coming so that it can do a lot more, uh, in terms of what it's actually interacting with.

Ashish rajan: Any examples that come to mind with and I guess you obviously mentioned the examples for the, the background check piece, which is the automation thing. 'Cause I guess I'm curious about the agent use cases as well. 'cause a lot of people still plug agent behind a chatbot and call it agentic as well.

I'm assuming that's not the use case you're referring to.

Dev Rishi: I think that, you know, at times it can be, but I, one of the best, uh, examples for agentic and a lot of organizations are these coding agents that I think are [00:09:00] increasingly being deployed across organizations. So, coding agents started off maybe similar to how we think about chat as auto complete suggestions.

Now coding agents generate unit and run unit testing. They, um, can file PRS and pull requests. Uh, and you know, we've seen instances where companies go pretty aggressive in the type of access that they give coding agents, but I think that's really one simple example. You know, the brilliant thing about agents is I don't think it's.

We, we, a lot of the demos that we run through for agents have this like one really impressive deep workflow. It's like, oh, the agent's planning my entire wedding for me. But a lot of the things I actually see in the enterprise is like something really bespoke and like really bite-sized. I just want an agent to be able to do you know, one very simple example is, um, every day I need to go and generate emails for like the Salesforce opportunities that I have.

I need to op, update my opportunities based on the call transcript notes and others. This is not like a groundbreaking use case in terms of 20 different systems that need to get orchestrated, but it's like maybe my Zoom transcripts and my Salesforce instances. I think that's like a great [00:10:00] example of something where you need to give it access to a couple of tools.

It needs to be able to read from one write to another. Uh, and what I see, rather than like one big agent as being successful is people really starting to hammer out across multiple of these

Caleb Sima: And like a you, you were to get sort of percentage in the way that you work with enterprises right now. Uh, I think you gave a little bit, like 80% are using foundational models, maybe 20, is it sort of 20% are using these more fine tuned open weight models or, and then when you say enterprise, are we thinking big Fortune 100 or like in the Fortune 5,000, right?

Kind of, yeah.

Dev Rishi: Yeah. So Rubrik and our customer base, we really think about as being like Global 2000 Enterprise, uh, in a large way. And it tends to be. The folks that I've spent a lot of time with over the last four months tend to be both IT and security leaders across those organizations. Look, I think if we think about foundation model usage versus open source model usage, it's probably pretty still weighted towards foundation models by use case count.

The big difference might come from that. That really [00:11:00] high volume use cases are the ones that are more likely to be, you know, more that where there's more effort being put behind, maybe tuning or otherwise. I think the other trend that I've noticed too is like. Where agents are starting to become a thing and what the trend there is.

And I actually have a little bit of data on that. Um, which is over the past three or four months, I've spoken to about 180 different organizations at like the IT and security level. And I broke down, where were they in four different phases of agent development. So phase zero actually was, we haven't started experimenting with agents.

Phase one was experimentation. Phase two was what we called formalization, uh, where they had, you know, started to roll out maybe their first, second or third agent. They had gone from experimentation into just early stages of production. Phase three was scaling. Then phase four was autonomous ai, just sort of a catchall for where we think the world is gonna go, but not anyone's really there yet when it comes to G 2K.

And what we saw was about 12% of organizations that I spoke to of the 180. They said, you know, we haven't even started. There's no agent project going on internally, but the vast majority, [00:12:00] 88% had at least started, 52% were in experimentation, and I wanna say about 22, 20 3% were in that formalization phase.

So definitely a lot of heavy experimentation phase with the enterprise. Still right now, in 2025. Hmm

Caleb Sima: so, you know, you're talking to these leaders, you, you're sort of seeing this adoption in agents now.

You're talking to like what you said it and security leaders. So you know, they're gonna have a different perspective on this. What have you found to be some of their top. Areas of both things that maybe excite them, that scare them, how's, how are they sort of reacting and what's been their response to this AI movement?

Dev Rishi: Yeah, definitely. So, I think the first thing that's important is maybe let's just define agents for a moment. Then we can talk about what's exciting and what's scary. I define agents and you know, at Rubrik we've been defining agents as just LLMs with access to tools. So we can think about that as models that can take action on behalf of a user or, be able to interact with some of the enterprise applications and production systems.

What excites them is [00:13:00] oftentimes seeing some of the examples of like a use case of the productivity lift they've seen externally. So, uh, you know, there's oftentimes a board level pressure that comes down based on what we've seen in the competitive landscape. Customer or company A is doing something, and that means company B, C, and D in that same space.

Need to do the same or better in order to make sure that they get, don't get leapfrog. So I think a lot of times it's coming from like this either board level dynamic or even what they see in terms of the demos that get built very quickly. Look, building an agent has never been easier. Uh, it takes days to be able to build an agent.

It's like a prompt plus some tools. What scares people, I think, is that it only takes days to build an agent. And so, you know what I, I would say hours. Even hours, right? Exactly. Days probably to be conservative. Right? And what scares people is it takes only hours or days to be able to build agents.

And a lot of people can do it. People can do it quickly. And by the definition I gave you of agents, what we're talking about is giving non-deterministic models and LLMs [00:14:00] access to enterprise IT and security tools that maybe it hasn't already been architected for. So the types of concerns that I hear. I broadly kind of bucket into three areas.

The first is there's a concern around, what I hear is like agents sprawl or you know, shadow a AI now becoming shadow agents. It's this idea that is there like this second digital and AI workforce that's being developed inside of my organization that I don't have a really good handle of. I background check all my employees.

I have management reports and others, but if anybody's building agents, I've spoken to a lot of organizations that said it took 'em a long time to get from one to five agents, but took them almost no time at all to get from five to 500. It's like once it starts, it can really become a snow, uh, snowball quickly.

So the first concern I hear is, do I have a handle on what's actually even happening? Observability? Like, do I know what's happening on that side? The second concern I have, uh, that I hear often, and I think this is maybe one of the most nebulous, but every organization, it seems to me now has an AI governance committee, your body really, [00:15:00] and that committee is responsible for coming up with these policies around what AI or agents can and can't do.

Yeah. A good example is like may a lot of organizations say agents should not give financial advice or should not make hiring or discrimination decisions. Sounds great. How do you actually enforce that in practice? Especially if people are building agents everywhere. I'll tell you what we did as a first stab.

We had, um, we had a committee meeting every few weeks that would review projects and we had Google forms that you'd go ahead and do and like attestations. It's like some small examples, for how we were thinking through this. And, um, I think what we, what we saw and ran into is if you can build agents and hours.

The committee meetings are gonna take weeks and the review process is there, and then it's only a onetime issue. Like, you know, you've, that it like operated like this on July 26th, but who's to say what's happening on August 26th? That really, I think, felt, um, uncompelling from like a real governance with teeth aspect.

So that's the second bucket of issues that I've heard. And it's an example

Caleb Sima: of governance, safety, and security, just getting in the way of [00:16:00] progress.

Dev Rishi: You know, I actually think about it a little bit more of like the problem of. Not having a technology solution for governance, security, and safety, and instead relying purely on people, process and paper.

Um, because what our engineers actually would tell you is we want, like, look, we want the agents to do the right thing too. So we believe strongly in like the governance and security guardrails. What we don't want is to have to go and review this every two to four weeks in a committee meeting. Tell us the rules and the framework.

Give us a way to be able to enforce those. And as long as we're operating in that, we move like, let us go. We

Caleb Sima: move. Yeah,

Dev Rishi: let's move. Exactly. And so it's really about like, I, I think about it as like transparency. Like do you have a set of controls defined and can you run those controls in a way that allow people to build and, you know, have the confidence that things are gonna actually work out in the way that that we want them to.

And then the third bucket, you know, so the first bucket was observability. The second bucket was governance with enforcement, I think about. And then the third bucket is, look, something's going to go wrong. And I think Rubrik has [00:17:00] had this mentality from early days called Assume Breach, which was to say we layer on a set of, uh, policies, a set of solutions to try and minimize the impact of anything that'll happen or minimize the likelihood that it will, but we'll operate in the real world and things will go wrong.

And the, I think, realistic nature of a agentic AI is it moves much more quickly than human threats. So it's probably more likely as well that good things will happen much more quickly and the occasional bad things will also happen. And so you need some way to like undo or remediate where things go wrong.

Um, and those are the three key things that I think I've been really hearing from enterprise IT and security leaders.

Ashish rajan: I was gonna say, is it possible to undo though?

Caleb Sima: Actually, that's exactly like what, like when you think about like, you know, I am also a, you know, thousand percent assume breach.

Yeah, I think most cybersecurity,

Ashish rajan: well most cybersecurity, yeah. In all of

Caleb Sima: my organizations, that was my number one principle. But assuming breach is way easy to say, way more difficult to to implement in any real scenarios. 'cause it's very expensive [00:18:00] actually, when you think about assumed breach. And so like when actually I think Ashish going to you like.

Take, uh, one of your examples, like, Hey, AI should not be making decisions on human resources. You acceptance or priority. Yeah. And then you assume breach here, or in this sense, it's not necessarily breach, but assume bad intent or accidental intent has occurred and AI has been put into a place where it now has been making decisions on resumes.

How do you even implement or think about that kind of strategy planning?

Dev Rishi: I think, um, maybe just to jump in on this idea of like, is it even possible to, you know, undo that action? We actually first started to get more into this space when we had this idea that we called Agent Rewind. We released it I think a few months ago, uh, was when we first started to talk about externally.

But the core idea was that Rubrik is backing up a lot of the most important production systems already for the global 2000 [00:19:00] enterprise customers that we work with. We're constantly, you know, on the policy level that's been set, taking the snapshots for like, what are all the deltas and the diffs that are being made towards these production systems.

And we need that for business continuity. But it can also be applied in a lot of settings for continuity in the case where an agent is taking an action. So, to give you an example one of the settings that I think became famous earlier this year is when a coding agent at one of the companies in the valley, you know, went rogue and, uh, deleted a production database.

The fastest way to sub, sub-optimize the routine is to be able to actually just delete the ba, that database entirely. Now, a lot of instances you kind of say, okay, the database is gone. What do I do here? The great thing I think we have with Rubrik is we actually have the previous healthy snapshots from before the agent took that action on, as an example, that database.

What we've really paired is we paired a new feature that we release around Asian observability that allowed us see one agents are taking certain actions and we paired that with [00:20:00] Rubriks core understanding of the enterprise backup and data. Uh, and so we say that if an organization has an agent that's taken an inadvertent or maliciously da, uh, you know, destructive action or just a deletion action on anything that Rubrik protects, we can actually allow you to one click recover from that previous healthy snapshot and backup.

For a lot of instances that look like data mutations, deletions, edits, and others, essentially, effectively look like a rewind for what that AI agent had done on that system. And so I think when we have started to explain it that way, actually, the, the reaction I get from a lot of IT leaders pretty similar, which is like, I didn't really think about these actions being reversible.

And there's some class of actions that today I think aren't reversible. But a lot of the ones that I think are talking about editing production systems and others internally. Are ones that Rubrik has a pretty interesting, um, ability to protect.

Caleb Sima: Yeah. You know, I would say that, you know, first of all, I really like that name agent rewind.

Yes, I, I would agree with this. Like I can understand where, database backup and recovery can [00:21:00] play a lot in sort of the assume. Damage sort of aspect of this, but a lot of agents are interacting in applications, changing configurations, changing SaaS applications or data in SaaS applications.

And these aspects where, you know, the power of an agent to your is this is the tools and interacting across all of these different applications. How do you know? There's configurations, there's workflows, there's SaaS applications that all get changed. Yeah. I'm assuming you know. I don't want to get, make this too much of a rubric commercial, but like, you know, is, is that then a problem that Rubrik or any other vendor can then say, oh, restore.

Right. That seems to be a very challenging area.

Dev Rishi: Yeah. I will say, not to make it a rubric commercial, but of ever, when I speak with folks, I haven't seen anybody else talk about these AI Agent Rewind capabilities. So as far as I'm aware, I think we're the, um, folks that are really leading in the [00:22:00] space to your point.

Are there SaaS applications and others, uh, also capable for it? Yes. Actually, Rubrik protects and backs up SaaS applications like Salesforce as an instance. So we're talking about, you know, the edit that shouldn't have happened. That's just one, uh, example. That's the type of property that Rubrik is actually protecting as well.

I think in large part, what we, what we see is actions can exist not just in the database side, but like configurations, which oftentimes are actually backed up as well. Um, SaaS applications as well. And then there's a class of other things which are transactions, um, which are not so much necessarily like, it's like actual, you know, money left a bank account or something along those lines, which aren't, uh, quite as relevant in the backup space.

Those are ones that we have, um, some early thinking on that, you know, are a little bit adjacent towards backup. But a lot of what I think organizations and agents are operating on right now already sort of fall in the purview of what we think about being important for business continuity.

Caleb Sima: Okay.

So moving forward on the Agentic AI sort of side of [00:23:00] things you know, we're, we're starting to see a lot of really interesting applications come out with ai, like some really powerful, uh. Things that are now, all these vendors are now, like, they're starting to come outta the woodwork with AI behind them.

But we have, when we talk to enterprises, you know, like the core enterprises that are doing these things, minus vendors I still get a pretty cold amount of feet of people saying real production. Pro, AI is actually running and doing things and serving their customers minus your chat bot, customer service sort of model.

Or NLP play. Like what? And I know we, you know, we talked a lot about this at the beginning, but I want to continue to put pressure here and find more info from you, uh, as to maybe why or are you seeing that same thing? What, you know, do you have any predictions around this?

Dev Rishi: Totally, yeah.

Um, I do and I also have a prediction around it. I think that [00:24:00] realistically, a lot of agents today, in fact, lemme just say almost all agents across the enterprise are in read mode, right? They're agents that like, essentially can maybe help me access a little bit of information. But a lot of the interesting use cases we often talk about, which is like, go get this information from here, write in this production system, or, you know, be able to take this action on my behalf.

Are currently being blocked because organizations have these policies that say the agents should be read only and to get right access. It's actually quite a bit of a leap inside of a lot of the companies. 'cause think about it, you want to give a non-deterministic model with a non-human identity. Right.

Access to our system. Right? Yeah. Yeah. There's no trust, and I think trust really will come down in two ways. I don't think we necessarily have to put the trust in the agent itself. It's hard. It's a model that is a little bit difficult to explain it. Otherwise, we need to have trust in the controls that exist across these agents.

One thing I saw is, um, that in 2025 and maybe part of 24, a lot of people predicted this future of [00:25:00] agentic ai and a lot of platforms came out that make it easy to build agents, no code, low code, pro code, everything in between to do open source orchestration of agents. That's not really the concern I hear from organizations today.

It's not like I need another orchestration framework, or I need another, you know, low-code agent builder. The concern I have is these agents are getting built all over the place. How do I actually have active realtime enforcement of guardrails that I want and what do I do if something goes wrong? So those two factors, I think haven't really hit the market in terms of like, I, I'm curious to hear as well from like listeners, otherwise if anyone's found vendors that have solved this trust problem for them.

Caleb Sima: I have 3000 startups in stealth mode right now that I have been pitched. That's right. They're all trying to solve that trust barrier, which I, I can go on a rant on that a little bit later, but maybe we'll get to that in this call.

Dev Rishi: And, and the truth is, it's hard for the Global 2000 enterprise. I mean, look, as a startup that sold to some of the global 2000 enterprise, it's hard.

Right? And, uh, like as you know, small seed stage south, stealth [00:26:00] mode startup to be able to actually. Be trusted with like the enterprise IT and security applications inside the whole system. And I think that until there's a real platform that works, that's like enterprise grade, we're gonna continue to be in read mode.

And as long as we're in read mode, we're gonna continue to get these questions on the ROI of AI of like, are agents actually doing anything cool or interesting? Because read mode agents tend to be something that's like better search, better knowledge retrieval and that is valuable. But a lot of the types of use cases that I think.

We all talk about and get excited about when you hear an exec talk about two to three X productivity improvements, it's not gonna come from like slightly better Google search internally. Yeah. Uh, and so we need to think about what are the controls organizations need before they're comfortable going from read to write?

Yeah.

Caleb Sima: It's like the difference between an AI reading my code and telling me what it's about versus writing code and building something. Yeah. Yeah, exactly.

Dev Rishi: Yeah. I'd say there's an order of magnitude, or at least a factor, maybe an order of magnitude like difference in the productivity gains that I [00:27:00] get from.

Reading its assessment versus just do it for me.

Caleb Sima: So then what do you like, what is your prediction? How do you think this problem will be solved? You know, I've got a lot of, and, and I'll give you some background, like I get a lot of these startups that will come and pitch. Yes. We need the, we need the control place and authorization and authentication of agents.

And then when you ask, well, how do you determine. Whether an agent is doing a good or bad thing, it all, every single one of these guys all comes down to. Oh, we'll just do some sort of anomalous behavior analysis, which is complete failure right from the start. Right. Um, and no one seems to really understand what it is or what it even means to be able to say, how do you put the right controls around an agent to go and do stuff?

What do you think, like, how do you think this is going to be solved?

Dev Rishi: I have a pretty specific thought on how [00:28:00] I think it'll, uh, likely get solved in. The first thing I'll just say before I go too into the weeds on like the tech behind it is I think there are two pillars that you're going to need before you get comfortable from control standpoint.

The first is you're going to wanna be able to do something proactively, so understand like preventative tools that you can put in. And then the second is I think you're gonna need to remediation layer again, the assumed breach mentality. I don't think people will give like Right. It's solution.

Caleb Sima: Yeah.

Dev Rishi: Yes, exactly.

Uh, the Rubric solution, because I don't think people will give right access. You know, even if you tell me I have some controls in place, if you tell me that the entire business is sunk, you know, if the agent makes the wrong mistake, it's just really too early to know all the different ways agents can, uh, operate.

But let me just, we've talked about rewind, which is like, how do I get a control when something goes wrong? Lemme tell you about how I think about these actual controls and governance. I think anomaly detection is. It's useful in the sense like there's this behavioral and um, analytics previous era that happened with cloud security too, in terms of like where configurations and other can go wrong.

[00:29:00] It's oftentimes attacking the problem a little bit too late, and it's also not even high recall enough in terms of the types of things that can go wrong.

Caleb Sima: And, and I'll tell you, just in my experience, you know, in going through in any decent enterprise I used to think that before being in an enterprise and then now I don't think there is such a thing as normal.

Uh, yes, I don't think there is a normal baseline in any enterprise. Uh, perfect. That's been, yeah,

Dev Rishi: I completely agree with that. I don't think there's a normal baseline, and I think where a lot of the tools fail today, like these guardrail tools that provide you the ability to, you know, say, oh, the AI won't hallucinate, or something along those lines is they take this one size fits all solution, the single guardrail, and they're like, every organization will want this.

The reality of what I see is every organization has its own quite bespoke policies. That they actually need to be able to enforce on like what it means for an agent to be able to go wrong. So we gave that HR discrimination example. I spoke with the um, company in the housing space that talked about how, you know, what an agent says with respect to housing and demographics around housing or [00:30:00] neighborhoods is also really sensitive.

There is no human way I can think about encoding that into a rule. Like I cannot imagine like putting a rule in some place that says do not, you know, talk about these demographic categories of classes. There's so many ways to express things in English language that I think the only way that organizations are gonna be able to get comfortable or be able to do this is you're going to need AI systems and models to be able to actually enforce these policies themselves.

'cause these policies are written up by legal or InfoSec in English. And you need to be able to do real time enforcement around them. Yeah. Now actually these models are relatively good at being discriminators, and what I mean by that is like looking at an input or an output and then judging, yeah, this looks okay, or no, this doesn't look okay.

If you give it some text and ask it, does it adhere to this? Models are pretty good at that type of simple decision making. The trick that you're going to run into is every time I make a LLM call. It's gonna get really expensive if I have to make like three or four or 10 other LLM calls as well that [00:31:00] say something like, Hey, is it violating this policy, or others?

And so what I think you're going to need is a series of a system where you can define a policy in natural language. That policy essentially gets distilled into a small language model itself that can run at low latency. I'm talking in order of magnitude lower than what a system could do. Also where you can multiplex all of these small language models to operate off the same model.

So it's not, it's not like you need 80 different models. You're actually running 80 indicators on top of the same model. Yep. We actually, we had not thought about this use case, but we open sourced some of the underlying infrastructure to make something like that possible. The open source project is called Lorax, like the Dr. Sues character. And there's some, uh, ML terminology puns for why that is. Oh. Uh, but that really, I think is forming the basis for how our take on the solution is. We think AI is gonna have to protect some of these AI situations 'cause it's too nuanced to leave to rule systems or otherwise, or to anomaly detection.

Caleb Sima: Yeah. So if I were to sort of say it in a, you know, simplistic way, [00:32:00] it's, it's building a system of AI judges or cops. Yeah, that will monitor all of the different calls at wherever AI is being used and can, uh, infer a set of policies and behavior and ensure that when those infractions occur alerts will be sent to, or restrictions can be applied.

Dev Rishi: Yeah, exactly. And I think that last piece is critical, which is some of these and you know, we've just started to speak with some of our early customers about this as well. So I think people are getting a sneak peek to some of the conversations we've been having. But some of these operate in an alert mode.

They're monitoring great. A lot of them need to operate in line in a block mode. They need to say, if you're doing this, drop the request. Uh, and that's really where some of the technical challenges around latency and others where you make sure that this is an adding overhead in a synchronous call come up.

Ashish rajan: Yeah. So is the future then a bun, a lot of sm s SLMs, for lack of better word, that are making that call so that it's a, it's a cost effective exercise to let that LLM be the judge. [00:33:00] And do you see that across? I guess there is a, there are two layers to this because the audience is from security. They're obviously thinking in their head, am I building SLM for my team?

Or what, where, so I guess it's worthwhile calling on a business may have. I, I think this is how I, I hear this, but, and correct me if I'm wrong, it's as a business, I have a public facing, let's just say in this particular case, chatbot or housing or whatever you wanna use. I have designed a bunch of s SLMs to make that judgment call for, is this right?

And whether it's inline or on the host or whatever the thing may be, that's more of my business alert comes out of it and goes to security and kind of or whoever it needs to go to. Developer wherever they triage and they do what the regular program is. Or do you see there's capabilities like these coming up there as well in those smaller teams.

Dev Rishi: So we went pretty in the weeds on like what the solution could look like in terms of like small language models, multiplexing and others. But I having worked as a company that helped organizations build [00:34:00] SLMs, there's no chance that I'd want to say every business and security leader needs to build these.

I think from a business standpoint, what it's gonna look like is you just are able to define the policies you need. In natural language at the level of detail that you want, and then you can set that to be in a monitor or block mode. That's what I think you know, from the business side, that's all you need.

The fact that it's using SLMs and that they're multiplexing, that's all under the hood from an implementation side.

Caleb Sima: So, lemme, lemme ask you, Dev, like, there, there are a lot, this, what you're explaining is effectively LLM firewalls, right? Mm-hmm. And there are a lot of them, like there are a lot of these sort of prompt guards, LLM firewall types of solutions in the market, and you haven't seen a lot of, at least pickup, at least that's been my experience, right?

You, you don't see a lot of pickup of these things. And yet that really is, to some extent what you're talking about. Help me understand sort of the dynamic.

Dev Rishi: I actually think, so LM firewalls often operate at like a network level and the [00:35:00] use cases, LM firewalls, I think have been being applied to tend to be like shadow AI discovery.

And then also like these relatively let's say generalized detections. What I mean by that is like these generalized detections are things like, Hey, is PII leaving ice in the system that's like the most,

Caleb Sima: or are you trying to prompt inject and or are you

Dev Rishi: trying to do a prompt injection attack? There's like six, eight or 10 of these indicators that become pretty popular and like these firewalls are all looking to be able to enforce that.

I think I really liked what you said earlier, which is like there's nothing general about like what happens inside of an enterprise. There's nothing out, right? These systems I think, work well for like that very first open-ended use case that gets someone comfortable using chat GBT, which is like, okay, I know that there's an LM firewall that that'll make sure I don't send PI to chat GPT or that someone doesn't decide to jailbreak chat GPT.

But I think what we're talking about. Is the actual ability to define your own policies. And in natural language as well that [00:36:00] can go run and be able to do violations or blocking. Now the violations of blocking is a capability firewalls have had, but the ability to actually define your own policies in natural language that say something like, Hey, my agent should not be able to provide discriminatory housing information.

Like, that's actually a very bespoke thing. Those are capabilities that I haven't seen really kind of within the all on firewall.

Caleb Sima: Let me ask you, do you think that is just a prompt or that is a fine tuned kind of feature that has to be done.

Dev Rishi: I think it's a workflow. And I think it's a workflow that's going to incorporate fu or, uh, you know, we should show you the product demo at some point.

In primitives. I think there's a prompt, but it's not just a simple prompt. There's actually, um, there's these cookbooks that have been published by some of the leading labs.

Around how they developed their own guardrail models, which operate intrinsic to the model we'd be applying extrinsic, but like they apply intrinsic. What are the types of information you need to be able to provide that oftentimes includes like a prompt, an example of like a an example an explanation [00:37:00] field.

So it's a structured prompt. This might be the one first thing I'll say, but then it goes beyond that. It also needs to be able to take feedback in from a user or a system as well. That's where things go into a workflow side, and then there's something proprietary about how you actually fine tune that.

The real purpose of fine tuning is twofold. The first to be able to incorporate some of that feedback. But then the second is to get that small footprint where you're not gonna add, you know, a hundred percent of network latency overhead. In order to be able to run the system, you're gonna add something very small.

So that's how I think about it.

Caleb Sima: Okay. So maybe two things. First, tell me if I'm correct in this, is what you're saying is. There's almost there. The process that you're talking about is almost a fine tuning of the prompt itself as it learns and gets the feedback required for that business context, correct?

Dev Rishi: Yes. Yes. And there is a additional piece around fine tuning that you'll do. Because this prompt might work great on a large model, but once you actually deploy it into [00:38:00] production, this is the classic example where we'll need a really low latency, small model. So we'll need to fine tune a little bit to be able to distill into a small model that can run as a guard.

Caleb Sima: And then how do you deal with the, where does it get placed? So what you're saying is, okay, let's get the this sort of workflow of consistently fine tuning the prompt for a very small, fast, cheap model to be able to do the judging. Where do you put it? Do you still put it in a network man, in the middle kind of place?

Where, like where does it sit in this scenario?

Dev Rishi: Most organizations that we're working with today have. Some centralization towards like how something like ai, a AI API, keys get distributed like call that a gateway solution or something similar. We think that there's a few different places where this ends up getting put, but one of the most init, one of the most common initial starting places is you have a central system that's like distributing keys, and that's like oftentimes routing.

It's like a gateway slash router routes [00:39:00] towards like Anthropic, OpenAI, anywhere else that you wanna do it. We attach to that or act as your standalone gateway if you don't have one already.

Caleb Sima: Yeah. So that's

Dev Rishi: where you put that solution.

Caleb Sima: It's like an open router for the enterprise. Um, exactly. Yeah.

Okay. Yeah.

Ashish rajan: Our out of curiosity. 'Cause this is something that, funny enough, I was at AWS and a lot of the conversation that I had at reinvent was around this very problem of being able to not have this. For lack of better, of a deviant from, Hey, I am an insurance company. I should only give financial advice. Right?

What? What? Because you come, come from that deep learning space from before. Was this a problem then, or is this a newer problem? Because now we have more in production where the deep learning was, Hey, I need to pump up this, like start the system, run for five minutes and hopefully we've had had the answer.

What was the difference as why is that problem now or was it already solved before and now we just, there's another. wave of the same problem, I guess.

Dev Rishi: So in DL we had like two sets of challenges. We had one that was like considered in compliance broadly. One of them was, um, drift detection. We always [00:40:00] talked about, Hey, you're getting new data, does the model still work?

That was more of a performance oriented challenge. And then you had a second challenge, which was like model bias. But the big difference was that in deep learning, we never really gave those models access to tools. They were like statistical models essentially. Okay. That were, um. You know, able to go ahead and operate within some very confined sighting.

Like you took the output, like you took the Zillow estimate as an example. Yeah. And you plugged it into some product or feature. Now I think the biggest thing that's happening is we're thinking about actually enabling these models to have access to tools. And you know, coming back to the original question, I think also Caleb was asking, it's like people are doing all these cool agent demos, but then you talk to the enterprise and is it real there?

No it's not. Why not? Well, they're still in read mode. They're not in write mode. What's stopping them from going to write mode? You know, these sets of guards, these, um, previous deep learning model, uh, instantiations never went into write mode. They never had to go really into right mode. Right. And so bias was a thing and we have some learnings from that, but this level of like, you know, autonomous guard railing, I think is different.

The second really quick thing is deep [00:41:00] learning models were almost exclusively built by PhDs and multi uh, function teams pretty much always took several months. Mm-hmm. Agents are being built by anybody vibe, coding and takes hours. And so like the scale of problem we're talking about is also different.

Ashish rajan: Someone just made one as we, as he, he or she heard the episode, I guess. I imagine. So they're like, we'll, just whiteboard this.

Caleb Sima: Yeah. Yeah. Do you find what other burning questions sort of, or burning things that are on your mind that you'd love to be able to get people to know and realize?

Dev Rishi: I think that the, um.

If I had to summarize like a couple of my thoughts, one of them is agents are going to inevitably graduate from read to write because I think what's gonna happen is people are gonna be on some part of the risk for a curve and the productivity gains are gonna be realized by the folks that allow more autonomous action.

I think one thing that I want people to know is like, undo is possible. We talked about that. I think another thing that I want people to know is that the current way that a lot of times governance policies are enforced is really painful. I don't think actually will work [00:42:00] as the system through the end of 2026.

So there will be a vendor that is going to solve that problem. Obviously, I hope it's us, but somebody is going to solve that problem. And I think the way it's gonna get solved is the way that we talked about in the architecture. And then the third thing that I, I want to talk about is just like something in the back of people's minds is, um, I think as people start to roll this out, there's gonna be a class of tooling that we haven't even talked about yet.

That's gonna be really interesting. That I've started to hear a little bit about, people talk about this idea of digital twin in the enterprise and you know, how agents interact with systems that are more simulated in nature. So I think there's some learnings we can pull from the autonomous vehicle space and others about what agents in the enterprise might even look like.

But we're still on day zero problems today. How do we make sure something's secure enough to grow from read to write? And I think that's where a lot of 26 will be focused.

Caleb Sima: Okay. And then my understanding is in order to do that, you need recovery. And the ability to be able to do that. And then also the right guardrails in place, which you're hoping lox is sort of that foundation.

Dev Rishi: I think that, I think you need three things in total. [00:43:00] The first is you need observability. You need something that's gonna hook into where all your agents are and provide, you know, what we call an auto discovered agent inventory. Mm-hmm. Then you take that observability and you plug in actual real-time enforcement of policies in that observability layer.

That's how we call our governance pillar. Lorax is, you know, an open source piece of technology that we, um, open source in 23, but it's one of the many pieces of tech that we will loop into, uh, that infrastructure. And then when, you know, that's working, hopefully the vast majority of things are totally fine, but everything goes wrong, then you need recovery.

So those three pillars are what I think about observability, governance, and remediation. And that's really the product that post acquisition we announced called the Rubrik Agent Cloud.

Ashish rajan: Outta cross city. You didn't mention identity as 'cause to what Caleb was saying. There's obviously, there's a huge wave of LLM firewalls, but there's also this NHI space, which is non-human identity, just also quite top of mind for a lot of people.

Yeah. I'm, I'm curious from a, I guess what you've called out, I think those are really good examples, like the observability [00:44:00] piece being able to rewind. I love those. I think where people may also be interested in is that some of the foundational things that they can absolutely apply. Maybe, is identity still playing a role here?

Dev Rishi: Oh, absolutely. Yeah. And it's actually a good example of one of the primitives that I think, there's two parts towards identity. There's like management around, uh, agent identity. So agentic identities need to get minted. They need to be credentialed and scoped. They need to be associated with the agents.

And a lot of the underlying identity providers are actually tackling these sites of the problems. Things like Okta or Entra inside of, you know, Microsoft as they were talking about Ignite. And then there's a second layer, which I think actually extends more towards observability and enforcement. But when it comes towards identities.

When it comes towards agents, you need to know what agents are running. But you also need to know the number one question I often get is what agents are running. Number two is what tools and data do they have access to? Well, the tools and data they have access to are fundamentally correlated towards what identity credentials are they actually using and what are the scopes on that.

And so that actually we bucket, you [00:45:00] know, that identity management piece we think is going to be well handled by that underlying identity provider substrate. And there's probably going to be a lot of companies that make a lot of money even in that space. But I think then there's gonna be that second piece of like identity observability and governance, and both of those we think fit into that same agent platform.

Ashish rajan: Oh, right. Okay. 'cause I was gonna say, how do you even, is this where the MCP and A2A world that kind of opened up for everyone? I don't know how many CPS are running at any given point in time in my organization.

Dev Rishi: Well, I think the biggest actual con security consideration for MCP tends to be around like governance and security.

How do I know if it's a registered MCP server? How do I know if like it's something that actually can use the secure credentials? So my take is people are very excited. I spoke to a CISO who was like, I'm very excited about MCP. I cannot wait until I can actually start to use it, but there's no way I can use it today.

And so I think. I see again, a number of startups that are operating in this space as well, but, uh, come back to some of the same concerns

Caleb Sima: I have. Maybe one final question for you, since we're on the MCP topic, MCP or A2A, [00:46:00] oh,

Ashish rajan: who's

Dev Rishi: gonna win?

Ashish rajan: Who's

Dev Rishi: gonna

Ashish rajan: win?

Dev Rishi: It's a good question and I mean, I think the.

You could probably see these acting somewhat harmoniously. So I think maybe set up for, to tackle those separate problems. I'll try and answer it, you know, quickly as well towards the end. But MCPI think about as being able to tackle the problem of how do I-A-P-F-I production applications everywhere that agents can start to go and access and then a two, a agent, a agent protocol.

I think for the idea that agents are not just gonna be interacting only with production systems, but directly also with other agents. And we need a way to be able to support that. I think MCP will get traction first. And I think one of the reasons is like there's like a really core set of concerns that MCP starts to be able to address that aren't super futuristic, but are banal to some extent.

Like things that we just need here and now. And I think the main thing that's actually just inhibiting MCP today is I think a really good security and governance framework around it.

Caleb Sima: I was, um, has all of that. And then so why is it not getting, it has the identity [00:47:00] authentication, everything, all the, all of the right security primitives to make it a very enterprise deployable solution.

Why do you think that has not gathered steam?

Dev Rishi: I think that it actually comes a little bit down to the underlying identity systems themselves. I think the underlying identity systems, and this is a flaw for both MCP and A2A. Like if we're saying A2A has the right, like security posture. I think one of the issues is that the underlying identity systems that A2A or MCP inherit don't have the level of granularity that I think a lot of times organizations want, for the agent applications they're gonna deliver as one example, like it real, like a lot of these systems might use like OAuth tokens as like their underlying credential, like a service account or an OAuth token.

Maybe that's been delegated from a user in order to be able to do it. That OAuth token might say something like, oh, you can send emails. Or like, that's the level of scopes right on it. It's like you can send emails, you can read emails or so forth.

Ashish rajan: Yep.

Dev Rishi: But what we talk about with agents is like, yes, you can send emails, but you can't send any emails [00:48:00] that, you know, maybe you can send emails to internal recipients, but not external recipients or something along those lines.

You actually want like a more granular level of control that exists. And I think that's one of the issues that probably both of the protocols. I don't think it necessarily gets fixed in the protocols themselves. I think it gets fixed maybe in some of the scaffolding that gets built across, uh, around these.

Caleb Sima: Well, I'm

Dev Rishi: curious

Caleb Sima: what your,

Dev Rishi: your take is though, Caleb.

Caleb Sima: Yeah, just for audience. A A2A has solved that. Uh, it allows you to define custom scopes that allow you to get more fine grain control and the way that they've set everything up in there. Is extremely strong. Like actually to, to me, first of all, Dev, I agree with you.

I mean, MCP is going to win. The reason why it's going to win is because Google is terrible at marketing. Um, and that, that to me has been the struggle because like A2A for all intents and purposes for an enterprise is almost a no-brainer. It seems to me when you look through and use [00:49:00] A2A if I'm gonna deploy agents in production, this is what I use, right?

MCP is a good it. Tool that I screw around with, but like you look at A2A and you're like, okay, this is like engineering quality, prod, scalability and it's got the security features that you need. It's just no one knows about it. That's what I think is the, yeah, it's just a Google marketing usability problem.

Dev Rishi: Do you think it's just marketing? Actually, I feel like I hear more, I mean, it's not at the same level of name recognition as MCP. Definitely. But I think if it's just a marketing concern. Look, I worked at Google for a while and, uh, you know, I, I understand what you're saying. I think we'll get over that through the end of 2026.

Um, if you told me that people spent similar marketing dollars, but I think it's a really interesting race to see who will win. Because I think maybe we're both backing the same horse, slightly different reasons. I, I don't know

Ashish rajan: guys, I'm just backing the, uh, A2A more, not the MCP one. I think, uh, I think we spoke about this a, uh, in a, for a long time, [00:50:00] but I know we are on time as well.

Was I'm, I think we should do a part two, but hey, because there's so much more to unpack here, where can people, uh, get in touch with you Dev to talk more about all of this and the stuff that Rubrik is doing?

Dev Rishi: Definitely if you go to the rubric.com website, there's actually a page on the Rubrik Agent Cloud.

You'll be able to see what we're doing. We are putting out like demo videos and others, so you can actually see a little bit of what this product look like. And then if you wanna just reach out to me too, I'm Dev Rishi and you know, more than happy to chat a little bit more about any of these topics.

It's something I'm spending a lot of time thinking about.

Ashish rajan: I'll put your LinkedIn in there as well. I assuming that's the socials for you. Yeah, LinkedIn is perfect. Awesome. All right, dude. Thanks so much for this and uh, for tuning in and being such a insightful conversation. I just had so much to, I had to take so many notes on this, at least mental note.

But thank you so much for doing this and thanks everyone for tuning in as well. Look for looking forward to part two, hopefully. Thank you for watching all listening to that episode of AI Security Podcast. This is brought to you by Tech riot.io. If you want to hear or watch more episodes of AI security, check that out on ai security podcast.com.

And in case you're interested in learning more [00:51:00] about cloud security, you should check our a sister podcast called Cloud Security Podcast, which is available on Cloud Security podcast.tv. Thank you for tuning in and I'll see you in the next episode. Peace.

‍