Graham Neray: [00:00:00] Our conclusion was no one's building real agents. Like everyone shipped something to make the board or the CEO happy because someone like me said that they had to

Ashish Rajan: What do you see as the blind spots that maybe people don't or they get surprised by?

Graham Neray: What makes these things dangerous is the same thing that makes them powerful, and that's hooking them up to production systems and data.

Graham Neray: They can be like so relentless in trying to help you that they will go and find a production cred to go do something bad. I hear this story all the time. I talked to a large private equity firm that said they use 7,000 different software vendors.

Caleb Sima: If you can do least privilege on an agent, why can't we do least privilege on a person?

Caleb Sima: We want blocking, but no one ever uses it.

Graham Neray: Everyone needs to sort of expand their imagination. There are obviously all these cases where they can act autonomously.

Ashish Rajan: We have been talking about AI security for some time on the AI Security Podcast, and it's definitely running on steroids on the internet with agent as the main context.

Ashish Rajan: So this week we have Graham from Oso, who is talking about agent security. We talk into how do you separate a signal from the noise. If you are someone who is looking [00:01:00] into agent security, what are some of the ways people are solving agent security? In this particular conversation, we talk about how people are approaching the agent security problem.

Ashish Rajan: What are some of the blind spots? How are you looking at the identity permissions, and what some of the challenges are still unsolved, and what do you need, especially if you already have a mix of EDR and everything else that a lot of people in security have already bought into for all these years. All that and a lot more in this conversation with Graham from Oso.

Ashish Rajan: If you are listening or watching an episode of AI Security Podcast for the second or third time, I would really appreciate if you take a quick second to hit the follow subscribe button. We are on all audio and video podcast platforms like Spotify, Apple, LinkedIn and YouTube or wherever you listen to your podcasts.

Ashish Rajan: And a huge shout-out to Oso for sponsoring this particular episode as well. I hope you enjoy this conversation, and I'll talk to you soon. Peace. Welcome to another episode of AI Security Podcast. I've got Graham with me. Hey man, thanks for coming on the show.

Graham Neray: Hey, thanks for having me.

Ashish Rajan: Um, maybe to quick start, if you could have an intro with your- from yourself, Graham, just [00:02:00] a brief background of yourself and what are you up to, man?

Graham Neray: Sure. Uh, so I'm founder and CEO of Oso. Uh, before starting Oso, I cut my teeth at MongoDB, where I spent a bunch of time early on. Um, at Oso, we are, uh, fundamentally a permissions company, and what we're helping companies do, uh, is for security leaders who are being, pushed to adopt and drive, uh, agents in their companies, provide them visibility and control over what their agents are doing, with, with access control really being at the heart of, of our view on the problem.

Graham Neray: Um, it's a wild time to be in the industry and kind of... I was like remarking with another friend of mine, like, it's kind of the most fun I think I've had in my professional career at this point. Um, uh, but yeah, that, that, that's Oso and what we do.

Ashish Rajan: Oh, awesome. Uh, maybe actually to set some context as well, I think it's like everything is called agent these days.

Ashish Rajan: When you say agent security, what do you mean specifically?

Graham Neray: Yeah. So specifically for us, where we're-- what we're focused on, and this will [00:03:00] expand over time, but where we see the most adoption of agents is, um, internally at companies starting with things like Claude Code, increasingly expanding to things like, you know, c- um, uh, Claude Cowork, um, and sort of outside of engineering.

Graham Neray: So these are people using LLMs in some kind of a loop to achieve some kind of task. Um, and in specific, we're focused on, you know, those that companies are buying as opposed to building. The main reason for that is just that's, that's where most of the adoption is if you look at, you know, Anthropic and Claude Code and all these other things.

Graham Neray: This is really where people are starting.

Ashish Rajan: Also more from the frontier model rather than the third party? Or is it-- 'Cause I know how people obviously have, say, I may have a Salesforce Agentforce or whatever, or m- I may have a, I don't know, insert another SaaS provider with an AI agent tool as well. So y- but at the moment, the-- when you're looking at or when you're d- going deep dive into the agent security world, you're look- looking primarily at the frontier model agents that are being adopted in organizations.

Graham Neray: Yes, yes. And, and also not into the [00:04:00] agents that people are building. By the way, we spent time talking to people about agents they were building last year. Our conclusion- Yeah ... was no one's building real agents. Like, everyone shipped something to make the board or the CEO happy because someone like me said that they had to.

Graham Neray: Everyone has like a little co-pilot screen. But in fact, like, no one is really pushing those much farther because of a bunch of limitations, primarily around model quality. So it's not people building their own agents. It's less so people adopting agents from SaaS providers like Salesforce, although I'm sure Marc Benioff would like to disagree with me.

Graham Neray: I don't believe the numbers that he's been telling the street. Um, and, um, and it's really about the core, like Claude Code, ChatGPT, the Core app, Codex. This is where we see the most adoption by far.

Ashish Rajan: O-out of curiosity, is there I guess, with the Coworks of the world and with the, um, coding agents specifically, 'cause obviously a lot of people who are probably listening to us or watching this, um, are in [00:05:00] enterprises.

Ashish Rajan: They've always had security. There's defense in depth and all of that as well. Is the, um, I, I guess with the frontier model agents that are there, what do you see as a blind spot that maybe people don't or, or they get surprised by when you have a conversation with them, uh, about how this is used versus what the security exposure is?

Graham Neray: I don't know if I would call this a blind spot, but what I would say is like, I think everyone that I speak with intuitively has this feeling of nervousness around deploying something like Claude Code. And there, and I think the, um, part of what we've been doing is like putting structure and words to what is causing that anxiety.

Graham Neray: Um, so specifically like we, we believe the reason why people are ner- like using Claude Code on your laptop, you know, in a sandbox or whatever, like no one cares. Doesn't matter. Not really a problem and nothing to be worried about. Uh, even using OpenClaw, not connecting it to a bunch of things, whatever, on, on its own machine, that's fine.

Graham Neray: What makes these things dangerous [00:06:00] is the same thing that makes them powerful, and that's hooking them up to production systems and data so, uh, so now why are people anxious about hooking up Claude Code and other systems like that to production systems and data? The reason they're worried about it is because everyone intuitively knows that all those systems are grossly over-permissioned, right?

Graham Neray: Like every app, every person in every company here today has way more permissions to the applications that they could use inside of their company than they actually will ever use. And the reason for that has to do with like the way that model works. Permissions are assigned statically, typically upfront when a person joins a company.

Graham Neray: They're almost never taken away. They might just get added as you take on new roles. This is-- and importantly, this has all been not ideal. Um, I think everyone has felt a little bit uncomfortable with this for a period of time, but this has been acceptable, an acceptable trade-off because agents have fundamentally different constraints than humans.

Graham Neray: The two key ones are one, humans apply [00:07:00] judgment. Like whether or not we want to, whether or not we think we should trust our users, that is fundamentally what we've been doing when we give them broad access. We trust that they wanna make good decisions, they're going to apply judgment, they don't wanna get fired, they don't wanna get sued, they don't wanna go to jail, and so they're generally speaking not gonna do bad and stupid things.

Graham Neray: And then two, there's an upper bound to the amount of damage that a human can do before someone on the security team notices. Um, but agents of course break both of these constraints. Um, agents do not apply judgment. They can be tricked, they can do silly things, they can hallucinate, uh, they can think that they're helping you.

Graham Neray: They can be like so relentless in trying to help you that they will go and find a production cred that they found somewhere in some local file system somewhere, you know, to go do something bad. I hear this story all the time. And then two, they operate at machine speed, so they can do a lot more damage than you or I could do inside of a minute than, you know, anyone else can do inside of an hour, uh, or a day or a week or a month or whatever.

Graham Neray: Um, so I don't know that I would call this like a, a blind [00:08:00] spot, but it's more like in the conversations that we've been having with CISOs and security leaders in the market, it's more about like putting to words an anxiety that everyone kind of intuitively feels like they have but isn't exactly sure how to explain.

Graham Neray: And the reason why it's important to have a mental model for how to explain and think about this problem is that only at that point can you start to build solutions for it.

Caleb Sima: Yeah, you know, maybe one area that I've been sort of confused about that you could help Um, is that on the human side, you know, least privilege and the capability of managing even humans and their both, I think, destructive path, quote unquote, and what they have access to has not been solved.

Caleb Sima: Like, we have not been able to get to a point where even slow, non-automatic humans have the ability to where they can access the things they need when they need it, at the time they need it, for only as long as they need it, right? This is the ultimate happy [00:09:00] path vision for a human, is that they can access the things they need with no friction at whenever they need it, and no longer when they don't.

Caleb Sima: Um, and yet now what we're doing is we, we are going through and saying we need to solve the agent security, uh, problem, which is clearly way faster and way more variability with, to your point, no accountability or fear in these things. Um, and then we're gonna solve that, but we haven't even solved the human side.

Caleb Sima: So, you know, I have, I always have had this sort of doubt when I see these things because I'm like, "Well, if we can solve the agent problem, then we can surely solve the human problem," and that is a way bigger today problem than the agent problem ever is. So if you can solve the agent, you could solve the human.

Caleb Sima: You would make way more money today, because there are CISOs everywhere who are just saying, "Hey, can I help solve the

Graham Neray: human problem?" You asking all of those who focus on human access now?

Caleb Sima: Well, [00:10:00] this is my question, right, is like, um, if you can solve the agent, then theoretically my logic says you can solve the human.

Caleb Sima: Um, but it seems like, you know, to me, there's some sort of mismatch. There's a, It's incongruent to be able to say, "I can solve agent, but I can't solve human."

Graham Neray: I, I hear where you're coming from. I guess for what it's worth, like we have, you know, we have plenty of ideas as to how to solve like human permissioning as well.

Graham Neray: I think the reality is, is just the juice hasn't been worth the squeeze. So this is less about our ability to solve the problem and more about the importance of solving the problem. Like I think everyone in security has been aware of like an over-permissioning as like the reality for a really long time.

Graham Neray: And I, I'm not sure that people haven't solved it because they don't know how. It's just never risen to the top of the list of priorities. By contrast-

Caleb Sima: Oh, I just... It seems like I, I push maybe a little, like it's... You talk to any CISO and you're [00:11:00] like, "Hey, is privilege on your people a problem?"

Caleb Sima: Absolutely. They're gonna be like, "Absolutely." "Hey, if you could solve that in this way where it's no friction on users, would you pay a lot?" "Oh, absolutely. Where is that product? I will give you my wallet." Right? I think the struggle has been, um, it's so difficult and so challenging to solve this problem because, you know, as an, as a CISO you're dealing with humans, right?

Caleb Sima: Which as we know not only, uh, have their pros, but are also like, "Hey, I don't wanna go through friction. I don't need more red tape. I don't need more policies, click, approve, approve. Like I wanna be able to do what I want when I wanna do it, and it's your job, security, to figure it out when I need it and when I don't.

Caleb Sima: I don't wanna get blocked," right? And so, you know, you have to then deal with the sort of the reaction of the culture of your employees in your company [00:12:00] alongside of what technology or stack will allow me to even do this well. Most forward-leaning, you know, larger companies have built their own versions of privilege management because of this problem, right?

Caleb Sima: Like, and so like, you know, this has been like a super hard challenge, and it's not that CISOs, this isn't at the top. It's definitely in the top five for sure. The biggest issue is there's no real solution that I think people have been able to get ahold of that will solve this problem in a way that works well.

Caleb Sima: Um, and so anyways, I don't wanna, Graham, I don't wanna go too much into, uh, that human problem, but that has always been, um, some of the challenges I've, I've felt when I hear agent security permission. You know, oh, you know, agent security is to your point about authentication and authorization. Yeah. But if you can do least privilege on an agent, why can't we do least privilege on a person?

Graham Neray: Well, yeah, we'll do that next. Don't worry, Caleb.

Ashish Rajan: I was also gonna extend that... [00:13:00] I, I was gonna extend that example a bit because, uh, one of the usage patterns for AI agent has been the fact that Ashish prefers to use his credentials for the AI agent to run rather than, um, I guess say giving the AI agent its own dedicated credentials.

Ashish Rajan: And at that point in time, me as a CISO or security person reviewing, or even a SOC person, I can't really differentiate between it's being Ashish versus Ashish's AI agent doing this activity. And I think maybe that's kind of where the analogy comes in from. We're ultimately trying to solve the human, uh, permission problem in a, in a way.

Ashish Rajan: Uh, but I'm curious as to with authentication authorization, 'cause at least my understanding was the standards like the OAuth 2.0 and steps are not enough, especially if you add the multi, uh, stage A- AI agents and tooling and all that. Maybe before I dive into that, do you wanna just set the scene for what is the AI agent security model today as per you from a authentication authorization perspective?

Ashish Rajan: How is this done? Um, like what are some of the options that [00:14:00] people, uh, come across? 'Cause I wanna at least lay the land for people who are less... 'cause we've al- we've obviously said authorization, authentication, but I'm sure there are other ways to solve this particular problem. How, what are some of the ways that people have tried solving agent security?

Ashish Rajan: Just start there, and then we can narrow down into authentication authorization, 'cause I'm curious as to, uh, why, why do- go down this path when you could've just gone, I don't know, endpoint security, block all agents, or URL back. I mean, this is so... I'm, I'm not gonna ask you the question, but curious as to what other things have you seen?

Graham Neray: Yeah. I mean, if you look at, um, if you look at like the market today, the way what my o- my observation, and I'm curious whether this tracks with what you see, is that everyone who's ex- been around, uh, is basically trying to pull the market towards them So, you know, the end, the EDR vendors are like, "No, agents happen on the endpoint.

Graham Neray: You need to, like, know what's going on on the endpoint." And then, like, the, the, um, the, like, DLP companies are like, "Yeah, endpoint maybe, but [00:15:00] really this is, like, all about data flow. That's what you really care about. You care about data flow." And then the, like, CASB-style companies are like no, no, it needs to happen outside the endpoint because it's all about..."

Graham Neray: You know, so I, like... And I think each of them is basically just playing to their strength. Similarly, like, you look at companies that have, you know, like, um, you know, do a lot around, like, uh, like core IDP-type stuff. These companies are saying, "No, it's all about inventory management and identity, and that's the most important thing."

Graham Neray: Um, and so To make a long story short, like I, I don't know that I've seen like a million different approaches to solving the same problem. It's more like different people sort of saying, "No, this slice of the problem is the one that you wanna focus on first." And obviously, you know, we have our own point of view on that as well, which is of course permissions.

Ashish Rajan: Yeah. And I guess to your point, b- uh, between authentication, authorization, permission, obviously we spoke about I think Caleb touched on this a bit. Least privilege has been like... how do you even define least privilege is a big question in itself. [00:16:00] Uh, I'm not even talking about humans.

Ashish Rajan: Let's just say even for AI agents as well, what are you finding as the, uh, some of the core problems in this space with authentication, authorization? Uh, and I'm not talking from a solution perspective, I'm thinking more from a this is how you see people use AI agents and this is where they see as security challenges, particularly with permissions and authentication.

Graham Neray: Yeah. So there's like, going back to your question from a second ago, there's like, I'm sure there's many ways to do this, but fundamentally roughly two models for how we're seeing people deploy agents internally. One is you give them their own identity and their own set of permissions, and then the other like sort of like a service account.

Graham Neray: And the other is where you have the agent adopt the permissions of the human that's operating it. Um, the pro of assigning agents their own unique identities and giving them their own permissions is you naturally are going to start with something much more tightly scoped, and so that sort of limits the blast radius.

Graham Neray: Um, the con is that, uh, or a key con that we've seen is that, [00:17:00] um, now those privileges are like not at all tied to those of any user that might be interacting with them, and that creates a risk of privilege escalation. So as an example, there's like a customer that we work with who was, uh, building an agent that they wanted to deploy.

Graham Neray: I won't share that many details, but basically in their alpha testing caught, the PM caught that she was able to escalate her own privileges, and the reason had to do with, you know, them architecting it in this way. I don't mean to pick on them, by the way. I think everyone's-- I've seen this like a bunch of different places.

Caleb Sima: I was gonna ask, you know, in that could you give some examples of, um, what technologies you've seen people give their own identity to these agents? Like, uh, I mean, they're clearly not doing it with like Claude Code, but like what are some like real-life examples of, oh, they've created an identity and attached it to this kind of agent?

Caleb Sima: What you're seeing in the enterprise is, um, go to Ashish's point, people will use your own, you know, sort of identity to run your agents, or people will [00:18:00] assign identity.

Caleb Sima: You said, hey, there are people in enterprises that assign identities to agents. Yeah. And love to get some like real world examples of that. Like, um, what have you seen, uh, what services, what software have you seen where you are assigning identities to those agents?

Graham Neray: Yeah. So the, the where I see that happening the most is in, um, any sort of like background agent.

Graham Neray: Um, so the use cases tend to be things like, you know, triage or review or something like that. Um, so, and, and like, I mean, it's early days, but like Notion is actually doing a lot of interesting work in this area. Um- And yeah, I think I don't, I don't, I don't know how aggressively everyone is adopting Notion agents, but I would say like, um, in terms of companies that have built a story around how you would have agents with their own identities doing their own thing, I think Notion's probably the furthest along.

Graham Neray: And part of that is that they're like painting the use cases for how you [00:19:00] might, what you might actually do. I think this is a big part of the challenge, by the way, is like everyone needs to sort of expand their imagination. They're used to like prompting an agent back and forth, but there are obviously all these cases where they can act autonomously.

Graham Neray: Um-

Caleb Sima: So more, more cloud agents is- Yes ... what you're saying. And Notion has a great example of where they're sort of building these things, um, uh, as integrator, integration background task agents is where they have their own identities.

Graham Neray: Yes. And I would be not at all surprised if this happens with Anthropic in the next three months.

Graham Neray: Like Notion is built on Claude managed agents, I think.

Caleb Sima: Yeah. Another couple good, good examples of this, Cursor has cloud agents- Yes ... which is super popular.

Graham Neray: Yes.

Caleb Sima: Um, you definitely have Perplexity Computer, which is now- Yes ... running cloud agents. Uh, you have Cognition with Devin that's also- Yes ... actually making a lot of enterprise plays with cloud agents background.

Caleb Sima: Yes. They, they, all of these sort of have their own identity. Which- Yes ... [00:20:00] what would actually be interesting is this is, this is maybe the gap, is do companies like Notion, Perplexity Computer or Cognition or Cursor cloud agents allow you to give them an identity, right? Like is, like and how do you do that?

Caleb Sima: How do, how does that work with them? Um, this is an aspect that would be interesting to know more details about so that, you know, Grant, if I would take your product, okay, well I have a permissioning identity o- auth Z product and system, how do I now apply it to Cursor cloud, you know, like, uh, Perplexity Computer, Notion agents so that they're within that ecosystem?

Graham Neray: Yeah. So our view on this, and again, this is, you know, uh, uh, just our take, um, is that the, the governance layer has to exist outside of those products. There's a few- Yeah ... different reasons why. One reason is, at [00:21:00] least today, there's no standardization. Everyone is in this mode of trying lots of different things.

Graham Neray: You see this in the data, like Datadog released a survey, I can't remember when it was, um, sometime in the last couple months, that showed that, um, the majority of people adopting these products are adopting multiple at once, and that that trend is only increasing. So like the number of companies using whatever it was, like six models has increased from, you know, last year to this year and so on and so forth.

Graham Neray: Um, and so the space is very, very fragmented. The, as a result, given that the space is fragmented and there's no clear winner- Investing in a single platform to define all your permissions and all that other stuff kind of makes no sense. That's the first reason. The second reason is the permissions that we really care about are the permissions of the other platforms that the agent is going to talk to So, uh, it's not so much about configuring Notion permissions or Claude permissions, it's about pr- configuring the permissions that that agent has when talking [00:22:00] to Slack or when talking to Google Drive or whatever.

Graham Neray: Um, and so it's, it's like this layer that really exists outside of those products. I mean, Anthropic has big ambitions. Maybe they wanna own it all. Who knows?

Caleb Sima: Can I ask one... You know, one other pro- I'm, I'll may- build this as a simple example. Yeah. Okay, I have an agent that integrates with Gmail. Yeah.

Caleb Sima: Um, and let's go back to your original point, which I think is a very big point. It, with humans, if I have an EA, my EA has, has access to my Gmail. Yeah. So clearly I know, or at least 99% hope through my interview process and hiring and et cetera, that my EA won't then, like, send all my personal information to random people on the internet through my Gmail, right?

Caleb Sima: Yes. Like- Yes ... that person will suffer, consequences, uh, for those actions. Now, agents on the other hand, obviously don't have that capability. And so there becomes this difference where normally in an EA, I will grant my [00:23:00] EA obviously access to my Gmail, which includes reading my Gmail, sending my Gmail, editing my Gmail, blah, blah, blah, blah.

Caleb Sima: But now when it comes to an agent, hey, perhaps I only want read access, or perhaps I, I only want draft compose access, where they don't have read access, or they do have read access, but only from certain types of senders. Because now when it comes to an agent, I actually need to require more stringent micro sort of permissions on this.

Caleb Sima: But, you know, our problem in our world of security is, hey, the providers don't offer that, right? Like, they either say, "You have access to Gmail or you don't," quote unquote, right? And there's no way to be able to say an agent has more micromanagement policies than, say, uh, an individual. Have you run into this?

Caleb Sima: Is there a solution for this? How are you sort of think about this problem?

Graham Neray: Was this deliberately a softball, Caleb?

Caleb Sima: No, it is not a softball. This is like a real- [00:24:00] Yeah, this is like a real problem. I'm glad it is a softball for you, but yeah, this is... Yeah, like I, for sure all readers I think will agree with this as the biggest, one of the biggest challenges when it comes to agents.

Caleb Sima: That would be my guess.

Graham Neray: Well, yeah. I mean, this is why you need this governance plane outside of those products too. I- I'll, uh, actually even take it a step further. You know, the average company, like I talked to a relatively small company the other day that told me they have 200, more than 200 SaaS products that they use.

Graham Neray: The- Yeah ... I talked to a large private equity firm that said they use 7,000 different software vendors, okay? And there's probably like, you know, everything in between. It's not typically the case that someone in security wants to go vendor by vendor and say, "Oh, this looks like a delete, and that looks like a write, and that looks like a this, and that looks like a that."

Graham Neray: So problem number one, as you just described, is you wanna be able to say within a single piece of software, "Hey, I want the agent to be able to do these things and not those things." I would say the problem's even bigger than that, and the problem [00:25:00] is not just inside of one product, but, like, across your fleet.

Graham Neray: A, a, an easy example is I think a lot, like I talked to the head of security at a large technology company, the household name that everyone would know, um, and what they were t- you know, I the question that they posed, um, to the group and I ultimately posed back to them is, like, what is the, the one red line that you will draw, like agents cannot do this?

Graham Neray: And specifically in her case, she said agents should not be able to take destructive actions. So that's actually a blanket rule that you could consider applying across anything that you connect any agent to. Um, but it's hard work because it requires going- Yeah ... and classifying all the different tools that they could have, and it involves having, you know, a policy layer in between the agent and those products that says what they can and can't do.

Graham Neray: I mean, spoiler alert, that's what Oso does, but, um, that's why I asked you if this was a softball.

Caleb Sima: Uh, so maybe, um, we wanna be vendor neutral here, but what I do would like to talk about that you would clearly know a lot [00:26:00] about is how. How do you, how does this happen, right? Like obviously when you think about it from, uh, a practitioner's perspective, this is a happy road.

Caleb Sima: Like if there's something that I can magically wave a wand on and it can then take all my third-party SaaS, all my host systems, all my production infrastructure, and then give microgranular permission sets to agents, you know, across the board very easy, and at the same time, like I don't wanna come up and manage those rules and permissions, so there needs to be some AI magic that says, "I've looked at your," you know, "what your agents need and don't need, and I've granularized them for you," and they don't add friction or there's a feedback loop to ensure that, you know, if there is any friction, it becomes easier.

Caleb Sima: Boom, magic is done. You now have quote unquote "least privilege" for agents in a way that doesn't require a lot of headache. So that is like the [00:27:00] magic, happy path. How would this even be accomplished? Like what's the deployment model? What is the, uh, ability for... How do we as, if you're saying, for example, I'm just gonna, you know, "Oso does this."

Caleb Sima: As a CISO, it's not just Oso that if you say you can do this, I need to both look at because that's truly valuable. But number two, I need to look at your competitors, and I need to look at anyone else who's doing this. So help us understand what should we expect from these kinds of solutions?

Caleb Sima: How are they deployed and implemented? And how is this working so that we can be smart about looking at these products and determining, "Oh, can we actually reach this nirvana? Is it possible?"

Ashish Rajan: I'll probably add the red line thing as well to what you said, Graham, earlier. With the red line in place as well.

Ashish Rajan: 'Cause I, I guess there's one thing to be able to blanket rule for least privilege across every agent, but also while maintaining that red line across. 'Cause I guess they're still two different things, minimum permission versus, hey, I don't-- you should not cross that red line, that [00:28:00] no destructive action.

Ashish Rajan: I'm curious about great question as well. So over to you, Graham.

Graham Neray: I mean, look, just to be clear, and I don't mean to, like, overstate the capabilities of Oso, like automated least privilege is absolutely... That is our North Star. Are we, you know, have we achieved the, like, magic wand vision that you just described, Caleb?

Graham Neray: Not yet, but that is, like, what my team is working on every day and every night, from now until the end of time.

Caleb Sima: Yeah, so maybe you can, you could take us to what should we expect? As, as buyers and operators here, what should we expect, from a real- a realistic perspective? And by, and by the way, like no CISO, well, except maybe a few, expect that there's this North Star that's gonna be delivered.

Caleb Sima: Um, you know, we all have practical, but what sh- you know, maybe from you, you can help us set, okay, how far are we, in getting to this vision? When I go look at this space and the products in this space, just educate us, right, on, on that.

Graham Neray: Well, look, I'll start by saying there's no shortage of companies attacking a problem [00:29:00] that looks roughly like this, and I don't claim to know about all of them.

Graham Neray: Um- Of course. Yeah ... there are de- there are definitely some patterns. I mean, one of the key patterns is what I was referring to before, where companies are kind of trying to pull the market towards their worldview. And so, you know, as an example, I was looking at, um, a customer asked me to look at, like, a DLP company they were looking at, um, the other day that has a focus on agents and, you know, DLP.

Graham Neray: And so their whole thing is all about, basically using AI to automate DLP. Um, and this is great. This is very interesting. Um, the appeal is that it should sort of naturally and intuitively apply across tools. The problem is it's, like, only focused on the data piece and sort of has no insight or control over like, uh, for instance, an agent going and deleting a production database or something like that.

Caleb Sima: So yeah, like what, what, where are we? Are we-- you had mentioned, hey, one of your prospects that you were talking to is like, "We don't want agents to... We have a policy, agents cannot have destructive-"

Graham Neray: Yeah ...

Caleb Sima: you know, actions. [00:30:00] I think that's a really good place that's, to me, both practical, sound, and a great starting point.

Graham Neray: Yeah.

Caleb Sima: Y- today, how do we expect when we go into this space of AuthZ for agents, how far are we from that? What does this look like? How do we deploy? You know, does it require all the integra- Is it a proxy that gets set up? Like, is it agents on hosts? How, uh, how much work do you think it takes for us to at least realize some vision of this agent cannot do destructive actions?

Graham Neray: You know, I wish there were like a silver bullet answer to give you. I would say my experience, our experience at Oso over the last year has been it's not as clean as just like deploy a proxy. I'll give you some examples in a sec. It's not as clean as just like do this one thing and then you have it. So-

Caleb Sima: Yeah, yeah

Graham Neray: But, but that's, but that's very important because, uh, or it's very relevant. One of our core, you know, there's like every company has sort of like [00:31:00] principles or values by which they're thinking about solving the problem that they have, and one of the core ways that we think about this problem is that the solution needs to be holistic.

Graham Neray: Why should the solution be holistic is because of the market dynamic that we described earlier on, which is that the market is extremely fragmented. Um, and so the idea of, you know, uh, well, I can support what's happening on the endpoint, but I don't know what's happening if they do it over there or whatever, like that's not really that helpful to security teams.

Caleb Sima: Oh, yeah. And, and Graham, I think we, you know, we, we get that. Like, you know, the thing that I'm, I'm sort of pushing on you for is, hey, how far does this go? Like, when we go and we onboard one of your, one of these products, your product or others, I don't want to be just specific about yours, but you should speak as an expert, you know, obviously in the space.

Caleb Sima: But like, do I... Is it, does it come with a proxy? Do I also have Chrome plugins? Do I also have endpoint? Do I have all three? Like w- like where have you found, uh, do I integrate with Salesforce, with Octas, with, where do you... How does this work to where I can say, [00:32:00] okay, all agents, no destructive, you know, properties?

Caleb Sima: Does this mean Claude Code gets funneled, all my engineers in Claude Code gets funneled in this? Does this mean it's just like people use Gamma.app as a presentation, you know, AI thing that gets funneled through this? Does this mean Notion agents get funneled? You know, like h- help me sort of what's the reality today?

Ashish Rajan: I guess, I guess another layer to this is also because a lot of people are curious about how, uh, what does this change in their organization as well? 'Cause obviously they already have, to what we were saying earlier, a lot of enterprises already have a EDR provider, DLP provider, all of that. So wh- and it also gives them a perspective for, uh, hey, uh, the permission model or the authen- authentication model.

Ashish Rajan: This is one way, uh, to approach this, but this, these are the ways you can approach this particular, solving this particular problem. Uh, we agree, we all agree that we have this problem of permission and agents. It's just about how are you seeing that the, uh, the practitioner on the other end can apply a solution to [00:33:00] this?

Ashish Rajan: That's kind of where we're coming from.

Graham Neray: Okay, cool. So I see a few different ways that companies are approaching solving this problem. Fundamentally, it's about like, yeah, wh- where you're doing any kind of like detection and enforcement. Um, so you have a chunk of companies that are really focused on the browser, some that like offer a secure browser, others, uh, like Oso that offer a browser extension.

Graham Neray: Um, and so that's one piece of the equation. The second piece of the equation is what's happening locally on the endpoint. And so this is, um, this is, you know, about, you know, agents like Claude Code that run actually on someone's machine, um, and having some sort... I mean, I, we've seen a couple different approaches.

Graham Neray: One is you have like a sensor, like a, an old school agent, uh, running locally on the machine that's watching what that thing is doing and reporting back. You see that as an approach. Um, we see some customers who like that and other customers who feel like they don't want to install yet another agent.

Graham Neray: Uh, you know, we had this issue with a customer I was talking to yesterday who found that, you [00:34:00] know, un- unsurprisingly installing a thing on the endpoint messed with their VPN, or I, I don't remember exactly what it was. Um, okay, so that's the next thing you could do is you could install an agent on the machine itself.

Graham Neray: Um, a third model is, uh, using an edge proxy. You, there's actually, you could, you could run a proxy locally on the machine or you could run an edge proxy. Um, you see this oftentimes, it sort of takes the shape of s- something that people might call like a gateway or an MCP gateway, um, where you're funneling all the traffic through that.

Graham Neray: Um, that, that solution is good for a piece of the problem, but not the whole thing. It can... If you can configure all the sanctioned AI to flow through that, then you get all that visibility, but you don't know about anything that's not flowing through that proxy. Um, and so it's a long-winded... Where I was coming from originally is like I don't, there's no single one thing.

Graham Neray: I think you basically have to use a combination of these things, um, in order to get the full picture. So browser extension, you know, some version of something running on the endpoint or an integration with an EDR also works because they already [00:35:00] have something running there, and some way to ensure that the traffic that you care about is flowing through somewhere that you can see

Caleb Sima: yeah, it's... And, and that's exactly the right... So like, okay, hey, listen, like in deployment, hey operators, here's what you should expect.

Caleb Sima: Yeah. You're gonna need an endpoint agent. Yeah. You're gonna need some browser plugin. You're gonna need a proxy. Yeah, th- this basically mimics, you know, the, the 10 years ago problem of you need endpoint, you need network, you need, app level, right? Like this is sort of the- Yeah. You're just now, you just need, you need the same levels as, but now you're looking at something different, right?

Caleb Sima: Yeah. Um, so you've gotta have that. So anytime you expect to go talk to, you know, products or companies in this area, here's what you should expect. They're gonna come with all three, hopefully. Or if they're not, then, you know, there, there are gaps here that you've gotta- Yeah ... understand, right?

Graham Neray: Yeah.

Caleb Sima: And then I would say probably second to that is, you know, even a little bit more dive in the detail, which is, okay, great, you know, what is the user experience or setup or [00:36:00] configuration like?

Caleb Sima: So, you know, okay, if I'm coming in and I'm like, "Okay, great, I got all three of these things," do I have the ability for Notion agents to go through this system today? Is that done through the browser plugin? Is that done through the proxy? Do I have the ability to, I think... Or hey, right now you can do coding agents.

Caleb Sima: That should be expected, like table stakes or anyone in this, you know, hey, coding agents, you should be able to make a policy that says no destruction, and it should be able to do it. Um- Yep ... or, or common things like Salesforce agents or other kinds of things going there, or only production development agents that your company or building needs to go in this.

Caleb Sima: Yeah. Is Claude CoWork able to go through these things? Like just help, help us understand our expectation there.

Graham Neray: Yeah, I wish I had a simple answer for you. I would say it varies like crazy, and this is part of the job of companies like us, I'm not saying it's only us, uh, is to basically solve for all these cases.

Graham Neray: Um, and it's a discussion that we're always [00:37:00] having. It's like, okay, well, uh, if you wanna do, you can use Claude Hooks for that piece of the equation. Cursor has a Hooks implementation, but it's less feature-ish than Claude Code. These guys claim they have a Hooks implementation, but it's not documented.

Graham Neray: Those guys have no API. Long story short, like it really varies. Like it really, really varies. Um, and so a lot of what I would ex- I think if I were a customer, I would be asking not necessarily how you solve for all these things at once today, because it is evolving very quickly. I would be asking like architecturally or philosophically, what's your approach to supporting all these things, assuming that there will be agents in all these things?

Caleb Sima: And what do you think, um, are the ones that you have seen so far in your experience that people are, you know, top three, these are the things for sure everyone wants a no destruction or policy management on?

Graham Neray: Uh, yeah, no disruptive actions. I see that a lot. I think everyone wants an audit log because everyone's afraid [00:38:00] about, you know, f- being able to go back and explain what happened when something bad happens.

Graham Neray: The, yeah, the, the more co- the, so the common ones that I've seen in terms of, like, policies would be like, tell me when anyone is using a new piece of a new AI tool that I haven't seen before Um, just like alert me on that. And unre- anything unreviewed, let me know. Uh, and actually I've seen very few...

Graham Neray: Everyone says they want blocking, but I don't really think most people want blocking. I think what people really want is like human in the loop. Um-

Caleb Sima: Yeah, this is the, this is the always untold story of security. We want blocking- Yes ... but no one ever uses it.

Graham Neray: Exactly. Exactly right. Exactly right.

Caleb Sima: Decade, decades old feature request.

Graham Neray: Exactly right. And honestly, and the... I thought I was being so clever. This is like probably late last year, I was like, "Well, we don't need to build blocking. Don't worry, like no one's ever gonna use it." But like the reality is everyone wants it anyways, and so obviously that's what we've done. What I think people will, and I don't have the evidence to support this yet, but like let's check in in three months.

Graham Neray: What I think people will start to turn [00:39:00] on is human in the loop. That is to say they still are trusting their user to apply reasonable judgment. They just want them to actually look closely at what the agent is doing, um, and not just hit sh- you know, smash on dangerously skip permissions or whatever. I mean, so a lot of the work we're doing at Oso is actually thinking about what is a good user experience that gets the end user to like closely consider what they're doing, uh, without com- you know, and without hitting the smash button, but without also being a huge pain in their butt.

Caleb Sima: So some of my struggles have been, especially when it comes to quote unquote "agent security," and by the way, I'm going to... I know there's a difference between sort of agent security per se versus agent auth Z. Sure. Although I feel like it's all getting kinda munged. Um, but the one thing that I struggle with is everyone that I've talked to, every company, every startup pitch I've ever seen all has the same how.

Caleb Sima: They all can get all the [00:40:00] analytics, right? They can get the visibility. Every single one of them knows eBPF hooks- Yeah ... browser plugins, proxies, you know- Yeah ... like they could all deliver, "Here's exactly what your AI agent does." But what I have not seen very clear is people being able to say, "Here is why it does it," or whether this is good or bad.

Caleb Sima: And the only thing that has happened here is blacklists of things like, "Oh, RM-RF/ is clearly bad." Yeah. You know? Like we- Yeah ... are gonna blacklist that. Uh, "Oh, things that look like malware are bad, so we're gonna- Right ... blacklist this." Right. Um, but they have not yet been able... No one has been able to, to do the important part.

Caleb Sima: Like to me, it's like table stakes to get the analytics, but no one has been able to the important part, which is, okay, well what do you do with it, right? Like why? Like to, to your example, if an agent is doing something, should it have permission [00:41:00] to do it or should it not have permission to do it? And how can you tell?

Caleb Sima: Without manually updating a policy. Like for example, rm-rf/ might actually be a viable and correct action depending upon the context of what's going on, right? And it just doesn't seem like I've seen any ones about like anomalous detection or, but no one can say whether something is good or bad.

Caleb Sima: How are we-- What are we gonna do to sort of solve that problem?

Caleb Sima: If I put a policy that says all agents should not do destructive action, right?

Caleb Sima: Yep. This goes back to our blocking, uh, quick discussion, right? Yeah. So you say, "Okay, no destructive action." But actually speaking, if you're now using, thinking about Claude Code, destructive actions happen all the time.

Graham Neray: All the time.

Caleb Sima: Right? So you- Yeah ... need to be able to do destructive actions,

Graham Neray: right? Yes.

Caleb Sima: Now the difference is, well, are you doing dest- destructive actions like RMF, RMF-ing the host?

Caleb Sima: [00:42:00] Yeah. Or are you doing destructive actions as in you are RM-ing a database in production? Yes. Like where do you, how do you know, like, okay, w- when I create that policy, no enterprise can create that policy in that sense because they'll be full- I see ... of false positives, right?

Graham Neray: Right, right, right. Okay,

Caleb Sima: okay, okay.

Caleb Sima: And, and you cannot block that because of the full of false positives. Yeah. So, and you cannot get any real analysis out of it because of the full amount of false positives. Yes. Yes. Because you're not able to say the why. If I'm RM-ing- Right ... something, is, uh, is this bad or good, and how- Yeah ... can you tell?

Caleb Sima: And so- Yeah ... when I think about both these agent security solutions and I think about permissioning in AuthZ, this becomes super critical problem. Um- Yeah ... how do I tell that this permission should be applied or not applied if I don't know the why?

Graham Neray: Yeah. So I think like, it's a great question. I think today the way I expect people to solve it in the coming months is through human in the [00:43:00] loop rather than...

Graham Neray: So, you know, you, you say block all destructive actions or really just escalate to the user. Maybe there's certain kinds of actions you wanna escalate to the security team because they're too important, administrative actions or something like that. Um, but escalate to the user because ultimately we're trusting our users to know what to delete and what not to delete, and just making sure that they're looking closely at what the agent is doing.

Graham Neray: I think medium and longer term, there's actually a lot more that we can do. So as an example, everything that we've just talked about at a permission, around permissions thus far has been kind of at the action level or the tool level, you know, can delete or cannot delete. Um, there, you go a click deeper, um, to the, like the resource or the data level, and that allows you to be a lot more precise in the kinds of things that you're willing to allow or not allow.

Graham Neray: So for instance, you might say You know, I'm, I'm okay with the agent deleting things on the host. I'm not okay with the agent deleting anything via the AWS CLI or, uh, [00:44:00] actually that's probably not, not the example I really wanted. Maybe in GitHub you might say, "I'm okay with people doing whatever they want inside of this repo, but this repo is sacred, and so the only thing you can do from that is like read operations."

Graham Neray: Um, and so, so yeah, two things. One, I think people will... And this is, you know, Oso's direction as well. W- we're gonna start going a click deeper beyond from the action level down to the data level. And then the second thing is being able to take into account what has happened previously in the session when informing what should be allowed next.

Graham Neray: So like a, a kind of a trivial example would be like, you know, I can read from a local Postgres server on my machine, I can write to a public Notion page. I, as Graham, know that that's not something I should do in sequence. Um, Claude has all my permissions and could conceivably do that in sequence, but I should be able to put in place a policy that says something like, "You can't expand the audience of data from something private [00:45:00] to something public.

Graham Neray: An agent can't do that." Um, and so these are sort of like more the flavor of the kinds of things that I think we're gonna start to see in the next six to 12 months.

Ashish Rajan: Okay. Awesome. One final question, uh, from, uh, from at least from, from my perspective, because we've understood so far what's the possible ways people can s- approach agent security.

Ashish Rajan: We've heard about AuthZ, aut- aut- authentication authorization as well. I'm curious as to people who are deploying this, clearly an enterprise have already got a lot of ecosystem of security products already present. Uh, if you were to kind of tell people who are approaching this problem today and are very conscious about, "Hey, I wanna do agent security and I do, I wanna do it right.

Ashish Rajan: I may be even I already have an EDR or DLT or whatever," what do you see is the right way to approach the agent security problem today? Um, and obviously you can't boil the entire ocean. Are there any specific one or two things that you normally recommend CISOs to go start with, uh, especially who may not be at that stage, uh, [00:46:00] are getting to that stage where, hey, I'm seeing this explosion happening in my organization?

Graham Neray: Yeah, I think there's like one or two things that I would be keeping in mind becau- and, and it all has to do with the fragmentation. So because of the fragmentation, I would be looking at like, where do I think I'm going to get the most breadth and, and like today and over the next six to 12 months? And then related to that, like you want to be able to hook into your EDR.

Graham Neray: You want to pipe all these alerts to your SIEM. You you may have a bunch of other infrastructure in place, and all these new products are not gonna replace all those things. You want ones that can slide into those. Um, and so the... I guess those are the two things that I would be thinking about.

Ashish Rajan: Also, take the existing ecosystem that you have to your advantage rather than just building an entire new one.

Graham Neray: You can find OpenClaw using CrowdStrike. So just find the thing that integrates with CrowdStrike rather than buying something that deploys a separate agent on the machine or something like that, mm.

Ashish Rajan: Cool. That's all the questions that we had, but where can people find, uh, [00:47:00] more about Oso learn more about what you guys do and, uh, get to connect with you and maybe ask some further questions about this as well?

Graham Neray: Osohq.com. osohq.com.

Ashish Rajan: osohq.com. I'll put that in the show notes as well, and, uh, I guess I'll put your LinkedIn as well, so people do connect with you and,

Graham Neray: uh-

Ashish Rajan: Yes, please ... learn from you as well, man.

Ashish Rajan: Dude, but thank you so much for doing this. Yeah. And, uh, thanks everyone for tuning in as well. Uh, looking forward to more conversation. Thanks, man.

Graham Neray: Okay, thanks for having me.

Ashish Rajan: Thank you for watching or listening to that episode of AI Security Podcast. This was brought to you by Techriot.io. If you wanna hear or watch more episodes of AI Security, check that out on aisecuritypodcast.com.

Ashish Rajan: And in case you are interested in learning more about cloud security, you should check out our sister podcast called Cloud Security Podcast, which is available on cloudsecuritypodcast.tv. Thank you for tuning in, and I'll see you in the next episode. Peace.

‍