Caleb Sima: [00:00:00] AI is a better reverser, better exploit generator, better web app attacker than I would say 98, 99% of the industry is today.

Ashish Rajan: All those years of spending and everything and you're like, "What was the point of all that?"

Caleb Sima: The point of it was to give knowledge to AI to know how to do it. So I can be a 14-year-old, and I can still do this.

Caleb Sima: Can we identify vulnerabilities in our software and our codes really, really well? Yes. We could do that in seconds, in minutes.

Ashish Rajan: You have the technical patches, but you still need to convince Bob in accounting or finance or wherever to patch the thing.

Caleb Sima: Attackers move at machine speed, but defenders move at meeting speed.

Caleb Sima: Does that mean anyone can just run a model, call it misconfiguration, have it hack things, and you not be held legally responsible for it?

Ashish Rajan: In one of the AI readiness workshops that I was running for a security team to basically build up the security with AI capabilities to they can be AI forward, [00:01:00] we were discussing the Anthropic Threat Report that came out in September 2026, and some of the insights over there were very fascinating, so we wanted to talk about that on the AI Security Podcast as well.

Ashish Rajan: So in this particular episode, Caleb and I break down some of the scenarios that happened around the whole OpenAI agent being loose, Anthropic, Meta, Google, everyone basically having their agents go out on the internet and start attacking other infrastructure. We also speak about some of the interesting takeaways that we found in the report particularly interesting, around the lines of how attackers are now building AI capabilities where while your incident response team is responding to an incident, the attacker's AI harness has detected the fact that it has been picked up by a security product, and it starts rebuilding it All that and a lot more in this episode of AI Security Podcast.

Ashish Rajan: As always, if you are here for a second or third time and have been finding the episodes valuable, I would really appreciate if you take a quick second to drop, to hit a follow or subscribe [00:02:00] button, whichever podcast platform you listen or watch us on. We are on Apple, Spotify, YouTube, and LinkedIn, and wherever you consume your podcasts from.

Ashish Rajan: I hope you enjoy this episode, and I'll talk to you soon. Peace. Hello, and welcome to another episode of the AI Security Podcast. Today, we're talking about the Anthropic Threat Intel report that came out in September 2026. Yes, not that far ago. But it's a regular pattern of them re-releasing report. The first ones were in March, then August, then November.

Ashish Rajan: But they were all in last year, so 2025. We got a chance to discuss this back then as well, but this time around it's a bit different. Maybe I'll set some context on the report, and would love to hear Caleb and I just basically... We-- Basically, Caleb and I have taken the time to kinda go through the report, and we're gonna share and talk about what we found interesting from the report.

Ashish Rajan: For context, as I said, the report is shared on a periodic basis. This is basically everything that has been I guess I was gonna say used, but misused, uh, misuse of Claude. Yeah.

Caleb Sima: And could I make it maybe a little bit broader? [00:03:00] Yeah. I think it should just be... Because obviously it's not just Anthropic, it's OpenAI and Google, you know- yeah

Caleb Sima: having built their, their AIs that have gone out and attacked people.

Ashish Rajan: Uh, yeah, so- I mean, now we're talking Meta, Qwen, everyone else has a, had a, what do you call- Yeah ... what did they have? Like a sandbox escape everyone

Caleb Sima: had. Yeah, sand... So I think we should, like, make this just broader about AI attacking people, right?

Caleb Sima: And maybe even- I like that ... the sub, the, the subline of this is, how in the world are these companies not controlling their AI? Which by the way- Wait, I need

Ashish Rajan: you to...

Caleb Sima: There is an answer to that. Sorry. There is a single answer to that question.

Ashish Rajan: Oh, no, but I was also gonna say I think we probably would pin this, and we'll come back to this, 'cause I think I remember you and I were talking at BlackHat about the TechCrunch research report as well, where they investigated why after the Hugging Face scenario, somehow OpenAI was not sued.

Ashish Rajan: Considering all the young people who are hacking, I guess [00:04:00] maliciously or, um, or not maliciously, they all get put into jail. But somehow when OpenAI's agent does it- Yeah ... nothing's wrong. Okay. So to set the scene- At the moment, obviously for... A few things stood out for me. One was, and this was fascinating because until the last year's report or people, how they were reporting it, a lot of it was more about, hey, there was a deep fake as a scenario.

Ashish Rajan: There was a scam scenarios. There were a lot more, I won't say that technical skills that were called out as, hey, hacker or malicious people on the internet are using a highly sophisticated skill set of AI capabilities. And I don't say that lightly. What I mean by that is more that they're using deep fake, they were able to phish betters, be able to get credentials easy, but they were all out there.

Ashish Rajan: It's not that they were not out there, it's just that this year's report, the one thing that stood out for me is that it felt like the, I guess let's just say, the people on the internet who have a malicious [00:05:00] intent, all of them have become a lot more sophisticated. There is a whole section in the report about, I think there were 10 skills that they were noticing that were regularly used by the malicious hackers on the internet, ranging all the way from do initial recon, get human review, and go exploit, and then going into further build a C2 infrastructure, and if detected by a security product, rebuild the security architecture.

Ashish Rajan: Like things like that, which were fascinating, and I guess worthwhile calling out. You don't have to be a nation state anymore to have that capability. But I'll start there, but I'm sure, Caleb, you have some points you want to share as well.

Caleb Sima: Yeah, I have a bunch.

Ashish Rajan: I'm curious to hear what, what was the most important thing that stood out for you that you're like, "Oh, that," that would be number one for you. Let's start there.

Caleb Sima: If we recall, maybe I would say about two years ago, we were on this podcast, and we were talking about the ability for AI to do offensive [00:06:00] things, right?

Caleb Sima: AIxCCC, you know, like all these, What were the areas that we thought AI could go? And I made comments around, AI can do a lot of great things. It can run the scripts. It could do the recon. But I was very doubtful that AI could take things that were, I would call non-related and put them together.

Caleb Sima: Like for example, it could find an S3 bucket with a document in it, and in that document it could have what looked like usernames and passwords in it, but it wasn't called usernames and passwords, and then could relate that to a login page and try those entries into the login page to be able to see if it worked, right?

Caleb Sima: This is what hackers would do. Clearly.

Ashish Rajan: Yeah.

Caleb Sima: But would AI really be able to take these, y- you know, sort of non-related things and make them work, and/or go through multi-step processes in order to achieve its infiltration? And I was [00:07:00] super doubtful that this would be possible, and I clearly, I am wrong.

Caleb Sima: Like today, in this AI has not only, I think, done that, but is now I think almost far better than most people are. So even the experts in this field, it starts to feel like AI is a better reverser, better exploit generator, better, you know, web app attacker than I would say 98, 99% of the industry is today.

Ashish Rajan: Can I push back on that a bit?

Caleb Sima: Sure.

Ashish Rajan: I was gonna say I agree, but it, we, it is known to be non-deterministic though, right? Yes, you are right, broadly speaking, it is re- quite skilled. Yes, you can connect multiple things together. But would you say the fact that if I have, I don't know co- uh, if I have to cre- create a kill chain for an application, it's at least [00:08:00] today as it stands, the human is still providing the broader context.

Ashish Rajan: I may say the goal is, I don't know, container escape in this context, but I'm still driving the initiative. Hey, we have system one, system two, system three. It's doesn't have the ability, or at least right now, I again, you can correct me if I'm wrong. My thinking here was that, yes, it can connect dots, but human is still the one who is guiding it in that direction.

Ashish Rajan: Or is it, are you saying it's gone, "Hey, slash goal and go for it," if that makes sense?

Caleb Sima: Yeah. Yeah, I mean, I think that's... for example, if you had access to Mythos or OpenAI's, day, you know, red, what do they call it? Six, 5.6,

Ashish Rajan: yeah. 5.6-

Caleb Sima: Yeah. Daybreak ... or Daybreak. Daybreak. Daybreak. Yeah. Daybreak Red.

Caleb Sima: I think you could just slash goal hack the planet, and that's- Yeah ... all you need, right? Mm-hmm. If you have no restrictions and no safety boundaries, the ability for it to recon, pick its targets, identify its issues, infiltrate that target, [00:09:00] identify its data that it wants to gather, exfiltrate that data, and continue in worm-like attacker fashion continuously until stopped, is absolutely possible today.

Ashish Rajan: Yeah. So I guess, would you say a contribution to this is even though your prediction was two years ago, you were right for that time, it's just that the models being way this better was hard for us to predict. So I don't think you were wrong. Right. It's just the model acceleration was way quicker than all of us could have imagined.

Caleb Sima: Yeah, but it's, it's amazing to be able to see this, like, in full belief when you read sort of the Anthropic's paper and you see the level of intelligence in its ability to do it, and then you combine it with the reports that OpenAI has produced around its, you know, reverse engineering capabilities.

Caleb Sima: Or when you look at the patch capabilities that are being produced by all of these companies. Like look at Microsoft and Google and Chrome, the level of patches have skyro- rocketed since [00:10:00] its ability to use these models. It's clear, like a model can go pick Cisco, find a way to grab its operating system, reverse engineer remote 0days, go and tackle and use those remote 0days, and then inf- infiltrate.

Caleb Sima: Like it's absolutely possible without any, I think, um, hand-holding for it to go do that.

Ashish Rajan: Do you find that now that the open weight models are not that far from the frontier model, I think the gap at the moment seems to be between anywhere between three to six months, and at least most of the security researchers that I've been working with, they all seem to believe that even today's open weight models are good enough without the guardrails, without access to Daybreak or Mythos.

Ashish Rajan: They're good enough that if you are a decent pen tester or a decent person... Sorry decent person, I hope everyone's a different decent person. But more of a decent skill set for security, you could easily build something which could easily break a existing ecosystem of products [00:11:00] very easily.

Caleb Sima: Yeah, I mean, there's...

Caleb Sima: It, it's clear, right? We are at a state where the attacking capabilities of models are so sophisticated that it doesn't take a lot in order to go and do what you need. And you have all the things available, open weight models- You can run it. All you need is the hardware. The hardware and the inference is the, is the expensive part.

Caleb Sima: Oh, that's

Ashish Rajan: the

Caleb Sima: bottleneck? That's the bottleneck. Um, but-

Ashish Rajan: Oh, right, right. I thought people were running Mac Minis and stuff. So do we need like a, one of those Nvidia chips for it?

Caleb Sima: You don't need an Nvidia chip. You can use a Mac Mini. You have to use several Mac Minis, I think- ... if you wanna run really sophisticated models.

Caleb Sima: But yeah.

Ashish Rajan: Yeah.

Caleb Sima: You need some compute. You need some capability in order to do it properly. Although, if I were an, an attacker, I would just rent out that compute from some Neo lab somewhere, right? Yeah, yeah. Like, that's probably not too hard.

Ashish Rajan: Probably someone else's, some poor guy or person's laptop

Caleb Sima: somewhere.

Caleb Sima: Yeah, use, use someone else's. Yeah.

Ashish Rajan: Yeah, yeah,

yeah.

Caleb Sima: By the way, just getting back to Anthropic and [00:12:00] OpenAI, and I... One thing that, that was a clear interesting data point. So everyone hears in the news, obviously Google just announced their models hacked somebody, OpenAI did, and Anthropic did. Yeah. But no one has called out...

Caleb Sima: I mean, yes, people and the writers have called out. It all comes from one firm Did you see that? Oh, the,

Ashish Rajan: the research yeah.

Caleb Sima: The research- Yeah, the Irregular firm Yeah, yeah ... the Israeli firm was the one. Yeah. First of all, it's basically my theory about this was, okay, Irregular had access to all these O-day models from these guys.

Caleb Sima: I guess they were supposed to be testers.

Ashish Rajan: Yeah.

Caleb Sima: And then they just did nothing. They just let it go to the internet and said, "Hack the internet." And then, like, let them all just run muck across everything with no real controls or... Which was my first question. Remember when Mythos announced this? Yeah.

Caleb Sima: And my first question is, who is not monitoring this stuff, right? Like, you would clearly build a true container, and you would monitor the [00:13:00] crap out of this. Yeah. Like, how would you not see this occurring through this system? Oh, it turns out, oh, it's because it's some contractor out of Israel that is doing all of this for all three models, and that-- they're the core reason why it got out on the internet and just hacked everything.

Caleb Sima: Right. It's basically just saying, "I got access to Daybreak Red. Hack everything. Go."

Ashish Rajan: Ah. Wait, actually, you should probably set some context as well for people because people may not know Irregular, may not have read the story. So do you want to set some context for that Google News of the Google doing that, oops, our agents also hacked someone?

Caleb Sima: Yeah, okay, so the context is Irregular is a contracting firm that have access to all three AI companies and their models, and they were supposed to be doing security testing for these models, but apparently decided to do security testing in very lax defensive requirements and allowed all the models access to the internet, clearly- which, uh, created all of these stories.

Ashish Rajan: Yeah. [00:14:00] And so it was never the model itself. It was basically misconfiguration to an extent.

Caleb Sima: Yes, which also goes back to my comment in my earlier podcast was, does that mean anyone can just run a model, call it misconfiguration, have it hack things, and you not be held legally re- responsible for it

Ashish Rajan: I think that's a good point 'cause, you know, the research, and I think worthwhile to probably bring that up as well, 'cause I remember you mentioned, or, uh, actually, yeah, we, you and I were talking about this at BlackHat where TechCrunch did a research on why was it that OpenAI was just, it came out, why was no one from AI, OpenAI held responsible when- Yeah

Ashish Rajan: in general, if a human does the same activity, they are put in jail. And- Yes ... do you have some interesting insight there as well?

Caleb Sima: Yeah. What they said that it was very difficult because this situation has never occurred, where an AI is, quote unquote, "responsible." But to, to me, you know, so legally, in legalese- Yeah

Caleb Sima: apparently, it's very difficult to be able to do this because this is unprecedented territory, [00:15:00] and that is why it is something that's... And obviously, in OpenAI's defense, they are a model company. Clearly, their purpose and their intent is not to go hack everybody. However, you know, the, in sort of their defense, they don't have intent to go and do this.

Caleb Sima: However, it does go back to this accountability story around agents and who is then really responsible from a legal perspective. For now, what if I run agents and their goal is to hack my CTF, but somehow it gets out to the internet and my agent goes rogue and hacks things through the CTF? Am I now legally held responsible when my intent was not that?

Caleb Sima: And then do I get thrown in prison for it?

Ashish Rajan: But how do you measure intent? I

Caleb Sima: mean, humans- Or, or any- ... change their minds

Ashish Rajan: all the time ...

Caleb Sima: or any company for that matter, right? Let, now security teams and companies are running AI [00:16:00] agents. If these AI agents go out and decide to hack other companies, who then is held responsible for this?

Caleb Sima: There's a bunch of these things that I have lots of questions on that I feel- Mm ... like there's no answers to. But I don't wanna, I digress. I don't wanna go down that, the legal rabbit hole. No, I

Ashish Rajan: mean, but I think it kinda leads to another question that we have based on the reports and the activity as well.

Ashish Rajan: For people who are on the blue team side, I think I have some ideas on where they should start and what they should start putting down as, I won't say defenses is the right word for it, but to be able to even detect something has gone wrong in their environment, which is beyond just the SQL injection or cross-site scripting being picked up by SOC level one.

Ashish Rajan: And maybe if you wanna share some thoughts on this, and I'll, I'll share my side. But I think, yes, AI has become sophisticated, uh, but I'm also curious on the, what are you seeing or doing on the defensive side that people can actually match against or should at least think about or start [00:17:00] thinking about- Yeah

Ashish Rajan: in their security programs.

Caleb Sima: Yeah. This is, the problem today, again, goes back to this accountability. So AI is doing such amazing things from an attacker perspective, but of course the defensive side is lagging. And the defensive side is lagging not because we don't have similar capabilities.

Caleb Sima: Actually, AI on the defensive side can do some pretty amazing stuff. But the problem comes back to, I call it attackers move at machine speed, but I did this presentation actually it was at BlackHat, around but defenders move at meeting speed.

Ashish Rajan: Yeah, well said. I mean, that should be on a T-shirt.

Caleb Sima: Yeah. And, like, that's the chal- the challenge is we, you know, attackers have no qualms of, pointing things at things and just letting it go. Yeah. Versus on defenders, you cannot make a, the decision without impact. And so, for example, AI allows us to do two things super [00:18:00] easily. Can we identify vulnerabilities in, in our software and our codes really, really well?

Caleb Sima: Yes. We can do that in seconds, in minutes. Yeah.

Ashish Rajan: We

Caleb Sima: have the tool

Ashish Rajan: for

Caleb Sima: it. Can... Yeah. Can we patch those things also just as fast? Absolutely, we can. So then the problem is we can do both of those things really easily. It's the thing in between that causes the problem. It's when I find the vulnerability-

Ashish Rajan: Yeah

Caleb Sima: and I produce the patch, in order for me to deploy the patch, I have to have all these meetings to determine whether it will break something.

Ashish Rajan: Yeah.

Caleb Sima: Right? Yeah. And who will get mad is the meeting speed problem. And so that has not changed. In fact, I would state that I think it's getting worse. And so, like, I think this part continues to be a long, long tail, and that is what's causing defenders not to be able to protect themselves, right?

Caleb Sima: Yeah. It's not that we can't find the problem, it's the fact that we can, we can still not [00:19:00] fix that problem in any relatively decent amount of time.

Ashish Rajan: Um- I'm with you. I'm with you. Yeah. I think I did a talk at C- Cloud Security Alliance London, and the conversation was more about, in a similar vein, it was about being AI forward versus being AI ready.

Ashish Rajan: It came from a workshop that I did. I basically taught a team how to build a harness and connect all the cloud components and how do you investigate, do research and I guess basically be able to, going back to the machine speed part, how do you kind of enable your team to be, quote-unquote, "AI ready"?

Ashish Rajan: I think that's kind of where at least the initial thinking was. And one of the things that came out of all the work we did and the workshops we ran was exactly what you said, that we all know that we are slow at patching. Like at least the previous version of patching we had where we would have, I don't know 24 to 48 hours for a SEV 1, five days for a SEV 2 or a SEV 3, depending on what the company's feeling like.

Ashish Rajan: SEV 4, SEV 5, no one really cares. Like the... It's like the low and medium is [00:20:00] like, "Ah, I don't know if I'll get to it. I'll probably get to it next sprint," but that never really happens. That becomes tech debt and moves on. But the problem that still remains today is If going back to what the report is, if the capability is already present, and even if it is a small percentage of that nefarious ecosystem out there, it means that if we today, uh, the, the human is still trying to, or the, the defenders are still trying to struggle through, how do I convince the, the engineering manager, the project manager, the product owner, the CTO, who all have pressures to push forward with AI and have more AI releases with the balance of, "Oh, we need to patch this."

Ashish Rajan: Or I think the example, we were talking about this last time as well, someone wanted to give access to SharePoint to their AI system environment. And one of the things that the reason they were stopping was because they knew that data security was not done well within that ecosystem of content that they had in the organization, and the [00:21:00] organization wanted to go ahead and release AI into that ecosystem.

Ashish Rajan: And you're like it's already, for lack of a better word, it's a rotten bag of tomatoes, and now you're basically throwing the grinder in there just to mix all of it up even more to create a bigger mess. And I genuinely feel... I know we used to talk about human is the, uh, the weakest link. To this is probably the human is the slowest link in this context, I guess- Yeah

Ashish Rajan: to move with AI.

Caleb Sima: Which also, funny enough, is also the weakest link. Uh- That's the... But look at, uh, let's go look at another example. In the Anthropic report, they call this out, and now you would say like, you know, 90% or the majority of these incidents, their entryway for the AI was through credentials, right?

Caleb Sima: Yeah. Some sort of set of credentials that was then their initial entry point into all of these systems. Take, let's take that as an example. If you want to now protect your organization from this problem, this is a multi-year problem, depending [00:22:00] upon the size of your company and the complexity of your company.

Caleb Sima: Which is, let's move from fixed set credentials to short-term credentials in use. Or, and let's MFA, ensure MFA is across all of our employees. Wow. Okay, how long will it take for your company To do that

Ashish Rajan: I can already hear people sighing in the background, whoever listens to this, 'cause they all know the answer.

Ashish Rajan: This is not an easy answer.

Caleb Sima: This is not... Yeah, it will take you years, depending upon- Yeah ... the size of your company. Oh, what if you wanna do least privilege, right? Oh, well, 'cause the other thing that you notice in these AI attacks is they're able to get this privilege, escalate their privileges and move within the system.

Ashish Rajan: Yeah.

Caleb Sima: Right? Okay- Yeah ... well, I need to apply least privilege. That takes forever. I'll give you an example, by the way, of how AI for a defender works really, really well here. I went, I said, "Okay, in my company, I wanna use AI to [00:23:00] identify all of my least privilege problems." Mm. So I launched a job that went through my entire company, and our...

Caleb Sima: It ran for a couple days, and it produced a list of tier zero, one, two, and three accounts that were overprivileged, that had issues, gave exact recommendations as to how to reduce these privileges, and produced a phenomenal report, which by the way, was 99% accurate.

Ashish Rajan: Oh.

Caleb Sima: Right? And so we had, we had exactly the list.

Caleb Sima: And by the way, every company has... not every, but most companies, you can go find out, you can use AI right now, no interaction, and it could run through your organizations and by and large get most of your privileges, understand where they are, tell you how to reduce them, and even give you recommendations around the least way to fix them with what level of impact, right?

Caleb Sima: But now we're going through and following the guide to go actually implement it.

Ashish Rajan: Yeah.

Caleb Sima: It's [00:24:00] taking forever.

Ashish Rajan: That's the meeting part.

Caleb Sima: It's the meeting part.

Ashish Rajan: Yeah.

Caleb Sima: It's the, okay, well I know AI will tell me, "Change this group, remove it, put these people over here, eliminate these people." Well, you can't just eliminate them.

Caleb Sima: No. You have to go and ask. Yeah. Even though the technology tells you, the tech will tell you, "This account has not been used in eight months."

Ashish Rajan: Yeah.

Caleb Sima: It's never, it's been not used in eight months. Well, but we don't know. We can't make that decision without going to the engineering team and getting a req- Well, the engineering team can't just give you that answer.

Caleb Sima: They gotta research it themselves.

Ashish Rajan: Yeah.

Caleb Sima: So they have to spend time. Well, they don't have time to do it because they got other priorities right now.

Ashish Rajan: Yeah.

Caleb Sima: So they'll get to it in two weeks. Yeah. So it's gonna take you two weeks for them to even research it to tell you to remove a redundant inactive account so that you can safely go do that without getting in trouble.

Ashish Rajan: Even if it's an active account, I think the number of times you hear, "But I may need this in another few months, so I wanna [00:25:00] hold onto this."

Caleb Sima: Yeah.

Ashish Rajan: What

Caleb Sima: do you- And like this is the, this is... Yeah, you can't do that. And mo- And by the way, certain companies can set hard laws and hard rules, but the majority cannot.

Ashish Rajan: No. Yeah, you're right. Uh, maybe to, I guess, to frame this another way, is there a way maybe we use the AI drive in the organization as a leverage to push these agendas forward? Does that work to push those meetings? Because I, at the, at the end of the day, most organizations today have a governance committee, probably even two governance committee.

Ashish Rajan: One is for risk, one is for technical governance, or... But is there a way, or at least you've seen someone push, use AI as the agenda to push forward these things, that, "Hey, if you wanna use this especially if you're regulated-" You probably are still required to be, quote unquote, compliant, reliable, or that maybe if there was a compliance standard that fills this gap or a regulatory standard that fills this gap to, "Hey, you should have your police privilege covered before you kinda can do it or use AI for this."

Ashish Rajan: 'Cause I'm also thinking [00:26:00] more from a, is there an answer here that maybe we're just missing because we're thinking like humans here?

Caleb Sima: Yeah. So you're talking about forcing functions, right? Regulatory and compliance can be a good forcing function to, you know, sort of force companies to do this. But I think to your point, the adoption of AI being forced by companies is going to get us more in a place.

Caleb Sima: Like for example oh, AI bots are submitting PRs. The problem is there's humans who are going through and reviewing every PR in order to approve it- Yeah ... in order to ensure that it doesn't, you know, it takes and moves. And so the problem here is obviously the human in the loop is the problem, and you need to find ways to remove the human in the loop.

Caleb Sima: And so you either tackle it in certain ways. You, you have AI evaluate this thing, and also you need to come to a place in your organization where breaking things actually happens a lot, and you're able to recover and fix very quickly, right? So it needs to be this adoption of the ability to understand this painful [00:27:00] process of things breaking-

Ashish Rajan: Yeah

Caleb Sima: and you being able to recover and fix quickly using AI to help you do that, and then build in the guardrails and the pipeline, again, using AI to do the evaluation so that it doesn't happen again. And when you get to fully automated, then you're right, you can start doing things. But even then, things change.

Caleb Sima: Think, think about removing, doing least privilege. Oh, hey, our build box has too much privileges, right? Mm-hmm. Because, a year ago, we had to do some... We had to push something in prod that required some privilege, so we added it, and no one ever removed it, right? Like, all of these things need to be adapted and moved quickly so that you can take these things, chain, add a privilege in real time, remove that privilege in real time, and if when machines start doing it, then you get into a better place.

Caleb Sima: But we are so far from that. Yeah. And attackers, just like before AI world, are moving way faster than we can in order to defend. Yeah.

Ashish Rajan: I think another pattern [00:28:00] that I've been seeing in all the work that I do or the workshops that I've been running is also, it's very easy for us to drop what was, quote unquote, "old school," and by that what I mean anything which is pre-AI, which cannot be AI-fied, for lack of a better word.

Ashish Rajan: Let's drop that and start fresh, because always better to start fresh and drop the bag that was, uh, the rot- the rotten tomato bag, for lack of a better word. I definitely see that pattern quite a bit as well. But, I, I take your point, man. I think I'm, I'm with you on the fact that the humans are still the hardest part to beat in this particular context.

Ashish Rajan: I do wanna give something to people that, and maybe, I don't know if you agree with this, but even in the report and in general, what has been found is the fundamentals of what we've been doing so far with security, like, I mean, to your point, least privilege is a problem. We all understand that. But at least we know that Ashish is the employee of the company.

Ashish Rajan: And yes, the fundamentals may seem that, hey, we have done that, but would they stand this path of AI security or the [00:29:00] new ecosystem we are in? Maybe not for the newer problems we have, like prompt ingestion, others, but it definitely reduces the blast radius for how bad things can go. So security fundamentals haven't really gone away, and I think it was called out in the report too that what was picked up to what you called out, there was credential exposure, there was, uh, c- cloud compromise of cloud, supply chain, identity exposure, things that we have n- it's a known problem.

Ashish Rajan: It's not that we did not know what those things are. It's just that we let something that was potentially vulnerable exposed to being attacked, and that's when basically to what you said everything at the beginning of the conversation, then you basically you have a day break or something that just lets loose, and suddenly we are back in the problem of, okay, I don't have a way to detect this.

Ashish Rajan: I'm in this new ecosystem of attacks happening in microseconds versus like a... I think an example someone used was really interesting one, that we're almost like a high fr- high frequency trading where [00:30:00] transactions happen- Yeah ... in subseconds. We're in that kind of an ecosystem with cybersecurity attacks as well.

Ashish Rajan: Once the agent, let's just say the rogue agent is in your environment, even Hugging Face reported about 11,000 or s- I don't know, some-something in the thousands of attacks that they saw in a matter of minutes. And you realize we are not... So in a normal scenario, I don't think many people are equipped enough to even understand the patterns coming up across the entire organization, because we're already d- subsectioned into, oh, I have a SOC team, detection team, cloud sec team, app sec team who are all looking at individual silos, but no one's pulling all that back together with some kind of a custom harness based on what the organization is doing to understand, oh, okay, look at this.

Ashish Rajan: There's be- 11,000 attacks across the cloud, the supply chain, and everything else. There's something wrong here. I don't know if you're- Yeah ... seeing the same as well.

Caleb Sima: Yeah, I think the difference here is when you think about agents and the way at which swarms move- [00:31:00]

Ashish Rajan: Yeah ...

Caleb Sima: um, you just, things like detection are almost pointless unless you have a way of c- crowd control with response.

Caleb Sima: Right?

Ashish Rajan: Like- Yeah.

Caleb Sima: That's

Ashish Rajan: a good word. Yeah, yeah.

Caleb Sima: Yeah. There's just... We have been built on the independent attacker. Yeah. The single threat that kind of moves through your organization at human speed, and we are not built for the crowds that come in and attack. Yeah ... and so your security controls are just nonexistent because, you know, similar to, you know, if you think about, m- mobs during, like, riots-

Ashish Rajan: Yeah

Caleb Sima: right? In st- and, you know, grocery stores, you know? Like, they are similarly, they are built for single threat.

Ashish Rajan: Yeah.

Caleb Sima: People coming in, stealing something, you know, they can control those people. But when a riot happens and 50 people come into the s- the store and start, taking everything, you know, your control, your security cameras are pointless.

Caleb Sima: People know, peop- You can see [00:32:00] the people clearly stealing everything. Yeah. And you got one security guard who basically will not intervere- interfere unless he's gonna get beat sensele- senseless.

Ashish Rajan: Yeah, yeah.

Caleb Sima: Right? And so you just have to let it happen. Yeah. And the only thing you can do is after the fact, forensically trace down, through your detections, every single attacker who went through your store-

Ashish Rajan: Yeah

Caleb Sima: and then be able to have them held accountable, right? Because of what you were able to do. But

Ashish Rajan: that's

Caleb Sima: aftermath. But like, again-

Ashish Rajan: It's not

Caleb Sima: real time ... yeah, that's the aftermath. Yeah, yeah. So how do you now, in a defensive organization... You don't, which is why I have kept sort of really pounding on deception as becoming a new vector here.

Ashish Rajan: Yeah.

Caleb Sima: Because it can allow me crowd control, right? I can deceive the crowd into thinking that the, the thing that's important or what they're after may be located in a certain area. I can make them loop around or potentially stop based on what [00:33:00] information I can feed them.

Ashish Rajan: Yeah.

Caleb Sima: You need a sense of crowd control in a way where all these agents are now thousands of, or hundreds of them hoarding through your enterprise to both detect it and then deceive them.

Caleb Sima: And that, I think, is very possible today for defenders, but, uh, we don't have the tech yet to do it.

Ashish Rajan: But to your point, yes, we may be able to use deception to slow down, uh, the mob, however, that problem, that meeting problem that you were referring to earlier, we still need to work on that. It doesn't go away.

Ashish Rajan: We still need to have a faster way to do patching, a faster way to identify, say, what are some of the least privileged people we should be probably impacting s- the quickest. Someone has been failing the security awareness training for a long time, you probably should pay e- extra attention to that person instead of just, "Ah, Bob would correct himself in the next report."

Ashish Rajan: I think those kind of things are becoming more important, I feel.

Caleb Sima: I also think, you know, if you're a startup [00:34:00] person or a founder, the things I would focus on right now is about, my, my prediction is in the next three years, I would hope that we can see technology that is instant response Instant containment, instant remediation, right?

Caleb Sima: So, 'cause we can instantly already identify. The problem isn't identifying the issue, the problem is remediating, recovering, and resolving, these issues. And I think that is- Or even

Ashish Rajan: consolidation as well.

Caleb Sima: The what?

Ashish Rajan: Consolidation as well.

Caleb Sima: Yeah. I mean, consolidation is always helpful- Yeah ... uh, for sure.

Ashish Rajan: 'Cause I think too, what... Going back to the example we had of the sandbox breaking out, which linked to the irregular folks, but based on how the models are improving and a lot of people use the current benchmarks of, uh, exploit gym and et cetera. But there's also, and I'm not a AI expert here who has done a data scientist degree, but at least the way I understood it is [00:35:00] that over time, when you keep using the same benchmark on the same models, it starts to learn where it made the mistake last time, so it gets im- better and better, but it doesn't train or it doesn't give us a better feeling for if I were to use the latest model, I am able to pick up on every new vulnerability that's gonna be announced.

Ashish Rajan: And where I'm going with this is that we talk about zero days quite often, that, hey, I point Daybreak at a system or Glasswing at a system, and I may be able to get a zero day, which I would've never thought I would, it was possible until, I don't know, 25 years later or however.

Ashish Rajan: Yeah. I definitely feel there is a case to be made here between should people be proactive about running those models internally themselves and building that skill set in-house versus, you know, the build versus buy is where I'm going with this. 'Cause yes, remediation is a problem. Yes, threat are, threats are [00:36:00] getting, uh, a lot more complex with your attackers and hackers on the internet who are malicious intent are using infrastructure that they can push kill and recreate using AI.

Ashish Rajan: Having access to Daybreak or Mythos or Glasswing, whatever, that does not change any of this scenario. It also doesn't change the fact that just because you know your zero days, the meetings no longer need to be a thing. It can't... to what you were saying earlier, you have the technical patches, but you still need to convince Bob in accounting or finance or whatever to patch the damn thing.

Ashish Rajan: Like-

Caleb Sima: Yeah. Y- yeah. You, you know, there's, y- you know, where Mythos and I think these other products add their most value I think right now is not necessarily general operations.

Ashish Rajan: Yeah.

Caleb Sima: It's in product security, right? If you're Cisco or Palo Alto or Unifi,

Ashish Rajan: Yeah ...

Caleb Sima: like you need Mythos because y- [00:37:00] your product is foundational to the security of so many companies.

Caleb Sima: And so you need to focus, you need to point Mythos at a product, reverse engineer that product, and then produce the exploits so that you can clearly patch them, right? And, and so Mythos and others, although the advantage from the attacker perspective is strong, the advantage in the defenders is also strong if you are- Yeah

Caleb Sima: building product.

Ashish Rajan: Yeah.

Caleb Sima: Because there is a world here where you could say that the frontier models will provide the ability for you to build zero, quote unquote, "defect product," or at least to the model's intelligence level, right? Yeah. So I can, if I'm using Mythos today and I'm in Chrome, I can at least for the time being, be relatively strong in my ability to produce a secure product Right?

Caleb Sima: 'Cause I've had Mythos, Daybreak consistently hitting it. I [00:38:00] know that I am at a top-tier level of being able to solve and remediate security vulnerabilities, and that's a, th- that's a very big advantage at a scale that was not possible before.

Ashish Rajan: And I guess I don't have to be that skilled of a pen tester or a programmer to be able to build some of the harnesses around it as well.

Ashish Rajan: I just, as long as I know what I'm trying to create there, we should be able to use that. I think it, another related topic on this with people using it for product security or internal safety and security standards, I guess. In terms of the other side, which is d- we spoke about detection response, we spoke about the the statistic of the governance in this particular case as well.

Ashish Rajan: We also spoke about that the attacks themselves ha- have evolved, that the AIP ecosystem is being created. The, the swarm of agents, I wanna flip that around. Is there a va- is there value in having the swarms of AI agents [00:39:00] being let loose within the organization, even if it's not a product security organization?

Ashish Rajan: To what you said earlier, you can create better products, and obviously I understand it's a broad question because it could be depends on the value of the organization and everything. But in general, considering the norm now for attack, getting attacked, quote unquote, is, "Hey, there's gonna be a swarm of AI agents going after you," is it worthwhile for people to start building that kind of, maybe then let's use the word test harness internally, to start building the defense capabilities for it as well?

Ashish Rajan: Because that's where we're all headed, whether we like it or not.

Caleb Sima: Yeah, I mean, I think I talked about this with, when Clint was on our episode from OpenAI, and I was talking about the restriction of these models, right? And the fact that in order to have a good defense, you should have a good offense.

Ashish Rajan: Yeah.

Caleb Sima: Um, and we need access to these types of things. Which also goes back to actually, you know, something I would love to see happen, and this could be a controversial statement, but we need to be... [00:40:00] Since now models allow a level of sophistication and attacker that you can do this at scale, I wanna see the government, the US government and these frontier models partnering to attacking every single US company it can.

Caleb Sima: We need free red teaming across the board for every single company. When you ru- like, we should be able to partner with these guys, and no matter whether you're, you're a lawyer shop, a city or a town government, it doesn't matter. We need to let these agents loose so that we are red teaming and identifying these issues before these attackers are doing it, which they clearly have the scale and capability.

Caleb Sima: There should be an appointed capability for AI models and the government to create a function that allows absolute offensive red team against every company in the US. And that way we can fix these issues or at least have knowledge of these issues to [00:41:00] force or either fund or remediate these problems bu- before our attackers take advantage of it.

Caleb Sima: And that is not something that we've had Right? Mm. And our... Because obviously no one has been able to fund that level of talent to go after and do that.

Ashish Rajan: No.

Caleb Sima: But you don't need to do that anymore, right? You have the capability, and we should be using a good offense in order to have a good defense just across our nation, period.

Ashish Rajan: Even if it's against the critical infrastructure as well, right? 'Cause, that is what the... While most countries are standing up, again, to your point, it doesn't need to be just the US, but it could be globally speaking, considering now those programs are global, which, you know, I probably would extend that probably to the fact that, hey, ev- each government probably should think of a program where if they already have access to Daybreak or Glasswing and start using that capability to uplift an overall just l- whatever the base level is.

Ashish Rajan: And I'm sure it's, it would have its own challenges depending on the country. But, and I, but I'm going with this, I agree [00:42:00] with you on having a capability like this. I also feel that would also help organizations appoint someone who is responsible for being able to respond to these. So, and of course, it goes back to what we were saying earlier, that, hey, uh, le- letting those swarms of agents across our organization, whether it's through the government or through our own custom harnesses that we build, we irrespective need to build that capability.

Ashish Rajan: And we need- We need to build it ... be able to detect that.

Caleb Sima: Yeah. And you know the, and I know what the challenge is going to be, but I think this is a worthwhile effort for governments to obviously support, is, you know, similar, you know, obviously I can only speak on behalf of, what I think the US should do versus obviously I don't know, Europe-

Ashish Rajan: Other,

Caleb Sima: yeah

Caleb Sima: very well. But like, uh, th- there should be an investment and the cost of inference and management of this program is the big p- key part. But like we need to be able to use this not just against our enemies, quote unquote- Yeah ... but against [00:43:00] ourselves so that we can be better defended. And it's gonna be a cost problem, right?

Caleb Sima: Like, you have to build your own data center and your own inference in order to be able to do it, which clearly our governments are doing, right? They are doing that. They're just doing it for different reasons. But being able to be able to have a good offense against where we're at, I think is... L- you read this paper You write, Ashish, and you're like, all this took is one company, one contractor-

Caleb Sima: who had access to the models who just let it loose, quote unquote, by accident, right? Whether it's on purpose or not will be TBD.

Ashish Rajan: Yeah.

Caleb Sima: But like that w- that's one contractor who let these things loose. Yeah. We should-- And the level of access and damage it had capable, capability of doing is unprecedented.

Ashish Rajan: Yeah.

Caleb Sima: So like, it feels like it'd be a worthwhile for us to be able to make this a standard so that companies can be contacted by your government and say, "Here is a list of problems that our AI red [00:44:00] team agent has already identified for you, and you are-- you have to go fix this." Yeah. Yeah. Like, these are, in our defense of our nation, you know, you've gotta go fix these things.

Ashish Rajan: Yeah. And I guess to add to what you're saying as well, most governments already have a department or building a department working on AI problems, then-

Caleb Sima: Correct ...

Ashish Rajan: as every other technology, you think about the, the benefits it can provide, but you also need to think about how it can be, uh, attacked as well.

Ashish Rajan: We do that as government everywhere. It doesn't really matter where which country you're talking about. I definitely agree on the premise, and I, I also agree that driving a program like that considering every country wants to be sovereign AI, as they like to call themselves there, there are definitely gonna be challenges for what that could look like and depending on where they land.

Ashish Rajan: But in the meanwhile, if I bring it back to the enterprises that we know we talk about in other places they-- I'm on the mindset that they should still consider some form of open weight models to start [00:45:00] building such ecosystems, because the normal model that you may have access to, which is , non-Daybreak and non-Glasswing models, they have the cybersecurity guardrail stuff, so they...

Ashish Rajan: You can't do any of this. Even if you have the most skilled pen tester, security engineer in your team who understands the entire ecosystem, you would still need to consider as part of your program to build that capability, even if it's with open weight models internally.

Caleb Sima: Yeah.

Ashish Rajan: In the meanwhile. Yeah.

Ashish Rajan: Another thing I wanna call about the report is the fact that... So we spoke about the different departments, we spoke about the common theme that stood out. One thing we spoke about the detection speed as well. One thing we were talking about offline about this, and I think a common friend of ours, Sergej, had started the zero hour clock, which has been stopped, unfortunately.

Ashish Rajan: And for context for people, the whole reason behind that project was that it was tracking from the time of an exploit being discovered to the time of exploitation. And there seems to be, obviously, there's a whole debate on the internet about whether the [00:46:00] KEV versus CVE, what is the right way to measure the me- the vulnerability, et cetera.

Ashish Rajan: Keeping that aside, there is a whole conversation about, uh, the number of bugs that are being identified. Why is the zero day clock? Because the time is so, I won't say minuscule, and maybe, uh, I think the word you used was negative in terms of discovery to exploit. Can you say share some lights on that as well, man?

Ashish Rajan: Like, in terms of what the theory was there. Well, the theory- And maybe it's not specific to zero hour, but at least what your, what your understanding of, uh... 'Cause at least my understanding of this was that, hey, now people are harvesting zero days. They're not releasing them as quickly. And the idea is that the hours that we were counting down to or for a zero day to be released publicly and then being exploited, it's probably more beneficial for people to hold onto a zero day personally and exploit it for as long as they can before they release it publicly and get the street creds for it.

Caleb Sima: Yeah. I guess my, you know, I sort of made that comment sort of it's negative more as [00:47:00] a catchy term in how I think about it-

Ashish Rajan: Yeah ...

Caleb Sima: is that, okay, you know, standards is, oh, I find an exploit, I publish it, and within seconds it is now being exploited in the wild, right?

Ashish Rajan: Yeah.

Caleb Sima: That is sort of the countdown clock and what its sort of purpose was for.

Caleb Sima: Yeah. When I say, well, the countdown clock is negative, it's not necessarily in the sense... it is sort of, Ashish, to what you're saying, but also more in the fi- in the aspect of if there's a piece of software or an exploit that I want, I actually just don't need published exploits anymore. I can just point my harness at it, find my own 0days, and then just use them.

Caleb Sima: And so it's almost, you know, to me the reason why I call it negative is publication of exploits and then using them is just You know, why would people use them? It's 'cause it's easy. Someone did the work, someone found the exploit, and then I can take that exploit and then use it to my advantage. Now, if I'm interested in exploiting a piece of [00:48:00] software, I can just point a harness at it, it will find the 0day for me, create the exploitation, and then I can use it.

Caleb Sima: I don't need to get-- use someone else's work anymore. So I don't need to publish, I don't need to go take anyone else's work. I can just do it in the depths of my own... You know, I can be a 14-year-old and I can still do this. Mm-hmm. You know, uh, you don't need the talent or the skill to go do it. And so that's what I mean by negative.

Caleb Sima: Like, look at OpenAI at their conference for announcing Astra.

Ashish Rajan: Yeah.

Caleb Sima: Came out and talked about how they used Astra to find 400 0days. Privilege escalation 0days in Linux from normal user to super user. It, on average, it took them two hours to identify the exploit. And at the end of it, they had working exploit, working binaries for every single one of them.

Caleb Sima: And they said, what was it? 14 of them-

Ashish Rajan: Yeah ...

Caleb Sima: were so complex, it took several days of human people to go review and identify [00:49:00] exactly how the exploit even worked, uh, in order to determine what happened. And so, like, when you get to that level where you can find 400 of them in just the general Linux of which is open source and what you can find, you can point at anything.

Caleb Sima: "Oh, you know what? I want an 0day in how, in cellular communications. So I should be able to trans-transmit something to a cellular phone and use it to exploit some, broadband firmware exploit."

Ashish Rajan: Yeah.

Caleb Sima: Go figure it out, point a model at it, find your 0day, done. You know? Like, you don't need anybody else anymore.

Ashish Rajan: No. I, I'm with you, and I think that that's the, and I think that's what scares most people as well, that I... It's funny, I think, uh, as you were saying this I'm doing some research on identifying how do you know if an agent has gone rogue. Like, you know, there's a big debate, at least, uh, in the, all the research that I've been trying to find.

Ashish Rajan: Is the telemetry enough? Are you looking at the reasoning? What are you truly using [00:50:00] as a way to identify that your specific agent, whatever that purpose is of that agent, has suddenly turned rogue and maybe not the same as the irregular scenario where someone left the door open, but more like, "Hey, is this is now drifting away from what I initially planned for."

Ashish Rajan: And goes back to, to add to what you were saying, some of the scenarios you almost feel like AI should be, and talking to people who are researching similar things as well A lot of them are not, not technical. They're just curious about the problem. They point at a blog site. They have no idea about cross-site scripting, SQL injection, none of that.

Ashish Rajan: They just point it- Yeah ... and go, "Hey, I-- can you find a SQL injection in here?" And it kinda goes away and does the thing, and you're like, "Huh?" Like, all those years of spending, I don't know, web application hacker handbook and everything, and you're like, "What was the point of all that when I can just point-"

Caleb Sima: The point of it was to give knowledge to AI to, uh, know how to do it.

Ashish Rajan: That's literally-- such a high note. We should wrap there now. [00:51:00] Why? But I mean-

Caleb Sima: It is, it is true. It is

Ashish Rajan: true. It is, it is. Yeah, and I, I think I'm definitely finding the, that's probably the part that I'm finding the most daunting. What does defense look like in a world where everything we thought, which was our, for lack of a better word, secret sauce, that I knew that I have SQL injection hidden somewhere in my organization, but only I know because I've spent 25 years in the organization.

Ashish Rajan: Now you point AI at it, it just, I don't know, to your example, takes two hours and comes back with, "By the way, I found three more," which you may have missed that Joe down the desk he knew about, but he wasn't sharing as, until now. So yeah, it's a, it's a very brave new world we are go- moving towards, man.

Caleb Sima: Yeah. And y- that's the reason why I went from just being a VC to going back and being an operator-

Ashish Rajan: Yeah ...

Caleb Sima: is because I needed to know in this world, how can I use this technology to somehow break the meeting speed problem, right?

Ashish Rajan: Yeah.

Caleb Sima: Because I think it's so important to be able- Yeah ... to [00:52:00] figure this out.

Ashish Rajan: Yeah. I, I think I'm definitely finding that even... Yeah, I, I, I'm with you. It's almost like you have to be in the trenches to be able to identify these kind of use cases. And I'm not saying that the product companies would find it harder, they obviously have a lot of conversation, but in all the conversations you can have would not bridge that gap that easily until you start using the AI, and you're like, "Oh, wow, okay, it can go...

Ashish Rajan: I can just point at a SQL injection article, it understands at a deep level, and I don't have to be a technical expert anymore. I just wanna know what I'm trying to get towards or get out of this, and just guide it." And yeah, it's, it's quite scary, but in the, all the workshops that we've been running for AI readiness, where we're talking about all this, it's very...

Ashish Rajan: Yeah, we definitely are changing the way we have traditionally done security, and maybe going back to what you said about the meeting part, I also would add another layer there saying that the way we have done security in terms of the team structure, I don't [00:53:00] think that can work in this world where I have an appsec, cloudsec.

Ashish Rajan: Like, maybe for the open on-premise per ecosystem, which is still microservices, your non-AI components. But for all the AI components moving forward, I would go ahead and say maybe that model that we used to work with, where I had a SOC team, I had a appsec team, cloudsec team, GRC team, there needs to be almost like a, you need to pull all those thread into some kind of a, I don't wanna call it a SWAT team, but like a cross-functional team inside security.

Ashish Rajan: I, I

Caleb Sima: actually call it a SWAT team.

Ashish Rajan: Oh, you actually... Yeah. Yeah. Oh, there you go. It sounds like a cool name. I mean, maybe inside, but outside you're like, "Oh, what are you..." But, so do you think that, am I right in saying that, where like the traditional-

Caleb Sima: You, you are. I have the way that I'm building my security team is very different.

Caleb Sima: Although slightly... So there's a couple things that I'm changing. Number one, you don't need as many people, quote unquote. That is the theory. Yeah. That is yet to be tested, by the way-

Ashish Rajan: Mm-hmm ...

Caleb Sima: [00:54:00] uh, for me. But I'm hiring what's called now, I think it, there, it's, there's a term for it called barbell hiring.

Caleb Sima: Right. Um, which is, what I'm hiring is I'm hiring very, very top senior ICs- Mm-hmm ... who are experts in their area. For example, I do need a detection and response principal level engineer IC, right? Who is very AI forward. And I am hiring a little bit in those areas, software security, IoT, OT, experts who know that, but at a principal level.

Ashish Rajan: Yeah.

Caleb Sima: I'm go- so I'm gonna, let's say that you have a thin layer. Use, let's say you have, like, six or seven of those guys, right? And then I'm only hiring then junior intern level engineers. Call them no specific area of expertise. Sure. But they are engineering focused.

Ashish Rajan: Yeah.

Caleb Sima: And they are heavy in, early in their careers, very early in their careers.

Ashish Rajan: Yeah.

Caleb Sima: And nothing in the middle. [00:55:00] So there's no middle management. There's nothing in between. Okay. And they all, the junior interns and the principal ICs all direct report to me.

Caleb Sima: Right? And we are using... And I have a SWAT team that is very specifically called SWAT team because their entire job is to just go after all the fires that are going on and fix and remediate all of the...

Caleb Sima: It doesn't matter what it is, whether it's in detection, offense, red team- Yeah ... you know, defense- Yep ... cloud. Agentic. They're just, they, that's all they do. They're like firemen, SWAT teams. You just shoot, shoot them after whatever's going on.

Ashish Rajan: Yeah.

Caleb Sima: And that team is a mix and can be dynamically created by any of these people.

Ashish Rajan: Yeah.

Caleb Sima: Right?

Ashish Rajan: Yeah, yeah. I, that- Yeah ... totally makes sense.

Caleb Sima: And

Ashish Rajan: that's- 'Cause I think

Caleb Sima: knowledge is power ... and you can You cap it. Like normally where I'm at, you would have created a team of like 30 people- Mm-hmm ... 40 people already. I'm capping my team at 12 to 15, right? Yeah.

Ashish Rajan: Just the right [00:56:00] skillset to be- So...

Ashish Rajan: Yeah ... and with, uh, being AI forward, that kinda helps you catapult, like scale your team's capability without scaling the actual people in the team.

Caleb Sima: Yeah, like, you know, I told you earlier in this podcast, I myself personally generated a report with AI that did least privilege analysis and architecture- Yeah

Caleb Sima: across our entire org, and it was correct.

Ashish Rajan: Yep.

Caleb Sima: 98% correct- Yeah ... in its analysis.

Ashish Rajan: Mm-hmm.

Caleb Sima: Um, right, so you, a CISO, one person with, I would call it five prompts, can generate that level of intelligence now if you- Yeah ... AI correctly.

Ashish Rajan: Yeah.

Caleb Sima: And that is unheard of- Yeah ... three years ago. I,

Ashish Rajan: I think I hear that, and the first thought that come to mind is a lot of times when you go into those frustrating meetings, to what you were saying earlier, where you're trying to convince people, a lot of times you're missing the right kind of data points to raise up the problem in the right way.

Ashish Rajan: And I think when I, when I hear you say that, in my mind I'm like, "Oh, wait. So now [00:57:00] if I am someone who probably was trying to push this for- agenda forward of how do we do AI security better, could get a lot more data points to justify their point with the right information in hand." And obviously soft skills, everything still just assum- Yeah

Ashish Rajan: assume it's already there. But outside of that, a lot of times w- it would take us, I don't know, how many weeks or months to get the right report. It's like, oh, I'm waiting for the pen testers to come back, or I'm waiting for this to come back. Like there used to be- Yeah. Yeah ... a lot of slowing down of information as well.

Ashish Rajan: That I- Yeah ... feel is just like that should not be the case anymore.

Caleb Sima: Not at all. Actually, you know, when you throw a report down that says, "Well, hey, guess what? We have 462 global admins in our company."

Ashish Rajan: Yeah.

Caleb Sima: Do you think that's a problem? Yeah. Yeah.

Ashish Rajan: That's right. It's like, it's... That's what I feel.

Ashish Rajan: Like, I think there, there's definitely for lack of a better word, yin to my yang, I guess. But I think it's like the, as much as the offensive people have an opportunity, we do [00:58:00] have as well as an opportunity. We just need to look at the problems that we've been beating our head with a different lens.

Ashish Rajan: And maybe I, I think that's a good place to wrap this episode as well. Uh, but I'll put the link to the threat report in the show notes as well so people can review it, have their thoughts on it. Uh, definitely leave your comments for what you think is the right way to approach this. Would you change your team for the new world of AI that we're moving towards, or have a SWAT team, as Caleb was saying earlier?

Ashish Rajan: But I hope you enjoyed the episode. We'll talk to you soon. Thank you for watching or listening to that episode of AI Security podcast. This was brought to you by techriot.io. If you wanna hear or watch more episodes of AI Security, check that out on aisecuritypodcast.com. And in case you are interested in learning more about cloud security, you should check out our sister podcast called Cloud Security Podcast, which is available on cloudsecuritypodcast.tv.

Ashish Rajan: Thank you for tuning in, and I'll see you in the next episode. Peace.

‍