Caleb Sima: [00:00:00] Security will not go away

Ashish Rajan: no. Uh- it just cannot

Caleb Sima: Yeah. The question though that I might propose is, will people go away? I have never looked at a single line of code in that application. I haven't even opened the GitHub repo, and these things are off running and self-managing themselves.

Ashish Rajan: AI kind of gives you the illusion that it's very easy to create a prototype.

Caleb Sima: A year later they're like, "Oh, ." Yes, it does do that, but it's super, super ad hoc and not reliable and not consistent. We are in need of a forest fire in our landscape of startups. We have so much junk and no one can figure out what to buy versus not.

Ashish Rajan: Software is no longer the moat, then what is the moat for an AI security company?

Caleb Sima: Moat is the biggest question for almost any company right now and it is the hardest to answer.

Ashish Rajan: Coding is almost disappearing with AI, and Topic CEO came and recently spoke about the fact that coding is [00:01:00] already disappearing and soon software engineering would as well. But no one's really talking about what does it mean for security, what does it mean for AppSec people, people who are building companies around cl- code security.

Ashish Rajan: So in this episode, Caleb and I dissect what the role of security would be in AI, and what does it really mean when the new language of programming is going to be markdowns? What is the role that we play in this as a security architect, as a software engineer, as a security engineer, as someone who's just basically been in this field for decades and trying to figure out what our role is.

Ashish Rajan: And also, if you are considering being a design partner for an AI security startup or you are an AI security startup as well. And so for everyone who's been on that build versus buy journey, we definitely have some hot takes in the episode as well. If you know someone who's looking into the security of AI-generated code or AI coding assistant, definitely share this episode with them.

Ashish Rajan: And as always, if you are here for a second or third time and have been enjoying episodes of AI Security Podcast, I really appreciate if you take a quick second to drop us a follow or subscribe on whichever podcast platform you listen to or [00:02:00] watch us on. We are on Spotify, Apple, YouTube, LinkedIn, and anywhere else you listen to podcasts on.

Ashish Rajan: I hope you enjoy this episode, and I'll talk to you soon. Hello, welcome to another episode of AI Security Podcast. Today, I think we are talking about something which has been, I would say, trending on the internet, for lack of a better word. The Anthropic CEO came out and said that, hey, coding is going away, and soon software engineering itself would.

Ashish Rajan: But there was no opinion on security, and I think so this week, Caleb and I are talking about the so the changes that the volume of coding being replaced by AI is doing for security, with security, and what does that even mean? But to give you context for people who are listening or watching this, there was a survey done by Anthropic where they interviewed 81,000 Claude users as part of the, uh, survey.

Ashish Rajan: And what they found was a lot of people who already were u- heavy users of AI were concerned that their jobs would be replaced by AI. Specifically, developers were even more concerned that, hey, any junior developers coming in probably [00:03:00] would be the worst impacted, uh, because they would have zero or no exposure to AI at all.

Ashish Rajan: And that followed by the CEO's interview on popular news channels talking about how, according to him, in six months' time, coding would not exist and potentially soon after, software engineering. Now, I obviously had some opinion on this, but I've, I've maybe I wanna start with Caleb, your opinion on this, what I can j- what I just shared about coding as a field disappearing and software engineering.

Ashish Rajan: But I would also like to bring the security lens onto this as well. What does that mean when all the code is being replaced by AI, and do, would s- security really be required? 'Cause we recently had the whole Claude Mythos thing kind of, uh, drive the stock market down for a lot of cybersecurity people, which went back up again.

Ashish Rajan: Now, I think this week it's all the SaaS providers, but hey. I'm curious to, uh, keen to know your opinion first, and I can share mine as well.

Caleb Sima: But this, you know, to me, this is just rehash. It's almost, yeah, we've talked about this. Everyone since AI has talked about this, coding is [00:04:00] going away. So when was the last ti- I don't...

Caleb Sima: Do we have a date of when? 'Cause he mentioned this before, which was, like, I don't know, was it six months ago or a year ago where, where the Anthropic CEO also made this claim that software, you

Ashish Rajan: know- No, I think he just came in January at the Davos, the-

Caleb Sima: At Davos, yes.

Ashish Rajan: Yeah, the, the- So- ... thing that happens in Switzerland, I think.

Ashish Rajan: So it would've been January. Yeah. So we've already been, like, I mean, we're already month three, and but then he made another claim again last month that six months is... I think he's, he, he has a spiel for six months. He just keeps saying six months every single time, so- Yeah ... uh, I think that's what he's aiming for at the moment.

Caleb Sima: I think his difference, though, is i- in this one, if I were reading into it, in January, he said coding would go away, and it, and now he's saying software engineering will go away, right?

Ashish Rajan: Yep.

Caleb Sima: So let's look at what he said. So coding. Did coding go away in six months? Well, we're three months in.

Caleb Sima: I would say anyone Who is at the beginning [00:05:00] projects or prototyping or building features or doing anything in code bases, are they using AI to code? I feel like that's almost a pretty resounding yes.

Ashish Rajan: Like- Yeah. Yeah Yeah ... about to,

Caleb Sima: about to do that

Ashish Rajan: already, so yeah.

Caleb Sima: Yeah. Very, very few people almost look at their code or review their code which is both bad and good.

Caleb Sima: I would say if you're working in enterprise products, an existing product that runs and works, uh, you're a bit more careful on what you're using AI for, but you're still using AI pretty decently, I would say. If the models continue in the next six months, at their capability, I do feel many people will build products and software where they don't actually write the code.

Caleb Sima: And whether they understand the code or not is a whole different thing. But to me, as someone who is eyeballs deep in this every day I could see it. Look, there's still a lot of problems in AI-generated code. Yeah. [00:06:00] But, like, I could see it. So then the question comes into software engineering. What is software engineering and how do you separate that from code?

Caleb Sima: Software engineering to me is the rest of the things surrounding code, right? How do you deploy code? How do you manage code? How do you operate the a- software application? Infrastructure, all, desktop, you know, OS, server maintenance. All of these things to me is what I would say is software engineering if I were to define that.

Caleb Sima: And then he's saying in six months AI will automate that. That I have a- Wait, no. He didn't say software engineering would be automated in

Ashish Rajan: six months. He said, like, soon after, to kind of quote him, soon after- Okay ... software engineering would be replaced. So I- Yes ... don't wanna misquote him. Yeah. But yeah.

Caleb Sima: Okay,

Ashish Rajan: good. Six months for, six months for coding, and then soon after software engineering would not be required as well.

Caleb Sima: You know, I'm, you know, it's, it's funny, I, I do feel like a majority of generic startup applications and [00:07:00] software in six months could be you don't really mess with the code.

Caleb Sima: And actually I think that generates a whole new problem which drives into security, which is people are building software applications and managing these software applications at scale, Ashish, you and I were just talking about the ones I built Yeah. That I have never looked at a single line of code in that application

Caleb Sima: I haven't even opened the GitHub repo to even look at the code that I've built, and these things are off running and self-managing themselves. Yeah. Right? So which is scary. So I think that the new opportunity that is coming into play is gonna be about, well, if code itself becomes not black box-ish, but effectively a black box, what is it that becomes needed?

Caleb Sima: And I think validation, management, and security are key parts [00:08:00] of this next sort of phase and focus, which is, hey, you are going-- From a security perspective, I do believe the right direction and where we are going is that when AI writes the code, it will write secure code. You will define security mechanisms.

Caleb Sima: It will-- It is best at auditing itself, it is best at patching itself, and it is best at building the right infrastructure for it. But you need an auditor. You need an ability to verify that what you believe has been done is correctly done.

Ashish Rajan: Yeah. I-I would say I'll probably go a step further, right? I think I love what you said about the coding part and I 100% agree.

Ashish Rajan: What I would add for the way I-- the... When I first read that, I was like, oh, for me, software engineering is what makes it usable at an enterprise scale. You can write code, Python, Java, whatever, you may be an expert, but what makes it usable is the architecture that someone has spent the time to solution the thing, review the [00:09:00] security architecture for decisions being made as to how it would be integrated across the entire organization.

Ashish Rajan: Again, there's a lot more moving parts than simply I have replaced code, therefore I have replaced software engineering. And I think he probably oversimplified it the way I saw it, because in my mind it was. That's what the context of the interview was, and that's what being pulled out on the inter- uh, on the internet.

Ashish Rajan: I'm sure if we sit down and have a conversation, I'm sure he'll have a much more context. But the the important part for me was kind of what you said last about- Coding has a place that you do not care about what code is being used, may it be a Python or Java expert, but you-- what, but what you do care about is the fact that, hey, can this be operational?

Ashish Rajan: Let's say if I'm a bank, and I know for some reason we use Bank of America as an example in this-- in the, in the podcast very often. But let's just say if I was Bank of America, I already have all the old code that I have written for decades. That is not going away. Maybe I can enhance it or make it better, but that's gonna be there.

Ashish Rajan: My COBOL running on [00:10:00] mainframe is still gonna be there even after-- long after AI has done its job with it. So I would keep-

Caleb Sima: I don't know, maybe AI rewrites that in Rust, so

Ashish Rajan: Oh. It would-- I would love it, 'cause-- but I don't know if there's enough documentation of COBOL that is digital even. I don't know.

Ashish Rajan: But maybe there is. But the point being, it's like the, the-- it's not going away at a scale that he may be saying that the field is gonna go away. So I definitely feel like there-- to put more of a security filter, you kind of need to talk about both sides. First is, hey, there's already a lot of legacy code that all of us have lived, breathed, uh, sometimes patched, sometimes not patched, sometimes it's accepted.

Ashish Rajan: So that part is not going away. That's not being written in-- at least not being written with AI that quickly any moment, because it's gonna be costly exercise, and over time it may. But the second half, all the new code, the new features, the new software is being built, that to what you said as well, 100% agree that it's being driven in that direction for [00:11:00] all new features are being built using an AI coding assistant.

Ashish Rajan: All new softwares, people are thinking about how do I use AI coding assistant. The thing that I wanna add there is that, and- Is that the more you write code, the more volume increases, and let's just say the humans have used-- started using AI for code review, what it does not change is the fact that now you have a lot more exposed area for risk, and you also have this opportunity for let's just say, 'cause all the other components that we use, say we have the cloud, say, let's say Bedrock or Vertex or whatever, those are not coding components.

Ashish Rajan: They are har- uh, not hardwares, they are infrastructure components. Maybe you start using AI there as well, but at least in the beginning, what code-- Claude Code does today, it does a great job of solving a software problem where I can create a software. It does not solve my infrastructure problem required.

Ashish Rajan: That is assuming I still have to go through my SDLC, and obviously there's a whole argument around do we even need an SDLC? And we can go into that conversation as well, but at least as it stands [00:12:00] today, there will be stages of gates that people have to go through because the reason why security maintenance to what you said exists is that I can write my own code and certify, yes, this is the most secure code possible, but this is where the verification comes in.

Ashish Rajan: This is where maybe there's a security harness in the future, but it's not,

Caleb Sima: Security will not go away, right?

Caleb Sima: Security will not go away.

Ashish Rajan: No. Uh. It just cannot. Yeah. It doesn't make sense. It does-

Caleb Sima: the, the question though that I might propose is, will people go away?

Caleb Sima: Right.

Ashish Rajan: Ooh.

Caleb Sima: The need for the security will always be there. The question really focuses on will the security team be there? And that I think can be debatable, right? When you think about all of the aspects that you need to focus on in an SDLC, let's start with just coding as an example. Yeah. Okay.

Caleb Sima: W- it should be pretty paramount if you get the right process in place. [00:13:00] Every piece of code or module, or let's just say sprint, at which AI completes, there will be a self-running audit, both of security vulnerabilities, both of security structure and architecture, both from white box source code side to black box exploitation side.

Caleb Sima: This can all be fully automated by AI. And by the way, use different models to ensure that you have, quote-unquote, different viewpoints, or even fix your models, SLM or standards or where there is, so you get a consistency in how you're auditing or looking at security. So you can produce a piece of code that- Yeah

Caleb Sima: comes out, quote-unquote, mostly secure all completely without actual humans involved in it. And so, like all the way from white box to black box, I think this is very doable. And so then you can then produce these gates, and these gates are then held by AI, not [00:14:00] necessarily held by manual review. So can you now take a 30-person AppSec team and reduce it to four people?

Caleb Sima: Probably, right?

Ashish Rajan: Probably if you just build it. I mean, yeah, but would you say that, would that just remove software engineering though?

Caleb Sima: No, but it doesn't remove so- software engineering is a thing. The question I think is, does it remove the people? You know, like y- softwa- like security doesn't g- same, similar to security.

Caleb Sima: Security doesn't go away, it's just now way more automated, right? Yeah. And so, yeah, it's not like the need goes away, it's does the intellect or human in the loop go away? And I think that is debatable. That's a topic that's, you know- Yeah. If I were, if I were this, the Anthropic CEO, th- that's what he's saying is, software engineering and coding are gonna just g- the people are gonna go away from that process.

Ashish Rajan: Yeah, and I, I think obviously I understand [00:15:00] there's other objectives he may have for driving the narration as well. But the point being, I, and I 100% agree with you on this, where AppSec people can maybe amplify their productivity with this AI harness as a lot of people are calling it, or building your own AI scaffolding.

Ashish Rajan: But I also want-- We did an interview with Heather recently, uh, the CISO of box.com. She was talking about how she's made the four bets, the four big bets she's made for AI in terms of building the SOC. Start- starting to do some parts of AppSec as well. The goal is not to replace people, but the goal is to improve the overall output of the, uh, security.

Ashish Rajan: Maybe not adding more people, but at least being able to produce a lot more efficiently from the existing security team she has. Like, she had like four bets that she had put in. One of them was AI SOC, one of them was more on core security, uh, vulnerability management, and all of that as well. I feel like it's worthwhile calling out, because you're obviously on the start of ecosystem [00:16:00] too.

Ashish Rajan: When core tends to be more marked down versus SQL injection, it, uh, at one point it does integrate and it's, let's just say the AI is doing your, Your security testing part. Are we saying that the time to live for a lot of these AI security startup is very short? ' Cause technically a lot of us are, aren't we all trying to either if you zoom out, we're either trying to solve a CVE, we find a CVE, we find, "Hey, by the way you, you have a notification service from an AI security company to say there's a problem, go fix it,' or, "There's a security bug, go fix it."

Ashish Rajan: Or the other thing used to be that, "Hey, uh, the code is using a library that is old, go fix it." If you're saying those two, which are like kinda like the bigger piece of rocks that we had to move as security people, that can be for a lot of reasons taken away by AI, does that mean the AI security ecosystem who are focusing on those components have a very short time to live in terms of-

Caleb Sima: Um, y- yeah, so it just has the [00:17:00] difference, there's a difference between A- AI security.

Caleb Sima: When you say AI security, you're talking about secure, like securing AI, right? Or are you talking about- Yeah. So

Ashish Rajan: in this, in this like AI generated code is the, is the context I'm going with. Because, because to what you, what we are saying is that, hey, yes, security's not going away, but they would become more efficient with four people AppSec team instead of a 10 people AppSec team.

Ashish Rajan: Correct.

Caleb Sima: Yes. Yeah, and now we're talking about vendors. Will those vendors go away,

Ashish Rajan: right? Yeah. So the vendors who are operating in that space who may... 'Cause not every AppSec person may build a scaffolding for AI in their organization.

Caleb Sima: No, no, no.

Ashish Rajan: It may not be possible at scale as well, but they start doing, say, "Hey, I'm gonna use whatever automation tool and AI tool."

Caleb Sima: Yeah.

Ashish Rajan: And I may go down the path of saying, "Okay, I'm gonna use this X vendor for today to start off with." But I guess where, where my question is coming from, whatever vendor they end up choosing would that have a very short time to live if that's the case, and if that's what we're heading? 'Cause sounds like that's where we're heading with if cool- Yeah,

Caleb Sima: you know, the direct answer to your question is [00:18:00] absolutely vendors will thin out, right?

Caleb Sima: Like there will be... And, and I think this industry needs this. There... Unfortunately, I think we are in need of a forest fire in our landscape of startups. We have so much junk and no one can figure out what to buy versus not. That being said, however does security tooling and products go away?

Caleb Sima: Absolutely not. Like, let's take co- code for example. You can't just take AI, or at least today, and I don't know in three years, no one does, but I don't believe this to be true, and be able to get consistency and scalability and reliability out of it. What AI is good at is its capability of reasoning, being able to make intellectual decisions based off of things it sees, and be able to take those actions.

Caleb Sima: And as part of that, like I would never take AI blindly and just say, "Look at [00:19:00] my code, review it, and tell me if it's good," and that's what I rely on. I would never do that. I would say, "Hey, AI agent, you have access to Endor, Socket, Semgrep." Yeah. "You now go and do an analysis of my code alongside of using these tools to get your reliability, my, you know, my consistency to determine where I am."

Caleb Sima: And I don't think that ever goes away. I-- There is places, just like there are humans today, you need tools. You need things that are consistent and that can scale. And so like, I don't think that goes away. What you will have, and what I hope will happen in the market, is that there'll be a consolidation of a lot of this junk that we get to really focus on the tools that do the job really well.

Caleb Sima: And I do think there will be changes in how those tools are d- those tools and products are delivered with AI, but for sure my code team will be [00:20:00] buying and using security vendor products, right? As long- Right ... and AI will be using that.

Ashish Rajan: But to your point but I guess we are not there today, but the world we are moving towards, if it's filled with markdown specifically, should-- Like, and at least my-- in my opinion, the AppSec teams or CloudSec teams, doesn't really matter who- whichever security team you're at, you probably still need to start building the AI scaffolding or some kind of an AI capability within your, within your team.

Ashish Rajan: Because say, to your point, take any code security vendor can take the ones you mo- you, the ones you recommended, which is the SCA category, SaaS category, doesn't really matter which category you take. At a point, it would become that this, the volume of things... you would not wanna be the team that everyone else in the organization is using AI for scaling themselves with the limited team people they have or limited number of people they have on their team.

Ashish Rajan: You probably wanna build some kind of an AI scaffolding or capability in your team by yourself as a team, so you can plug into [00:21:00] these vendors when you need to for information. 'Cause you're not gonna build a threat intel feed for the the CVEs that are coming out. But you... What you may wanna build, which we already do as most organization, is to have a layer of, is this truly a hive for me?

Ashish Rajan: Is this... a lot of these vendors have ca- capabilities for reachability. And- Yeah ... hey, am I, is this reachable or not? Or there are certain parts of the code that, hey, I need to host this privately. I can't have this somewhere else. I definitely find that there is still a need for AppSec team to consider building an AI scaffolding inside, irrespective of the fact that yes, you're right, tools would not go away, but there would be things that would happen at volume that tools would never solve.

Ashish Rajan: Like it would not solve for my problem of the way my engineers may code in, say, Bank of America would be different in how my I don't know, an insurance company, let's just say American insurance, I don't know, insurance company, let's just say American insurance company, the way they code there [00:22:00] may be a bit different 'cause it's, there is no standard anymore.

Ashish Rajan: They were built on the fact, like all of these were built on the fact that there was a known standard for coding, but now we are saying there's no known focus standard for coding because the AI itself is gonna be the tester. Yeah. AI is gonna be the result. They may plug into these SCA tools, SaaS tools.

Ashish Rajan: Now what, what is the quote unquote standard that each one of us would have to define within our organization? Would you agree?

Caleb Sima: Yeah. I mean, you're, uh, if going back just to your original point, like prompt injection is a huge problem since data or execution is now marked down, right? However, I think that is a manageable problem, right?

Caleb Sima: Y- as an example, is it an easy problem to solve? No. It's just like SQL injection. SQL injection was not an easy problem to solve, but it was a manageable problem to solve. Today I think it's manageable. It's hard, but it's manageable to be able to say, to your point, build the AI framework or your harness or your orchestrator or whatever it happens to [00:23:00] be, to ensure that as, now data is communication, uh, the ability to identify or detect, prompt injection, which is, you know, the next generation SQL injection and cross-site scripting, you know, is it doable?

Caleb Sima: Yes. You can do it. Will there be vendors that will focus on preventing that? Yes. Will you need them? Debatable, but yes.

Ashish Rajan: Right. But I guess maybe to even put it more into the context of Claude Code as well, like I think, uh, an example that I came across recently how people download skills from the internet and just basically- Yes

Ashish Rajan: go, "Hey, I have a skills file." And to your point prompt injection could just be white text that I don't see, and I was basically, I accept someone's skill and I have g- like there's, there's so much more nuance to at least why I, I keep harping about people should build their own skills and people should not rely on the external skills provider.

Ashish Rajan: I could maybe take inspirations from it, but don't just literally copy paste the entire skill set in [00:24:00] there. So- If I was a CISO today, and I know we h- we recently had the whole Claude Mythos episode as well, and there was a whole conversation about how that's gonna change security and how security's just gonna be basically no, I wouldn't use the word eradicated, but basically shaken up completely.

Ashish Rajan: To your point, the forest fire is gonna be the Claude Mythos moment.

Caleb Sima: Yeah ...

Ashish Rajan: we also have the, uh, the extensions of the part where recently Vercel the hosting provider got hacked. But that was not a AI hack, not an AI hack to be precise. It was basically a third-party supply chain thing. There are things like that as well that software engineering technique doesn't really account for.

Ashish Rajan: So, and bringing it back to the point of what the Anthropic CISO, CEO said, that, "Hey software engineering would go away." I mean, we already spoke about the AppSec the coding side. There's a whole octopus tentacles into third party for performance monitoring, for AI productivity and all of that as well.

Ashish Rajan: This is [00:25:00] kind of where your non-human identity piece that people talk about, where that com- component comes in. Is that market, in your opinion, also one with a short li- short time to live or is that just we're just starting to s- see the surface of the- No,

Caleb Sima: I, I, like here's... Maybe I can clarify. I don't think that there is a...

Caleb Sima: There is not a market that has a short time to live. What I do think is that there is a th- there is a con- you know, a compression of it. Yeah. Right?

Ashish Rajan: So you're saying that at least... Oh, sorry, and this is in line with what I was thinking as well. So you're saying that overall sec- cybersecurity as a field would have this forest fire?

Caleb Sima: Yes. We're gonna- Is that what you are- We're gonna, r- right now, cybersecurity, and you could debate every industry could be seen as bloated, right? Yeah. Fair. And so, what AI is bringing is a pretty hardcore diet, right? In the sense that, hey, I no longer n- there's so many people who [00:26:00] just need, I just need, some software engineer to write this simple user experience interface on top of this simple process, and I-- that, then a whole vendor gets create- a whole company gets created around that, and then people pay money for it, right?

Caleb Sima: Yeah. I think a lot of that stuff goes away. This fluff, this bloat goes away, and I think we, in cyber, have a lot of that. We have a lot of very niche, very focused verticals because we are so reliant on the, arms race of keeping up with the attackers, and everyone needs these little niche things to go do whatever.

Caleb Sima: I need a, I need something that tells me whether a vulnerability is critical or not. I need something that tells me whether I can patch a system or not. I need something that tells me, whether there is a source code issue or not. The, all of these things I think just get compressed and thinned out, and none of them go away as a need or as a requirement.

Caleb Sima: Yeah. It's just [00:27:00] that now, everyday operators or companies have s- have way more capability to solve little glue, you know, niche vertical problems themselves. And therefore, it leaves and weeds out a lot of that bloat to the things that are primary and core to what you need to do really well at, to do it at scale- Yeah

Caleb Sima: and to do it consistently.

Ashish Rajan: Is there like a... And by the way, I 100% agree on this 'cause... And, and that's where the idea of the whole scaffolding came in from, at least in my mind. Do you feel that there are certain subsets of cybersecurity that would be impacted sooner rather than, like, later?

Ashish Rajan: Obviously, there's the stages, like the biggest one right now is the coding assistant space, the AI SOC space, and NHI space. These are... But not many people are talking about GRC. Not pe- people are talking about cloud security, not people are talking about security awareness, and there's a lot more to security than just looking [00:28:00] at code security as well as your NHI, the identity pieces.

Ashish Rajan: Uh, at least are there, is there a... I d- I don't imagine the entire cybersecurity forest is gonna light up in one day, or I guess in a very short time period altogether. But at least I f- in my opinion, I definitely find that your, uh The, the biggest ones distilled in my mind seems to be today, uh, is the AppSec piece is just going to be the first to hit in terms of the eradication.

Ashish Rajan: Would you agree?

Caleb Sima: Uh, yeah, I would say the best way to think about it is, is just follow where is the massive hit and impact where AI is focused on. So AI is focused on first coding and code generation, right?

Ashish Rajan: Yeah.

Caleb Sima: So all you have to do is follow that bubble. Okay, well, where in security is the code generation the most, you know, affected?

Caleb Sima: You know, these are obviously, code analysis tools, AppSec tools re-revolving around code. [00:29:00] However, what you see is you see, again, a little bit of s- you know, startups that just do standard static code analysis will die, right? Yep. But primary core people are going to blow up, right? You're gonna see un- you know, people...

Caleb Sima: Like, it's funny to me, people will say companies like Semgrep are just going to die because I can just emulate this in AI. And on the first impression, that is true. Well, because I ran Claude Code and it not only gave me better vulnerabilities than Semgrep did, it was smarter about it, right? But the problem is, well, if you actually move that into production, and this is what people are experiencing now, is a year later they're like, "Oh, sh- shit," yes, it does do that, but it's super, super ad hoc and not reliable and not consistent.

Caleb Sima: So like, but you... But the code, uh, world is blowing up, so Semgreps, Endors, all of these guys are going to [00:30:00] blow up as well in the sense that they need these tools to keep up with the growth that coding as an attack surface will create. Now we're moving into agentic, right? So this is agents with tools, with the ability to do operations to go and execute and do things.

Caleb Sima: So that is the big core of AI labs today. So I would say go look at the security industry. What are all the things revolving around that, right? Oh, well, I need to go execute something, authentication. Oh, I need to execute something. Oh, permissions. I need to go execute something, logging, visibility, SIEM, detection.

Caleb Sima: Like, all of... these are now, like, all you have to do is follow the wave of where AI is having its biggest impact, and then whatever is closely to that core in security is the things that you're, are gonna be impacted the soonest, right? And either both fluffing and getting rid of bloat to also blowing up and I think growing bigger companies [00:31:00] in the security industry because of it.

Ashish Rajan: Yeah. I mean, I, I won't disagree with it. I, I will also add to what you said, I, I 100% agree. That's where my, my notion of AppSec came in from. But I would also to what you are saying in terms of, uh, the, the short time to live, the reason I started looking at this is because recently I have been on a lot of calls where, uh, on the advisory service where the notion of who do I trust when the notion is build versus buy.

Ashish Rajan: And a lot of people who have started building are in that same-- Like people, obviously there, a lot of them are in different timelines. Some people are starting to build today, some people have built six months ago, and some people have built couple of years ago with the first version and now they're trying to go realize, oh, what else is required?

Ashish Rajan: One thing at least clear from the, all the three patterns was that AI kind of gives you the illusion that it's very easy to create a prototype. Now, I use the word prototype with intent because-

Caleb Sima: It's not an illusion. It does. It is, right? It makes, it makes it [00:32:00] easy to build a prototype.

Ashish Rajan: Yeah. 'Cause, 'cause you're almost like, at that point in time, the moment you think about building this in production, you start realizing, oh wait and obviously I'm using the example of Claude Code, but I'm sure it's the same across Codex and other places as well.

Ashish Rajan: Sometimes it just doesn't remember, even though it's technically in the memory, it may not remember things. And you're like, "But I'm pretty sure we spoke about this," and there is a memory.md file specifically calling this out. There, there are-- And then there is the whole pricing aspect of it as well.

Ashish Rajan: There's the whole aspect of what happens as my prompt evolves? What happens when I add more integration? 'Cause production systems are very live in their nature. It's not sitting on my laptop on a localhost 3000 port trying to emulate a, to your point, to your point, a coding analysis tool and going, "Oh my God, I can, I can do this."

Ashish Rajan: And I go to my boss and get an approval. Next thing you know, six months later, the people who are way more into the journey, what they find and the hypothesis there has been that we as security teams will continue to build some scaffolding [00:33:00] inside, which we have done in the past, but now they're, they're more AI-enabled, where it does not make sense for someone to build a full to your point, Semgrep or Endor or whatever else.

Ashish Rajan: It just-- there is just so much money and effort has been put onto it because the last thing you want, and we are back onto the cloud, uh, analogy again, where the question used to be, "Well, I can do this in data center how different is yours?" Or, "I can do security on my, uh, AWS environment. Why do I need a third-party security?

Ashish Rajan: I can just do this." They have-- They, there is no incentive for Claude, OpenAI, or any of the frontier models to be a, let's just say a, a Palo Alto, CrowdStrike, name X number of countries in diff- or name X number of companies in within cybersecurity verticals to do all of that. It just- Yeah ... does not make sense.

Caleb Sima: Do you remember we did a podcast an episode a little while ago where I was talking about the, I sh- I would just just do Palo Alto and get- Yeah, yeah. ... all the [00:34:00] basics and then build the things I need out of AI out of that. I do think there is definitely different ways of which security products will get delivered.

Caleb Sima: But going back to your point, um, and we have said this before, I even wrote an article about this is that, th- we're in this phase where the cutting edge people, the bleeding edge people in AI, they've now been trying to produce and build things for over a year, right? And what they figured out is, oh, on initial impression, it looks amazing.

Caleb Sima: Like, I'll give you a great example. Yeah. I can build a phenomenal vulnerability management or assessment tool or a source code analysis tool or a detection AI SOC tool, and it looks amazing with AI. And then I'm like, oh, I can screw these vendors- ... drop them on the floor, and then move on with my own stuff.

Caleb Sima: And then what they've learned, it's now been a year, and they've figured out, oh, I can't do that. Yep. Exactly. Like this is not working. So they are now, like I am seeing it where they are now [00:35:00] coming back and they are saying, "Okay, let's figure out a deal." Yeah. "I'm gonna get an AI SOC vendor. I'm gonna get a detection vendor.

Caleb Sima: I'm gonna get a vuln management. I'm gonna get a scanner." Like- Yep ... they're all like, "Okay, let's renew. Let's get back to where we are," because, like, this whole experiment has... They've learned a lot, and by the way, there is a huge, difference of capability that AI adds, but it absolutely just cannot eliminate the product.

Caleb Sima: That cutting edge wave, you know, now we're getting the... You're gonna see that wave continue in intermediate and mass. They haven't figured that out. So what they're gonna do is you're gonna see a huge load of customers that are gonna go back to a year ago where that, those cutting edge guys were, and they're gonna say, "Oh, I don't need vendors anymore.

Caleb Sima: I've looked at what AI can do. I'm just gonna do it myself." And you're gonna see security vendor revenue drop, right? And the security market go into a gutter, and VC money being very hard to invest in series Bs, series A companies because like, oh [00:36:00] my gosh, like all these companies are just saying they're gonna do it with AI.

Caleb Sima: And then what-- the same thing a year later from now they're gonna come back and they're gonna say, "Okay, well that didn't work." And so like I think there's this time period of these waves of where security vendors and others are gonna have a really rough time because the, you know, as AI goes through its adoption cycle- Yeah

Caleb Sima: you're gonna see this affect them. But overall I do think, and it- the good thing is we will see a compression, a cut of bloat and fat out of the security vendors that do exist. So-

Ashish Rajan: I guess to your point, I guess this would be a sign for maybe if we have people who are in that AI security vendor ecosystem to be, how do you weather through that to be-- Like 'cause to your point, software is no longer the moat, then what is the moat for an AI security company, uh, in this way?

Ashish Rajan: Unless you are obviously a br- big brand which has existed way before AI was, there's a lot more trust behind it. I can walk up to my CIO and say, "Hey, [00:37:00] I'm investing money in a cybersecurity company that is publicly listed or has been there for decades, has done series B, series C or whatever." It's a lot more easier yes than a, "I'm a, a new AI, new AI security company who's solving this problem."

Ashish Rajan: But the, I feel like they probably are going to be the wor- the worst hit with all this.

Caleb Sima: Listen, I just had this discussion like last week. I was talking with a security team And this discussion just happened. It was, "Hey, there's this new EDR that is coming out that is way AI forward," right? Yeah. "And does things much better than CrowdStrike."

Ashish Rajan: Yeah. "

Caleb Sima: We should use them instead of CrowdStrike." And then my instant reaction to that is, "Why?" Is like I feel like, well, they're like, "It catches more things than CrowdStrike does." And I'm like, "Listen, man, like, but for what risk am I now taking on?" So I'm [00:38:00] reducing zero day maybe, you know, a margin of threat that this vendor takes, but I'm also onboarding what threat?

Caleb Sima: A small, tiny startup with who knows what security measures, with no capabilities, no trust, no reputation, no like- Yeah ... all of this is now new technology that I'm gonna be the design partner for. Like that is a huge amount of now risk that you're taking just because... No. Just use... Like what is your biggest problem set right now?

Caleb Sima: I can guarantee you it's not EDR. So why spend so much time dealing with that problem? Just get CrowdStrike, get it done, get your f- your fundamentals, focus on your bigger risks and problems, and then you can, when you have an ability to say, "Let's go look at cool new stuff," then go look at that thing when that time happens, if it ever happens.

Caleb Sima: Yeah. But you know, like, same thing. To

Ashish Rajan: your point, it's a much easier yes for me to say, "Oh, I'm gonna sign up for a Palo Alto Networks or a Zscaler or [00:39:00] whatever," because there's a, there's publicly listed companies, they're not going anywhere, and maybe a C, C, C or E company, I guess. But I would add to what you were saying about the notion of going for an AI security company, what would become the moat is not, is like finding...

Ashish Rajan: 'Cause th- th- to your point, being a design partner for a new company is gonna be harder today 'cause it's gonna be a harder sell for me as a CISO internally to go, "Hey, I'm gonna sign up as, as a design partner." But to be fair to them as well, if you are solving a problem and maybe it is a problem that I j- genuinely care about and is e- Man, sometimes I don't have the budget, I would still consider you.

Ashish Rajan: It's not like I would not, but I'm taking that risk on that, hey, I may not be having this vendor in six months' time because they couldn't afford to keep the lights on. And not saying that every new AI security vendor is like that, but if it was to be the case where if people are feeling, "Hey, I don't know if I have a moat," h- in all the conversations you've had with the AI security [00:40:00] ecosystem, of vendors, have you found what becomes the moat in this world then?

Ashish Rajan: So they don't, they don't remain the fat. They become the muscle for like everybody. Wow, I'm going for, uh, fitness analogies now.

Caleb Sima: Yeah, I'd-- I would say that, you know, moat is the biggest question for almost any company right now. And it is the hardest to answer especially in the world where we are sitting in a very unknown space with not knowing where models will top off or not and what will come out in the next month or two months.

Caleb Sima: And so you are seeing a considerable hesitancy on money being invested in startups. You're seeing, uh, a lot of hesitancy from customers and adoption because of this same type of problem space at least in the early, you know, cutting-edge companies. And so yeah, that's a real problem. And so, you know, I don't know how to answer the moat question, but I can say maybe in a lot of generic patterns, [00:41:00] this is the things that I would think about.

Caleb Sima: First and foremost, a moat has to do with a considerable level of intelligence or data or what's the right word? Sort of, networking capability that brings value that an LLM could never really produce. Like, here's a great example of a moat. CrowdStrike's distribution of endpoints allows them to see and identify so many more threats, attacks, and problem spaces just as a standalone that no one could ever get without having that level of distribution, right?

Caleb Sima: That r- that returns value to a customer that AI cannot replicate right now, right? That's a good example of a moat. No, but- Another good- Are you saying, are they distri- going into the distribution? 'Cause this is a moat for the new company though, right? Not for Cloudflare. No, th- this is what I'm, this is what I'm saying, if you're a new company, [00:42:00] how do you think- You need to find a moat which is- Yeah, how do you find your moat, right?

Caleb Sima: And of course, when you're new, you're not gonna have that, but is your direction or capability, does it incentivize your customers to do that so you can obtain that kind of moat, right?

Ashish Rajan: Okay. That, that makes sense. Yeah, to your point, how, how can I get to a point where easy adoption, but then has a huge distribution potential as well across the board?

Caleb Sima: Co- correct. Yeah like, you know, y- you know, it's just a... Yeah, there's, there's so many ways to do that, but the pattern, think about the overall pattern- Yeah ... which is- Yeah ... the ability to get the networking effect, which produces a set of intelligence that produces value that you cannot get from an AI model, right?

Ashish Rajan: Yep. Yep.

Caleb Sima: And then the second part of this, uh, in terms of moat is what you're seeing actually happen not in security per se yet, but in the general market is hardware, right? Which is anything having to do with AI, the next, quote unquote, 15 years of [00:43:00] AI is gonna be about robotics, manufacturing, automation, things that interact with the physical world because these things are very hard to do, very difficult, from chips to robots to whatever it happens to be.

Caleb Sima: It's very challenging to be able to do that in an AI software only world, so that becomes you know, a pretty good moat that you have there. Uh, so I would say that's sort of second to think about is what does that look like for you if you are doing anything in security related to that? And then like third, anything having to do with politics, red tape process crud, right?

Caleb Sima: Like, you know, a good example in these, these types of things is okay, my moat here is that I have already figured out all of the red tape in Fortune 100 companies in order to do invoicing of products, right? Like it [00:44:00] is a huge moat to get that level of intelligence, that sort of subject matter expertise that a- Yeah

Caleb Sima: generic model from public knowledge will not have. Yeah. And if you have that in a, in your own model or SLM or your own processes or capabilities, then this becomes a moat for you. Because, unlike Si- you know, Silicon Valley, which is clearly the, you know, 0.1% of the 0.1%, the rest of the world is filled with this junk that you cannot just rip and replace with AI, right?

Caleb Sima: However, if you are very expert, ec- have expertise in these things and you know this very, very well, that's a, that subject matter expertise is a, is a moat, I think. So when you think about in the world of security, what does that look like? Obviously, you know, you see people going towards compliance in that area- Yeah

Caleb Sima: GRC in that area you know, all that sort of... So vulnerability management has been a big- Yeah ... [00:45:00] process red tape problem space, you know. So y- that, that is your sort of moats that you're seeing.

Ashish Rajan: Yeah. That's a, that's a good point though. And I, I, yeah, I, I f- agree with all four and to what you said, it's not easy to nail that down as well, especially when you don't have an inside context of how an organization operates.

Ashish Rajan: And when you're already at a disadvantage for you only get one meeting with the person, you probably don't have the right connections or whatever. But I, I, I was gonna say I might add one more in there, that the fact that just because you don't have the distribution today does not really mean you do not-- would not get the levels of CrowdStrike, Palo Alto, whatever.

Ashish Rajan: I think it just, uh, if you have enough of a problem statement that can last you the next three, four years until the people who are experimenting figure out that, "Hey, this is not gonna work." 'Cause there-- we obviously established this multiple times in the podcast that there is a need for a security tool.

Ashish Rajan: It would be. It's just that [00:46:00] because AI gives you the illusion of a great prototype, it does weather away people, and organizations that are large take sometimes months over, yeah, some- maybe sometimes even a year to go through that rigor and come back and go, "Oh, actually, it's a lot more than I thought." So having what maybe that moat is to whatever do I have, can I replace that with the volume of people that I can let help me weather through to the point that these big people come back and go, "Oh

Ashish Rajan: I tried experimenting." Or actually the, uh, the, the-- just to add to that, the, to the first point you were referring to where it's a distribution thing, yeah, you could also be an accessory to distribution as well. Where I think the data lake concept has come back again in a lot of people where in the detection engineering space, where people are going away from SIEM and building their own detection as long as they have some kind of a data lake to work on.

Ashish Rajan: So it could just be around like, it could be an accessory to a distribution. Doesn't really have to be that, "Hey, I [00:47:00] need to be a log aggregator." No, you could be an accessory to it in the beginning, and then maybe that becomes your moat for now as the companies come back.

Caleb Sima: Yeah, like I'll give you you know, like an interesting example.

Caleb Sima: There, to your point, people are saying, "Okay, well, I don't need a full Splunk I just need a thing at which I can push data into, and then I'll use AI to just run my queries through that data. Yeah. Right? Yeah. Like, you know, Databricks is great at this. Shove all your data in your thing, use AI to drum up your stuff, right?

Caleb Sima: Yeah. That's, that's a fantastic... people are just doing that. "Hey, this is great." Then SIEM, as sort of a moat, goes away. However, it produces all new different types of value. Like, I'll give you a great example. Just because you have access to data, and just because you can use AI to query data, how do you know what detections to run?

Caleb Sima: Yeah. Where does that come from? Does it come from your head? Does it come from open source libraries that are out there? Like, does it come from Threat [00:48:00] Intel? No, it clearly doesn't. So, like, well, how do you know what to detect? And then how often do you create new detections, or what are your gaps or your coverage, and all of this needs to be created somewhere.

Caleb Sima: And so, like, if you had a company, like I'll give you an example. I announced Spectrum Security, which is a company that I helped incubate in White Rabbit, focused on closing that gap, which is, hey, if we now are connected inside of 50 enterprises and you can automa- and it automatically generates the detections you need to in that AI platform so that you have the coverage, that's huge benefit, right?

Caleb Sima: Like, oh, Spectrum sees 50 Fortune 100 companies' detections and data 'cause it's seeing it, it's generating these detections, and it's deploying and managing them. I can now tell you, "Oh, Ashish, I'm just spinning one up. I'm gonna create, I'm gonna use Databricks, I'm gonna use this thing." Great. Here is, I've looked at your analysis, your data, and your products.

Caleb Sima: I've now taken [00:49:00] all the best of what I've learned in the Fortune 100, and then I've applied and auto-generated those detections for you, right? And then you're like, "Whoa, great." That's value learned from the network effect. And by the way, the ability that you don't have to create that. And so, like, I think that's, like, a really nice moat, right?

Ashish Rajan: Yeah, 100%. I, I, I definitely feel there's a lot of, uh, adjacent things possible even today, uh, 'cause as much as we were talking about how security's going away, but there is definitely a lot, lot of adjacent f- spaces that are also becoming available for people to take the opportunity if they want, want to go down that path.

Ashish Rajan: But I guess the, the big conclusion we are definitely agreeing on both so far is the fact that security's not going away. The moat is what would help the newer AI security startups weather through the storm of the build versus buy conversation. For people who have been on different stages of the journey, the people who are my- on the much la-later stage have realized that, oh, I do need to build some kind of an AS- AI scaffolding or harness [00:50:00] to connect and talk to these, uh, AI security vendors or cybersecurity vendors who've been there for a long time.

Ashish Rajan: It's not that I'm building an entire product in my, uh, in-house. I'm just building the connectivity, the scaffolding so I can stand on it 'cause tomorrow, I don't know, my organization may say, "Claude for enterprise is not the right thing. We should go for open source," or I don't know, maybe even switch to OpenAI.

Ashish Rajan: Whatever that thing may be, you don't wanna be that person who's switching that around, and it has happened for people who are tr- trying to think of doing this. It does happen. Companies do change minds, and sometimes organizations have both Claude and OpenAI, and you may be talking to teams that are talking specifically only to Claude because that's what they wanna use, and teams that are specifically talking to OpenAI.

Ashish Rajan: But I think we have a conclusion, and I think we also have a direction forward for both people who are making this decision and also moving forward with, uh, hopefully an AI security startup that they can be a design partner with as well. But that's all I wanted to cover in this particular episode.

Ashish Rajan: Thank you everyone tuning in [00:51:00] and, uh, let us know what you, what you think about, uh, your thoughts on this in the comment section as well. We've been, we've definitely been enjoying all the comments you've been sharing. And yeah, we'll, we'll talk to you next episode. Thank you. Peace. Thank you for watching or listening to that episode of AI Security Podcast.

Ashish Rajan: This was brought to you by Techriot.io. If you wanna hear or watch more episodes of AI Security, check that out on aisecuritypodcast.com. And in case you're interested in learning more about cloud security, you should check out our sister podcast called Cloud Security Podcast, which is available on cloudsecuritypodcast.tv.

Ashish Rajan: Thank you for tuning in, and I'll see you in the next episode. Peace.

​

‍