Caleb Sima: [00:00:00] I think true software engineers understand the danger of vibe coding because like they get like what's going on, but it opens the door to people who have no idea how to code and it truly brings this capability where I think what I always like to think is people who are now managers in engineering, VP of engineers, CTOs, product managers who just wanna build the things that they want now vibe coding is giving this them this ability to create very real applications and deep prototypes and mockups without actually having to understand the code itself.

Ashish Rajan: If you have been wondering about coding assistant AI and how they're shaping since, at least since past couple of years of AI becoming this massive thing that's integrated into all our organizations.

This is the episode with Guy Podjarny who is a returning guest who is a founder of a company called Tessl. And in this conversation, Caleb and I spoke to Guy about how much has AI assisted coding change, and we spoke about the three chapters as it [00:01:00] has evolved from being a coding assistant to where it is today, whether we can still have AI agent driven code, what is vibe coding?

Because finally, we have someone who's been deep in this space to at least talk about what is vibe coding, unlike me trying to make it up as I go, because isn't that the vibe? No pun intended. But overall, this is a great conversation for people who are trying to understand that world of how much coding has changed and the workflow behind it and why it would not make any sense if you just care too much about the code you're producing Instead, it's more about the business requirement and the testing you do behind it.

All that, and a lot more in this episode of AI Cybersecurity Podcast. If you know someone who is working in this space in your organization, or probably a colleague. Definitely share this episode with them and in case you are here for the second or third time and have been enjoying the episodes on AI Cybersecurity podcast, definitely give us a a follow.

If you are listening to this on Apple or Spotify or if you are watching this on YouTube, LinkedIn, definitely give us a follow subscribe there as well. It only takes a few seconds for you, but we really appreciate it because it lets us know that you are [00:02:00] enjoying the episodes and you wanted to produce more of it as well.

Thank you for the support and now enjoy the episode with Guy Podjarny. I'll talk to you soon. Welcome to AI Cybersecurity podcast. We have Guy Podjarny. Thank you for coming back again to the podcast.

Guy Podjarny: Oh, thanks for having me on. I guess I didn't embarrass myself too much last time, or maybe I did and was entertaining and we'll find out.

Ashish Rajan: We love you so much. We wanted to come bring you back again. But for people who may not have heard of you before, could you just give a 30 second version of a little bit about yourself, your background, and all of that?

Guy Podjarny: Sure. So I'm probably my kind of biggest claim to fame is having founded Snyk nearly a decade ago now of it which is a software security company, a developer security company kinda built to really integrate security solutions into the day-to-day of developers. And did that by really building a dev toing company to tackle security and kinda building the company and the products of that process SNYK did well, it's now a north of a thousand person and north of 300 million ARR.

It's like like a good established, I think, leader in the developers security space. And I accepted at some point that I'm an [00:03:00] addict and that I want to do it again. And at the beginning of last year, I left SNYK to found a new company called Tessl focusing on trying to kinda reimagine software development for the era of AI.

Prior to that, I did a few things. Most notable to this podcast is I competed with Caleb a little bit as I was a developer and product manager on AppScan when I competed with Web Inspect at the time. I will paraphrase history and say Caleb last time accepted that we won that race on it, and then.

I found out,

Caleb Sima: wait a minute, I'm pretty clear we won that. I think we had a resounding beating that you guys lost.

Guy Podjarny: Yeah, I think It was a pretty good journey. It was not successful at getting into developers, but I think both companies sold to some enterprise behemoth in sort of IBM and HP and and are not dead yet.

Because AppSec, like companies ever die. And then I founded a DevOps company called Blaze. Sold it to Akamai. I was a CTO there for a bunch of years, so yeah, probably longer than you expected, but I'm Dev Tools got into DevOps and then, so security, DevOps, security dev mixing those worlds.

Ashish Rajan: But, and [00:04:00] now creating, and now firmly

Guy Podjarny: GenAI. The fascinating and painful world of that it is.

Ashish Rajan: So the funny thing is as a kind of refresher of what we had spoken about last time, I was listening to our episode from last year, and the part which was interesting was we spoke about stack of flow steroids is how AI felt.

And since then, vibe coding has become a thing. And I'm like, maybe that's what we were predicting. We just didn't know the vibe coding would become a word, but maybe set some context. And I guess maybe we are at that stage where a lot of people would know, would already know what AI assisted code is, but maybe if you can set the context for how you define it and maybe you can use that as a starting point to build on what white coding is and if it's a reality.

Guy Podjarny: Yeah, for sure. Maybe chart a little bit of a journey. I've actually just spoken about this at the, a native devcon that we just hosted. Awesome. For a second time. We've been investing in the community. And I think using AI for software development or AI assisted software development probably has three chapters so far in its story.

The first one is really these coding [00:05:00] assistance starting from code completion. Code completion. The first incarnation of just better and better kind of completion snippets for you to do. Originally GitHub copilot was was had the zeitgeist in now Cursor now is at the helm, but really today there are dozens and dozens of coding assistance.

Just help you write code faster that happened alongside the world's discovery of chat interface, not just for not just for coding. And as we learned what to do with chat gt one of the primary uses was to have it create coding statements for us, a little bit of, a little bit of a search for a Quora and stack overflow for us a little bit of mix them in.

And so we'd ask for, whatever, create me a python function that charges a transaction using the Stripe API, and it would know what the stripe API is and what is charging a transaction and what is Python?

Caleb Sima: Fix this error.

Guy Podjarny: Yeah. And and then what happened was you didn't know, you were like, okay you copy pasted that thing in.

It got an error. It's I don't know, I didn't write any of this code. I don't understand this API. So you paste the error back in to ChatGPT. And so that brought that chat interface [00:06:00] into coding system and those kind of pillars remain here. So now we have kind of code completion in chat in the in the IDE.

And those have grown more and more to go a Cursor did like a multi-line and multi file edit, so it predicts if I'm gonna change this line of code, you hit tap, you go to another line of code. And and then in the chat you can give a command and like propose changes across seven seven files.

And so those kind of the evolutions of more and probably for security people, you appreciate that the more changes it makes for you that you're glossing over as you approve. The more trust you're putting into the system because you're trusting it to make you grow dependent on it, knowing where is that next tab?

Where is that next file? So anyways, that's chapter one that continues to evolve. Be Swim Lane one, right? Yeah. As it as it grows still the dominant use, but at this point, like well established people are like shadow it, using it whether you like it or not in the policy. There are enterprise versions of it for local models and all that, and very much dominant. And that led to chapter two maybe, [00:07:00] which is this a vibe coding. I. I dunno if you, have you covered Vibe coding at all in this? No.

Ashish Rajan: We were hoping you'll cover that for us.

Caleb Sima: Yeah, we are waiting for you. Yeah.

Guy Podjarny: Yeah. Vibe coding, it was coined by Andrej Karpathy, which is one sort of the most notable kind of thought leaders in the world of AI.

He had this sort of tweet where he pointed out that, he's gotten into this habit using Cursor and Composer, which just makes an ask gets a bunch of changes but doesn't actually review them. He just doesn't look at the diffs. It just says, okay. And then, or does that a little bit.

And at this point he doesn't understand the code. So if he was to review the code, it's like he doesn't even know it. That's fine. And even sometimes hits a dead end. But it says that's fine. 'cause it's just like for throwaway weekend project that's not a problem. And kinda this term got embraced and this is the coding version of it.

And so this is people that are coders that are in an IDE. But they're building applications with a bit more full delegation, if you will, to the AI. Shouldn't be used in the enterprise. Is used in the enterprise. Yeah.

Caleb Sima: One thing I was going to interject is that I feel like I. The [00:08:00] transition from this IntelliSense smart, stack overflow code and or edit specific functions.

It went from this, Hey, help me write something, then help me write a function, then help me write a class, and then it started going into this way more automated capability of being able to create files on its own, be able to run and compile the actual program itself, and then determine the errors via the console.

And then recursively, do this loop where it would help fix itself, generate more code or run it, compile, fix itself, generate more code. And I think that's where it really entered into this capability of truly autonomously just coding things. And it gave this sort of open door to, I think where vibe coating has really become hot is, I think true software engineers understand the danger of vibe coding because like they [00:09:00] get what's going on?

But it opens the door to people who have no idea how to code. And it truly brings this capability where I think what I always like to think is, people who are now managers in engineering, VP of engineers, CTOs, product managers who just want to build the things that they want now, vibe coding is giving this them this ability to create.

Very real applications and deep prototypes and mockups without actually having to understand the code itself. And yeah, that's a pretty incredible transition over what the past year that just this all happened.

Guy Podjarny: Yeah I think so, and I guess that was precisely the next chapter I got to, which is, I think up until now we've been describing like a developer in the ITE developing code and getting more and more assistance from the from the from.

Oh I jumped ahead. Sorry. Sorry, Guy. No, it's all good. It's it's precisely where I was going and I think that kind of I guess I, I refer to these as prompt centric vibe coding. I was struggling a little bit with the term, 'cause, like the [00:10:00] Cursor side vibe coding was, I. Just accept all and don't review it.

Cursor actually created a YOLO mode which they've sadly renamed auto run mode. I liked the term YOLO better that that just, that actually does decide automatically like to accept all by default. Claude code has a similar one.

Caleb Sima: But, and I think Klein was the first I would say Klein was the first to do this.

Guy Podjarny: That's fair. You're right. It doesn't have a, like I know it has it, I don't know precisely when, but I think the these tools have a tiny bit of agent of that verification. They don't really do much when they do the Yolo. But on the other side you have indeed these what I think is prompt centric vibe coding in which you can see the code, but by default you don't even see it.

So you're not, there's no guilty moment in which you accept the changes, which you didn't do. Like the idea is that it's still attended, but you're supposed to review the product, not review the code, and the code is you can fall back to it if you are a person that can understand the code.

Caleb Sima: It feels more like managing an engineer than being

Guy Podjarny: an engineer. Yeah. Yeah. And that and [00:11:00] they indeed opened up this sort of door to like many poor people to create applications. And, as we should get into a little bit. With no supervision, no lifecycle management, no notion of maintenance, no securities.

Who's heard of that? Like none of those. And I think of these as like it's really optimal for disposable apps. For things that are a prototype, you just wanna make a point for a quiz game. You're gonna show to your friends for an app you built for a workshop. For something you're gonna be the sole user of and you'll run in your network.

Or something like that. And so for those environments, they're actually super fun. We've been using them, like I've been toying with 'em. I went on a road trip with a family on it and I went to I got all my, the, all the songs we heard in the car outta Spotify down the data. Ironically, Spotify doesn't have an upload to create a playlist from that data feature on it.

So I created a vibe coding app and created that, right? Or we created things for workshops at Tessl. That did that. So they're super, super powerful. I do want to maybe cap that to say like chapter three, where we're in today is. It's a bit hard to really differentiate it, but you [00:12:00] say agentic development that is meant to be enterprise ready.

And so like alongside, like if that was like second swim lane that started before, started, after coding assistant swim lane three is this sort of a gen development that's broad based and it's like Devin is the a prototype, right? A prototypical sort of app that people talk about. But there are a whole bunch of them.

All hands is open source for instance. And what they try to do is they try to be much more sort of broad sweeping. Like you can give them an ask, they'll go Google and search the web to extend their requirements. They will they will deploy. They will iterate, they'll expand, and they're the vibe coding apps like Lovable and Bolt and Base44 and others, they work within more of a confined.

Kind of a garden of they use super base and netlify and like they work, they build on this very specific stack and they try to create an application within those confines. They can't create, for instance, a mobile app. But the agenting development is a bit more of a sheer AI labor that's less [00:13:00] specialized that then you can harness and as you go through those chains, you get less and less assurances of what it is that it's easier and easier to create and you might say riskier and riskier to create as you walk down that time.

Ashish Rajan: I was gonna say in terms of, how I, maybe this is, I don't know if this is the four chapter, but what I've also been talking to a lot of people around, like the Figma guys had a conference recently called Config, where they released 'cause they were traditionally the, Hey, I make a a workflow for this is what my app should look like and I pass it over to a development team that goes and develops it comes back and, now we have a prototype for a image. I think as of recently they've opened the feature where the product manager or whoever is the UX person just does the workflow and it automatically creates the app in the background.

Are we also saying there's, and I don't know if you guys have seen this, I'm just curious. There's also a whole pool of third parties that have started coding on your behalf Canva does this as well. Figma does this as well, where they write the [00:14:00] code for you. They host the website for you, they host the application for you as well.

You can, I think at this point in time you can't attach it to your company domain. It would

Caleb Sima: Versal you. You can do that. Yeah. Oh, you do that as

Ashish Rajan: well.

So to your point Guy, what you were saying earlier as well, would that be like the fourth chapter where, you know how no code became popular when people could, Hey, no code's gonna take over.

Where is the difference between a no code, which I look at all these Figma and Canva and all of that. It's like almost like a no coding thing. I don't have to know what I'm coding. I just need to know what I'm trying to do. Yeah, and it does the code and makes the prototype for me. Where is the difference between a vibe coding and no coding kind of chapter?

Is there one or is this like completely different to no coding?

Guy Podjarny: Everybody thinks their sort of part of the SDLC is extra special of the in that sort of journey. And depending on wherever you are there are tools now for sort of GenAI to produce a version of that. And so I.

You use these vibe coding apps. They basically produce a design, right? And there are actually a bunch of tools that help you create the design with prompts. Doing it the design tools. They have the handoff to create something that is a working [00:15:00] prototype. So the coding and implementation part is there.

So they use that. You go on to of course, so the application building, once you have a system, but then test creation, you can do with that documentation generation. You can do with that. I think the further you go down this route. The harder it is for GenAI to be adopted today because basically more responsible generation is required.

And so for instance, in the DevOps space, you do see a decent amount of GenAI. But people are less comfortable with generation and actual deployment. And so what in DevOps is you see it for used for like root cause analysis, right? Where you're recommending a thing, but if you got it wrong, you might've wasted me time.

But maybe like a little bit of infrastructure as code generation, to, to an extent. It's just infrastructure as code. There are a whole bunch of attempted and automated SRE. That actually orchestrates and modifies the changes. But the level of trust you need to have in such a system is just way higher at the moment than anything that the systems can actually deliver.[00:16:00]

And so the way, like the reality is today is that I think there's still a chasm between kind of this increasingly bigger and smarter generation and and like enterprise level or even like any professional development level reliability that you need and the. The primary way that this is solved, and I say that with air quotes, is to is to basically rely on human review, which is increasingly a fallacy, right?

As it makes many changes whether you're reviewing the product in a vibe coding setting or you're reviewing the code within the IDE you can only review so much, and you we're here in security, so like clearly identifying the omission of something like the lack of a security control. We can talk a little bit about Slopsquatting.

I think it's actually a really interesting attack. We should. Touch on, but those reviews are like massively limited. They're not fun. Okay. And the other but that's a way that it's being addressed today basically by rolling over. It's like the self-driving car and saying no, the driver is at [00:17:00] the helm.

They're supposed to grab the wheel when something goes wrong. And then the sort of, the second way that is being addressed is to just create things in a more narrow stack. So V Zero for instance, is put in the same bucket as as like a Lovable or or a Bolt or even a Devin, but it's actually a little bit different 'cause they're really building towards their own stack.

So they're like a little bit more constrained in what they create. And you notice it, if you Ashish to your question about no code.

Ashish Rajan: Yeah.

Guy Podjarny: No code was really about building blocks and the problem was creativity was limited. 'cause you can only use these like lighter, like a limited sort of

Ashish Rajan: the Lego pieces.

You have finite world,

Guy Podjarny: right? Of things that have been pre-created. Yeah. While, LLM is like unlimited creativity. But the more unlimited creativity is, the more limit, like the less, the more likely something will go wrong. So they try to niche.

Ashish Rajan: I love the analogy of the building blocks versus it's kinda like what people did with a SaaS application. People thought, Hey, thick client is not good enough. I want a SaaS application. But the SaaS application comes with its own limitation. That's the same thing. [00:18:00] It's just that you don't have all the bells and whistles and you can't customize everything you want to.

You just get like a, to your point, a V zero of what you wanted, maybe not the V one of what you wanna put out on the internet.

Guy Podjarny: Yeah. Yeah. And I love the name V zero. Like I, I give Versal a lot of credit, like it's absolutely embodied it. Yeah. Yeah. But and they don't really aim for the same audience as like a Bolt or Versal Base44, by the way, interestingly, has actually trying to give you a much more holistic, they actually have all sorts of other settings around turning your sort of SaaS app. Lovable is going in that direction as well. So there's some identities forming.

Ashish Rajan: What is Slopsquat? You mentioned briefly that Slotsquat. Squad,

Guy Podjarny: yeah. Yeah. And Caleb, if you've heard of it in the

Caleb Sima: yeah, of course. It was a great, it was a great advisory. Go ahead.

Guy Podjarny: I guess kind of two preCursor to that, right? One is AI slop is becoming the, sort of the term for Spanish for a wasteful hallucinations or damaging hallucinations of content that AI is creating. So hence the word slop. And then I think we're all familiar with typo squatting in which developers install a [00:19:00] library and maybe they misspell it or especially whether a lot of these are have cutesy names, like SNYK you might call them with misspell them or something like that.

And then attackers might create a malicious library that is that has a name that is like, like the original one, but then so you, you'd have a typo and you would pull in a malicious library. Slop squatting builds on the fact that LLMs make up library names. And it's actually it's amazing.

We've seen this actually heavily at Tessl. You tell it one, someone on our team called it Amy on our team. Said that it's the LLMs cry for help. It can't do something. And so it just invents a new reality. I wish this API existed, I wish this library existed. And it just makes it up.

And so it actually makes it up a lot in this study. The I think it was socket, right? We published. Yeah. It was Slopsquatting is a term from before, but the study founded a fifth of the libraries were made up. I, that's more than what I've seen, I think it's a little bit less than that typically.

But still many libraries are made up and highly predictable. If you use the same prompt and you use it, you run it sort of 10 times it's likely you're gonna encounter [00:20:00] the same library names, and so now attackers can create those libraries and then now put yourself in the developer's shoes.

You're you've asked the LLM to create something or an assistant. It created a piece of code, it has a library that sounds totally reasonable for it to exist. Are you expected at that point to say, oh, I know all the libraries in this domain and I know that this library is not the actual name of a library that actually exists on it.

Like I I don't know, do a blind test. You're gonna get at best, one in 20 developers spot that. And especially when we're doing things with like new APIs. And so I think a real risk and actually a much more real risk than typo squatting. And there isn't really a solution for it, right?

Like reviews are just not a solution. You have to put constraints. You have to put some form of guardrails of like reduction of the degrees of freedom or whitelist not many people seem very interested in constraining, right?

Caleb Sima: Yeah, I was saying that the same thing as he's talking about what, in order to solve this problem, you have to have some sort of external boundaries or rule sets to, so for example, you might [00:21:00] say here is a list of allowable libraries to be used in our code and flag any of those things through another tool that is not AI obviously.

To identify that those libraries, so for example, many enterprises have dependency boxes and you'll want to ensure that the dependencies at which you're using are matched with what's in your code. Yeah. And as more and more of this coding goes, AI, black box style, those kinds of things become pretty important.

Guy Podjarny: Yeah. And I think that's the. If we talk a little bit about like security concerns, like many of the ones that are most real. We've seen this in previous tech trends as well are actually the same ones we've had before. They're just multiplied now. Yeah. And so in a new Yeah, a new way.

Yeah. And you should have had the dependency checking before for vulnerabilities. You should have had gateways 'cause typo squatting was a thing. And also same libraries seemed kosher and they were kosher, but now they're like massively vulnerable or they change their license or they're deemed malicious.

And so you need to have some of these [00:22:00] protections. Same for vulnerabilities in code. And so like fundamentally the solution is the same solution you should have had in place, but now you really have to have it in place, which is automated security analysis that puts guardrails outside.

Yeah, and I think the new thing is really that you need, I. You need to be able to now also put guardrails on what's inside the business logic, the functionality. I think in a bigger way, right? Put some, which again, reduce the degrees of freedom, right?

Maybe the stacks used, maybe the the functionality provided, TDD

Caleb Sima: et, which I'd love to inject a GI Joe ad here around. This is something to always remember that by and large, most security vulnerabilities are exactly the same since inception. It's just now done and represents itself in a different tech stack and as more magnified due to scale.

But the same patterns are the exact same problems over and over again. You [00:23:00] can predict these things like the clock, so you know, the more you know yeah should help you here.

Guy Podjarny: Yeah, and sadly, like I think also the other pattern in history. Is that people do not do not care. I think the, like the thing that worries me most is not like hallucinations and doing it.

I think if you accept that the code that the elements created is not gonna be perfect. Yeah. Which I think you have to accept. Then then really it's all about things that are around. And I guess I think I just had another sort of a fireside chat there and told that the age of risky software and I think, with cloud we had this sort of, amazing embrace of the cloud.

I. Because it was so easy. Yeah. And and in the press of that, like it had no scrutiny. People like, like which resources did people create and what did they configure and what happened to it two months later, let alone two years later? And we, we basically found ourselves in a jam, right?

And it, the CSPM world [00:24:00] really only showed up like quite a few years after, and even then took a few more years until they got really cracked by Wiz to to make it easy enough and it was it, like today I think it's probably still the biggest sort of single cause for breaches is probably misconfigurations, in the cloud.

And I. It's hard to remember, like infrastructure used to be a fairly secure thing. Like we, we had patching, like servers needed to be, we had figured out firewall, so true.

Ashish Rajan: But then we decided to open it to the internet. Yeah.

Guy Podjarny: And it was also because it's easy. So like, when we make things easy to create, then everybody creates them and we're all very happy.

And then they create them irresponsibly with no controls. And then we're all very sad because they're doing it. And I think. Like we're, again, it's amazing that everybody's a software creator now. It's amazing that you can create code so much faster. Yeah. But my sense is that in, in a short period of time, right?

In in a couple years time, we're gonna find ourselves not knowing what code was even written and where were these vibe coded apps even deployed to, and what happened to this thing that was great two months ago, but then [00:25:00] I forgot about it, and a year later it's a liability. And vulnerabilities in their code and over time 'cause right now it's mostly code 'cause they get deployed onto like specific stacks, but over time their infrastructure configurations and others are gonna become the thing that everybody gets hacked on in the not too distant future.

Caleb Sima: Because you had, we had, we called it cloud sprawl and now it's app sprawl.

Yeah. Because there was a health, like with cloud, there's this sort of healthy almost sort of friction that was created and you had to stand up a server. You gotta plug the thing in, you gotta put in resources. And when you made it as easy as clicking a button, then clearly that is going to go crazy.

And I, we have done the same with apps now, right? There was a clear area of friction where you had to be an engineer. You had to have a lot of knowledge to be able to go build something much less now deploy, and now you're getting to the point where you can build and deploy without knowing anything.

Guy Podjarny: Yeah.

Caleb Sima: And [00:26:00] enterprises are gonna deal with this app sprawl. And especially I think the pressure right now what's nice is hopefully most enterprises have a good sort of segmentation between what is quote unquote production and the development pipelines that can be pushed to there.

But you're gonna create an internal problem where everyone can spawn their own little apps to go all of these problems and running these apps. Almost like actually Ashish and I were talking in our MCP episode about how hey, are people now being able to run on their laptops, sort of API servers that are being created through MCP on their little Claude code thing and you can just go reach it and go and run things that they've built in their own little apps.

And you're gonna get this going back to almost the nineties when, host machines ran their own services that you could go and use. Do you, do we start seeing this sort of trend happening here? Yeah.

Guy Podjarny: Yeah. And I think to a large degree we are, and it's the beauty of creation.

The world of possibilities opens up and. But [00:27:00] it's, but then it's, but then it's a mess, and I dunno I feel by the way, just to add to that, just in our sort of echo chamber of oh no. Which, I'm literally building companies help people do more of this.

More responsibly into Oh, I'm with you.

Caleb Sima: I'm with you Guy.

Guy Podjarny: Yeah. It's not opposed to it, right? But it is about doing it right. But fundamentally this is further augmented by the fact that data access is is being given very prominently.

'cause everybody wants to do stuff with AI. And AI builds on data. Yeah. And it is able to make use of it. Like before you could write code. And if you could get access to much data, you still needed to go learn it, you need to doing it. So like another area of friction is understanding data and that understanding of data has gone away.

'cause now you can just point the LLM into a source or multiple sources of data and it just gets it well enough to provide real value very quickly. And so these are like extra vulnerable systems, but they're also like extra permissive or like with even more data access than we had before, if someone did breach that laptop, they probably have [00:28:00] more to do with it, than than before.

Ashish Rajan: Would you say then, in terms of, where organizations are today, you kinda mentioned the three chapters in the beginning from starting with, Hey, we'll do coding a system, then we'll have vibe coding to an extent. And then perhaps you, we get to the stage where now the applications being created where you don't, may or may not see the code in terms of where you see in the people you're working with or the community you're building.

A re people at different stages? 'cause I almost feel is there like a pattern that's sort off for you? Hey, majority of people seem to be chapter one, chapter two, chapter three kind of a thing. Has there been like some sense of that at this point in time? Because we've almost covered two years of AI so far.

Yeah. Where we kept things from the beginning that, hey, we can't trust the code. It does seem like we, we haven't reached a stage where we have stopped hallucinating. I. Yeah. So where are people at the moment today in terms of those three chapters?

Guy Podjarny: Yeah, I think so companies adopt alongside those lines, and so really within companies predominantly [00:29:00] you are seeing this sort of this first category of coding assistance and you're just seeing more and more usage of it.

And so enterprises have gone from denial and trying to block it to shadow IT happening and then eventual acceptance to now leaning in and mandating it. So more and more company are saying, you have to use this. Interestingly by the way, very few of them are actually able to measure the impact.

Like I've yet to talk to any leader who says, and now I feel like my development team is producing twice as fast. No, like they say, my developer teams love it. They all feel super productive on it. And some of them say, but I haven't actually seen that in the numbers. And they ask, can you help me figure out how to measure productivity over here?

So there's a, there's still some questions, but this general sentiment is that it makes everybody notably more productive, not 10 times more productive. But maybe 20, 30, 50% more productive. And it definitely is a thing that I think once a developer starts using with it, it's like in the internet, right?

If you were developing, like me, you're an old person, you develop without, before the internet, and then the internet comes along. It's I cannot develop without the internet anymore. So it's like that, like I you'd have to pry it other hands

Caleb Sima: I would also say I think [00:30:00] people are still trying to figure out.

Get used to the sort of new workflow. And so I know when I started, it took a really long amount of time to start understanding and figuring out your workflow. And then there are also those swap outs, guy like you said, where I used to spend a lot of time researching the internet in order to find the cause of my error to fix it.

Now that is gone. AI does that for me. But I've now swapped that time for more, having to manage the AI appropriately and design my requirements better and, do these things. So then my time sink has shifted into another area versus where it was, and just trying to like balance, that really takes time.

Guy Podjarny: Yeah. So that learning process is happening right now, but I think enterprises are mandating that with that actually comes some sort of policies around the security aspect of it. And what is, I see this with sort of the SNYK hat on it. You see a lot of people mandating a scan.

As part of it, like mandating the controls there's some better tracking of the percentage code [00:31:00] written by, I'm not entirely sure what you, how useful that stat is, but they track it. It's definitely an easy relatively easy metric to track. And so they, they track that adoption, but they all embrace that.

I think similarly ancillary uses that are more focused are getting more and more adoption. Test generation is like very much top, especially for legacy systems. Legacy modernization is being helped with it because they use LLMs to be able to understand the code and and port it for different versions.

And so it's really like a helper, again, very review oriented, but there's still like substantial savings of time that happened there. And so everybody's a bit more in fill like mode. I think the only real pattern of like heavily used, increasingly widely adopted is the coding assistance.

And then you go into like agentic, everybody's in trial mode. Nobody's really using the kind of broaded based, sort of Devin style usage of it. I think what is the vibe coding version is really much more in Cursor. We're basically just an individual development in YOLO and just go mostly I think what you're seeing inside companies is you're seeing apps built like new apps [00:32:00] built and domains in which you would not have bothered writing apps.

And so you do see a lot of that, which is which, which is cool and a little bit concerning for security and generally still very lacking. You talk to CISOs a fair bit, maybe more than me these days. And my sense is that people, I. The nobody I've yet to really see proper policies, let alone actual functioning controls around tracking these vibe coding apps.

So they're a little bit more like either blocking or maybe choosing a platform, that is just, you're allowed to use this type of platform here. I think that's all still evolving. And then acknowledgement of the problem. At least like some of the conversations are happening. There are more like prompt firewalls and things like that are more runtime AI security tools, which are loaded outside of this conversation.

Caleb Sima: I think there's definitely some debate to be had. I think you and I had this debate Guy already, maybe a year ago, around who cares if it's AI generated code or not? At the end of the day, you as an engineering team or as an engineer are taking [00:33:00] accountability for whatever code you commit.

And the tools and processes at which you have in place for existing code today are the exact same tools and processes that scan any AI generated code. And so at the end of the day, this sort of worry about more code being generated could produce more vulnerabilities per se and the fact that it does increase attack surface, but at the end of the day, you are still scanning the code exactly the same as if you were a human generating it.

Guy Podjarny: Yeah. I think maybe I'll if you don't mind me doing a Tessl plug here a little bit here, which is I think our up until now, we talked about current reality and I I think. There's so much fomo, so much like you have to embrace this today. Like literally big banks that I talk to talk about how all the way up to the board, they have to report AI adoption and it is a hundred percent mandated.

They big companies like massive companies like the Giants, the hyperscalers, Google and [00:34:00] Microsoft report, percentage of code generated by AI , which as you pointed out Caleb right now, like who cares? And so there's so much drive. That there isn't really enough thinking about where does it lead us.

A lot of the attention is more about where can I use it today versus where is it leading me tomorrow? And there's a lot of reliance on reviews, which in turn creates a certain obstacle to like how much broad it can be used. I guess our conviction, just doubling down, Caleb, on your point around code, is that we shouldn't care about the code at all.

In fact, the code should be disposable and that really software should move into a totally different world in which it is defined by a specification. And in that specification you say, here's, here are the requirements, here's what you need to build. And then here are some like means of verification that you sufficiently trust.

It's your choice, right? It could be as simple as ask the LLM to evaluate does it look like it, that checkout function is working? Or it can be something that's much more hard coded and that's your choice, but that is, those are really now your controls. And then from there, once you've built [00:35:00] that layer, the LLM can get to work and the system can come along and it can iterate and it can run.

The problem I think, has moved, the bottleneck has moved from being the competency of the lms. The LMS have gotten really good. The reasoning models review themselves with agent systems that are now MCP is like full of problems, security and otherwise, but it also is opening up a world of, like an ecosystem of tools for agents to use and it, the power is there, I think wielding it right now.

Harnessing it. That's the sort of the challenge in front of us. And so it's just freeform and it's just like humans, right? If you go to a bunch of intelligent humans and you just say, Hey, create me a spreadsheet app, you ask for it. Once you ask another group of intelligent women to create a spreadsheet app, they'll create a totally different app with some overlap.

But substantial changes. We think it's gonna, going in that direction. The development has changed, but that's, it's it's hard. We're getting a lot of excitement on it and we're getting some early adopters. The product is not in market yet, but the but it's there.

There's no [00:36:00] tool of this nature that has gotten an adoption, in volume right now. 'cause everybody's much more focused on. What can I do to embrace it into my current workflows today?

Caleb Sima: And I'm gonna give Guy, I'm gonna give you some credit 'cause it was easily over a year ago. We were talking, and the way you explained this to me was so phenomenally light bulb ish.

It was basically like when the world goes to where code is no longer looked at. It becomes a black box generated by AI. There are two things that become the most important. It's the requirements that you write and the testing that you do. And this now becomes the new cycle because you don't care about the code.

It's just, is it writing what I want it to write and validating that it does, and then that cycle, it feeds on itself in that new world, right? Where if it doesn't. Va if it validation fails, then you rewrite your requirements and then this becomes the new cycle. And it's really something you predicted such a long time ago.

[00:37:00] And I continue, my confidence and conviction in that I think continues to build. As you vibe code, as you do this, you start seeing that is what's happening, right? You are managing an engineer. You are the better you build your requirements and how you define what you expect and the better you write your test or the better your LLM writes your tests.

But the more you define those tests and how they work is produces the better product. Yeah. And and I thought it was pretty fascinating. And then how, again, pitching a little bit to you, you were like, and in that world, you want your AI and the way you write your requirements, you're gonna need to help in order to do that.

And what's great about requirements is they're easy to understand and they can be standardized if you do the right things. And tell it the right ways, which is very cool.

Ashish Rajan: As you say that, I don't know if you guys saw ChatGPT made a direct integration with GitHub, I think a couple of days ago. So you can integrate basically a ChatGPT directly into your private and public repos.

Yeah. Internet kind of went into this divide of hey, how [00:38:00] would you let your million dollar software be plugged into ChatGPT? And it's not gonna learn from it versus to what you guys said, Hey, as long as the requirement is met, because ChatGPT has this memory capability, and if you define your business requirement for, hey, as we are an organization that is building the best software for.

I don't know, booking flights, I'll just say, and whatever is required to make this optimized and save on performance make sure it's secure, like whatever the basic requirements, the top three would be. Where do you guys stand on that, I guess at this point and where, considering what we have spoken about so far, all this while that yes, we should not care about code.

Ultimately it's the requirement and the rigorous testing around it, whether it's automated, as long as it matches the requirement we're, we are good. And I'm with you guys in the future. But today as it stance, when we do have an integration where we are saying that humans are gonna still require to be that person in the middle to validate, yes, this is a good change, bad change, should I only use it for new [00:39:00] microservices or API that I'm creating, or should I plug in my legacy code there?

What's the feeling about for people who are still hesitant to open up that door? 'Cause I, to Guy what you said I think I was watching a report on this. It seems like the majority AI use cases are in that larger financial organizations, the big techs, the big enterprises, they're the ones being pushed by the board.

A lot of audiences still hesitant as well. They completely go no block LLLM, because we don't know how to protect this. Policies are harder because I have to do all these lunch and learn sessions. You make sure that Ashish, the developer, doesn't use an LLM and hence creates the shadow LLM, but hey, this is a different story.

Guy Podjarny: Yeah.

Ashish Rajan: Where do you stand on making pe making those people who are on that fence for, I don't know. It doesn't sound like a good idea yet. Are we yeah. Are we at a confident state that this is going to get better and better? Or are we saying that chapter one that we're on, that's where we're gonna be for the next two, three years?

I guess where I'm coming from is how do they plan for this? Yeah. Is it delayed?

Guy Podjarny: I think a bunch of things to unpack there. One I think [00:40:00] people continue, we might have even touched on in our previous conversation, people over rotate about the, Hey, they will train on my data. I think when they make commitments, the big guys, if you're connecting to some sort of, like a tiny tiny company somewhere, maybe you have trust issues.

But I think the big is that when they commit to not training on data, they actually mean it. And so I think people overwrote it. It's not a zero risk, but I think it's way smaller than they think. I think Caleb, you were making this point in kinda my last conversation with you. The I do think we're past the point, so if I look at what has happened like in the meantime and I appreciate Caleb, the kind of kind words there, I.

But like in, in this year is, I think we've gone to a place in which the models can start delivering on it. And we've really seen a step change, especially since the reasoning models came into play in in their ability to to really generate really notable and impressive things and understand requirements and each rate.

And I think the other thing, I built an appreciation for. Is synthetic data. And the two are not unrelated. The world and specifically the big AI labs have built very good mechanisms today around creating code, [00:41:00] reasoning about it, running it, getting execution flow back into the systems.

They're spending so much money on GPUs that actually, this is not even notable. They're spend cycles on it. The embrace of coding assistance, while it doesn't share the data, it does share signals about co completions that are and aren't adopted. So those have improved things, and all of the labs have demonstrated that IP does not remain a mote.

It's like they all, which behind the scenes probably implies distillation, which is this notion of using a model to teach, to extract data. Teach DeepSeek was like maybe the most visible version of it, but they all do it. The research papers, and so I think the competence, the competency, the skills of the LLMs are on a trajectory which I feel more and more kind of bullish about.

And the question is really about how do we engage with it? One of those competencies is understanding your code base. As more and more code changes have been applied, again, that flywheel of seeing when it worked, when it didn't work. Bigger context windows to be able to take in [00:42:00] more data, just smarter and smarter models.

Today they're getting very good and we've gone from no context, it's just the generic developer landing in your IDE to like understanding your project to today. A bunch of these tools really offer you fairly thorough understanding of your intentional knowledge and they, they actually have a problem where they're still not amazing at the garbage in, garbage out of understanding what is good about your knowledge base and where your code base that should be replicated and what's not.

But but I think it is building and so people should focus. Absolutely. If you've got your head in the sand, everybody's gonna be using it.

Caleb Sima: I did a BSides keynote last year. If you recall, and in, in that, I predicted that we would have self updating documentation like wikis.

And just last week I saw a project called Deep Wiki. Which basically auto updates a wiki and creates a wiki of your code repo. So it will take your entire code, understand it, create an entire Wiki, and any changes that happen, it auto updates your documentation [00:43:00] for it. And I was like, this is, and same, similar, I think you're gonna see that's not just in code but in enterprise right?

About your team. Every, remember in Wikis. You're, you represent your team, changes in your team org, reporting structure, documentation. I think all of that is also gonna be very similar. Going down this road of documentation and representing change in documentation will just be automated.

Which is very cool.

Guy Podjarny: Yeah. That would, the only thing I would highlight there is that the computation is easier than code because nobody builds on it. Nobody runs on it. Like there, there is dependence. People look at it and they try to understand, but, the blast radius of a mistake is just that much smaller than a mistake in code, especially when you another thing that hasn't been cracked, I guess in the world of code generation is reusability and it's not by mistake.

And so these things just beyond making up libraries, they, yeah. They use libraries wherever they find them. By the way, they don't use them wisely. They use whatever is prevalent in their code base. Again, kind of garbage in, garbage out. Ah. So they're not really aware of [00:44:00] vulnerabilities, but importantly they don't break up their own code.

If you if on a left alone letting kind of these applications, this is a s build more and more, right? And more, more and more code, we'll get more and more duplication in our system. More and more inconsistency with different apps, logging differently, having different, things that

Caleb Sima: you have to specify that you need to refactor this into modular reusable components.

Guy Podjarny: Yeah. And even then there isn't really like natural sharing models. There isn't really a way, to think about how do you publish these things. And it comes back to like, when do you even know, like what is the level of knowledge you're supposed to have to be able to know, to tell?

It's to decompose into into systems as I, I note, in Tessl decomposition is a natural part of the, the the flow of it. You have to break it apart into pieces. And so I think there, there's a lot there, but the skill is there. I think attention should be put on.

How do we interact? There's like a new, there's new, a new kind of guest in the party, right? It used to [00:45:00] be, you have a human making decisions and then you had kind of code and software that was running. Now there's another party in here, which is the LLM, and you have to say how do, how does the human talk to the AI?

And how does, and then, how does AI write the code? And like it's a triangle and triangles are complicated, and we need to think about how do we. And that's before you do introduce what does it mean to organization? I wanna add one thing that's I think an interesting learning from the previous spec conversations.

And I hope Caleb, that I have learned something, like in the year since I have it, like I'm happy to have been right maybe before, but hopefully I'm learning. And and that is an appreciation of the workflow. And so if you take that analogy of the dev becomes a dev manager.

That kind of is managing a set of AI labor. Yeah. That is building it. Then, what does the dev manager do? Yeah. In part is you say, what is it that we're building and how are we gonna verify it? The other thing they decide is, what is my delivery process? And and that is necessary when you think about enterprise adoption every software that comes out, I need to know that it's been vetted for security.

I need it to be all [00:46:00] vulnerabilities have been fixed, or at least I've tried that, but it hasn't been omitted. The, it is auditable and traceable. It's deployed on its infrastructure, right? It it has been broken apart into into units, right? Or has made an attempt to reuse software as it could.

And if you're managing a capable set of people under you, then you don't specify everything. You choose what is the degrees of freedom that you leave to the intelligence below you to fill in. And and I think that's important. So I more and more think that an AI native developer really is in charge of two things, not one.

One is the spec and the second is the workflow.

Ashish Rajan: I love the conversation. I think I was gonna say, we spoke about the future, but in terms of I think we could have had a long conversation, but in terms of final thoughts. Where we are headed towards. ' Final thoughts on what's your favorite coding assistant do at the moment? Guy. Yeah.

Guy Podjarny: Cursor is my go-to. Not, I don't code much, but when I play with it, I do Cursor. I do vibe code with Lovable right now.

It's fun for side things.

Ashish Rajan: Oh, fair. And the second one which [00:47:00] basically. We spoke about the future of AI and AppSec. Is AppSec gonna evolve yes or no? With this.

Guy Podjarny: I think AppSec is more important than ever and I think it needs to truly double down on automation and and like leverage AI and sort of GenAI within the systems.

I think actually that world has already moved to being AI powered as it stands. Yeah, I think I think, no doubt I, coming back to that age of risk software, it's like asking before the cloud, is patch management going to, improve right? Or sort of infrastructure configuration detection.

I think I think AppSec is more table stakes than it has ever been. Yeah. And I think that would be, many times more true in in five to

Ashish Rajan: 10 years, what Caleb said earlier, that just the same problems in a different landscape, yeah, indeed. At scale. At scale, I love the

Guy Podjarny: actual a term on it.

I might steal that.

Ashish Rajan: Awesome. I feel like we should have another conversation about this whole topic again. Where can people find you and find about Tessl and everything that work that you're doing?

Guy Podjarny: Check out Tessl.io for platform sign up to the waiting list, hopefully not long before you can try the product.

In the meantime, check out a lot of great insights about AI [00:48:00] and development at AI native dev.io. We also have our own podcast. We get to talk to smart people like yourselves and and so we know many Dev. And AI leaders on it. So check out AI native dev.io. Maybe one thing is that the AI native dev landscape is something we released recently, which just tries to help people keep on top of the many AI dev tools out there.

I think developers, but also security people and kind of leaders would find it useful to just know what's out there and consider how they wanna handle it.

Ashish Rajan: Awesome. Thank you so much for that. Thanks very much tuning in as well, and we'll see you next episode. Thanks Guy for doing this, man. I really appreciate this.

Thanks for having me. Thank you so much for listening and watching this episode of AI Cybersecurity Podcast. If you want to hear more episodes like these or watch them, you can definitely find them on our YouTube for AI Cybersecurity podcast or also on our website, www.aicybersecuritypodcast.com. And if you are interested in Cloud, which is also assisted podcast called Cloud Security Podcast, where on a weekly basis we talk to cloud security practitioners, leaders who are trying to solve different clients cloud security challenges at scale across the three most [00:49:00] popular cloud provider. You can find more information about Cloud Security Podcast on www.cloudsecuritypodcast.tv. Thank you again for supporting us. I'll see you next time.

Peace.

‍