Caleb Sima: [00:00:00] This is not just writing code like Rucode Klein and others isn't just writing code. It compiles it, it runs it, it has access to the command line, like it gets the errors, it gets the feedback. So it ran Terraform deployed, got the errors, fixed the errors. Deployed again, got the error fixed, the errors deployed again, and it works.

And then it'll run test cases. So then once it deploys, it will run tests to ensure that the actual instances deployed in AWS work properly. And if it fails, it will see the error fix the Terraform redeploy. And by the way, all of this is happening within five minutes. So for me adding Terraform AWS deployment support took me about 10 minutes.

Yeah. Like I, it, it did the thing, it wrote it errored a couple times, fixed it all, deployed it, ran the test. It worked. And by the way, here's what's really interesting.

Ashish Rajan: Today's topic is vibe coding. Yes. You heard that right? We have been talking about vibe coding, at least offline for a long time on the internet for a while.

[00:01:00] And personally, I think experienced people like you and I should be more aware of vibe coding than people who are starting today building prototypes that are probably becoming memes on the internet. So in this episode, Caleb and I talk about what we did for Vibe Coding, our personal projects, our personal favorite ones.

We also spoke about where Vibe Coding is today. What would it take to get it to production, and what are potential challenges you would see as more people start Vibe coding in different departments around the organization and what that future could look like. If you know someone who's working on solving the Vibe coding, and I must say this may not truly be a only security episode, so there is a lot of technology conversation here.

Even if you're not technology person, tRust me, you'll still understand a lot of the things we are talking about. My belief is that more experienced people should be aware of Vibe coding and getting their hands dirty with the whole AI tools that are there today. How else do you know what you're building and securing in this organization that you're working for?

If you know someone who's working on vibe coding or just wanna know what that's about, what the gaps in the future are, [00:02:00] definitely check this episode out or share this with a colleague who is working on the same. And as always, if you are here for a second or third time watching, or listening to an AI Cybersecurity podcast episode because you have found it helpful, I would really appreciate if you could take a moment, whether it's on Apple or Spotify, or on YouTube or LinkedIn, to hit the follow subscribe button because it means a lot when he shows the support it takes only a few seconds.

But I appreciate the support. Alright, I'll let you enjoy the episode and I'll talk to you soon. Peace. Welcome to another episode of AI Cybersecurity podcast. Today is a bit of a different vibe and I use that word intentionally. We're talking about vibe, coding and maybe to set some context, Caleb, do you want to give an idea to people of why, what is vibe coding?

Caleb Sima: Yeah. I want to do this episode specifically because it's gonna be less security related. We'll talk about security at the end, but fundamentally people need to under, like with any new technology, the best way to understand the security impacts of it is to understand the actual thing itself.

Yep. And [00:03:00] so I've tend to define in conversations, everyone has heard of vibe coding, everyone talks about vibe coding. Everyone repeats mantras of things at which they've heard other people. But actually when you dig, I find many people have not actually vibe coded themselves. And I think it's really important that you actually do it.

And so what, in this episode, I really want to talk people through exactly how to set up. Go through Vibe code, something, ins and outs, tips and tricks, tools to use, sites to go to just everything that you can do to go do it yourself. Because I think it's super important. And by the way, with something like Vibe Coding, it's really easy to start and I just don't think people understand how easy it is.

So that's why I wanna go through it.

Ashish Rajan: Yeah, and I think I was saying this before we started recording, I think, at least for my personal reasons, I definitely would prefer more experienced people use Vibe Coding instead of people who are just [00:04:00] starting off. And maybe that's how we get to the next level of enterprise AI that all of us want to get you.

And probably we don't have enough data or people trying to scale it out. So maybe all the experience people trying it would definitely make a difference. But to, now that we have set the context for Vibe coding. What is a good place to start? Because people have so many options. GitHub Copilot. There is the enterprise options and then there is the internet options.

Which one do you wanna start with?

Caleb Sima: Yeah, I think the first thing is I want people to go download Visual Studio Code. That is, I think, step one. And people are gonna this is not intimidating, right? It is. Hey, I was gonna say people are scared already. Yeah. It's really simple. Like you just go download it. You open it up, it's gonna pull up this thing and it like, it's really easy to figure out how to go do these things. And then and actually once you open up Visual Studio. You don't, all you gotta do is install an extension, which is similar to installing [00:05:00] any sort of Chrome extension, which I think everybody has done.

Ashish Rajan: I'm assuming a lot of people even know what visual code is.

Caleb Sima: Yeah.

Ashish Rajan: So visual. Yeah.

Caleb Sima: Good. Go ahead. Visual Studio Code is an IDE, so this is what your engineers, probably more acronyms. Come on. IDE people are like, what is an IDE? Okay. If first of all, I'm pretty sure you, if you're listening to this, you've gotta at least know what an IDE is.

Ashish Rajan: You would be surprised . But I'm just mindful of people. I don't wanna integrated development

Caleb Sima: environment,

Ashish Rajan: Yeah, I know I was becoming more simple. I was gonna say, Hey, IDE is a fancy word to say, this is where you write code.

Caleb Sima: It's where you write code. Yeah. Literally. Yes. Literally. And visual Studio runs on Windows and on Mac. And so now, okay, before I actually even start down this why should you go and do this? And what I'm about to tell you guys to go do? First of all I wanna make sure that I set the tone.

If you are a head of engineering or a CTO or even a CISO who used to be technical but never code, and you're [00:06:00] like I, because I am in that position where it's just I used to be an engineer. I was a core engineer. I would call myself technical, but the thought of actually going through and writing code and having to look up the syntax, remember these things like, oh, how to do this, how to do that.

This is, perfect vibe coding is perfect for you because you literally never actually have to code, but yet you can see the code. And so this puts power back in your hands and gives you those people who always used to love coding, but it's just out of your reach due to time, due the level of effort due to the cognitive overhead.

This puts the power back in your hands. I cannot, I don't know, emphasize this enough. It is amazing once you start getting into it. So that being the case and to people who aren't engineers, if you're a product manager especially, this gives you the ability to create things without [00:07:00] having to have an engineer.

Vibe coding is very much like running an engineering team without you being the engineer. Alright, now with all that said, you gotta go download Visual Studio Code. You install an extension. Now there are a bunch of sites and places and tools to use. So before we get started on this, I want to talk a little bit about all the options of tools you can use, why you can go use these tools and what they're made of. 'Cause there are a lot of them. In fact I think in one of our last episodes guy, he has a website that you can go to lo go look up all the

Ashish Rajan: codes. Do you remember what the It's all, yeah. IO has a website for all the software development tools for AI assist software development tools. I can link put the link in the show notes as

Caleb Sima: well

Ashish Rajan: for people.

Caleb Sima: Yeah. So everyone has probably heard of Cursor. That is an automatic, now Cursor is not in Visual Studio Code Cursor is actually a a dedicated [00:08:00] version of Visual Studio just called Cursor. So when you go download Cursor, it is an entire app, although it is. Basically cloned version of Visual Studio Code made for their own.

There are a whole bunch of others. For example, there's one's called Klein which is an extension to Visual Studio Rucode, which is an extension to Visual Studio. These two, by the way, are my favorite. So as we go through, I'm gonna talk about specifically Rucode because that is my preferred I would say Vibe coding tool.

Have you tried windsurf yet? Yeah, there's Windsurf, which Windsurf is also its own version of Visual Studio Code, I believe. Similar to Cursor, also pretty good. There's Bolt New which I think is more UI centric more than backend centric. So it's great at generating front ends and UIs. There's Ader which is more command line.

This is if you really like console. Ader is there and there's also Claude Code, which is another favorite of mine, which [00:09:00] is similar to er, it's er on steroids, but it is Anthropics vibe coding tool. It is fantastic. By the way, in fact, I generally will swap between Rucode and Claude code when I'm doing things.

Ashish Rajan: Have you had a chance to look at the GPT-4 0.1 yet?

Caleb Sima: I have not used GPT-4 0.1 in code yet. I'm generally a Gemini two five person in Rucode and obviously Claude Code uses Anthropic. They use three seven, which by the way, they have a little, I think they've got some special backend tips and tricks with their client.

Yeah. That allows the, they, you get some special privileges with Anthropic and Claude three seven when you use Claude Code. Yeah it's quite good.

Ashish Rajan: I personally went down the rabbit hole of Windsurf. I think when we were talking about this particular project, I was I had this repo in my GitHub, which I have kept vulnerable for a long time intentionally. It was, it's one of those repos that I show off at a conference talk or [00:10:00] whatever. Hey, look, this is how you make a vulnerable thing. Yeah Windsurf was what caught my attention, but I did not look at Gemini 2.5, so I learned something there as well. But maybe, do you reckon it's worthwhile just sharing.

What do these serve as a purpose? Because I think you mentioned people download visual code. You mentioned, hey, now you have to have a cursor Rucode V zero. Yeah. What would this serve as a purpose? Why this?

Caleb Sima: Yeah. So generally, I think in the land of what I would call vibe coding, I think there's really f good frontend products and good backend products.

Yeah. Generally the way that I work is I create the backend first. Then what I'll do is I will make it an API and then I'll generate an API documentation and then I'll go to a front end, like for example, lovable bolt new V zero, and then I'll just pass in the description of what the product does and the API specifications and then say, [00:11:00] generate for me the UI front end.

Interesting. And then I'll keep these projects very independently separate.

Ashish Rajan: Oh, would you believe I go opposite? What's that? I would just believe I go opposite and maybe because I'm a visual person, I go front end first and then backend.

Caleb Sima: Oh, you could do either one. Yeah. For example, and actually this is where, we'll, we haven't gotten there yet, but we'll get there.

And is the first thing you do in any of these things that I do is I create a product requirements document, which includes a user experience front end. So so like I, you wanna generate your front end at the beginning with your requirements before you start doing any code. Yep. Yeah, which actually go, goes into maybe some tips, d do you want to finish the list of things or should we just jump right into ,

Ashish Rajan: now that you have set up the purpose people understand visual code. Hey, I downloaded it's a product from Microsoft. I picked a tool, whether it's pick root, code favorite, my favorite code.

Yeah. But at least you're in a position to start thinking about a project. [00:12:00] But before we go on the list of things maybe worthwhile talking about what kind of projects, because you have to, I mean to you what you said. People who could be listening or watching could be project managers who have done a lot of large scale projects, but at the same time could be technical CISOs or ex technical recovering CISOs.

What are some of the good projects to start off with before we dive into some of these things as well?

Caleb Sima: I think projects, it's anything and everything. It doesn't really matter what you want to build. I have the, I always start off in any project, this has been since I was a kid, learning how to code.

For some reason, I always start learning things with web crawlers. How do you write a web caller? For context. You did

Ashish Rajan: write a book around the whole was it, what was it? Hacking Web applications exposed, that's right. I can see what. If you were doing this since as a much earlier in your life, I can see why you would've gone down that path of web crawler.

So you've went down the web path of web crawler for vibe coding

Caleb Sima: [00:13:00] as well. Y Yeah, because I just, the one thing that's great about web crawlers is in its most simplistic form, it's really simple. But web crawlers over a series of iterations can get very complex and can be very simple or can be scaled out at high performance and high distribution, and also be very complicated.

For example, I always start out with, okay, just crawl a webpage. Can I give you, can I pass with you the maximum amount of webpage? Can I pass, how many URLs, how many loops, which your depth versus your width. Like all of these things are very simple parameters. Yeah. And then you can, and then when I do, and by the way, we're going off off structure, but your web crawlers

Ashish Rajan: are No, but I think it's important because the reason I like what you shared is also because maybe people can choose things that they're most comfortable, where to start off with, like something they would've done in the past at, they're pretty familiar with.

Even at a foundational level, they can use that as a starting point to vibe [00:14:00] code 'cause like web crawler is your favorite. For me, it was the whole AppSec pipeline. Infra pipeline is when I, the part that I took because that's what I was comfortable with. Yeah. So people listening and watching. I guess maybe that the lesson here is you don't have to go with either of the examples you and I are sharing, but something that's.

They understand, they love, they've done many years of it, so they have some idea and maybe use that as a project. Would you agree?

Caleb Sima: Or just like annoying things, like I'll give you an example of a project. I have, I can't even tell you how many, hundreds of thousands of random documents on my Google drive or in my folders.

And I just wanted to find something that would take a snippet out of that file, summarize it, auto categorize it, and then label it and then move it into its folder structure. And so I vibe coded a system that did this, it just went through every single file, determined what type of file is, what is it pertinent to, and then tagged it.

And then moved. It's for example, it'll read a [00:15:00] Excel file and go, oh, this is about finances. Technically this is about taxes and finances. Therefore label it as tax finance and move it in the tax finance holder. Super, super simple utilitarian thing. I vibe coded it.

Ashish Rajan: That's a great use case as well. And so it could be just a problem that's niggling you right now that, hey, I wonder there was a solution for this.

You just make a quick prototype and I guess, and to you, to be fair, I. In a normal world, this would've been a bit complicated. 'cause you need be, you need to be able to talk to Google. API have some kind of a AI model. Yeah, understand it.

Caleb Sima: You gotta get a contractor. Yeah. You gotta get an engineer to go build it.

Yeah.

Ashish Rajan: How long did this vibe coding session last? Outta curiosity? Three days

Caleb Sima: Really, I would say. Yeah.

Ashish Rajan: And you had the whole thing.

Caleb Sima: Yeah. Yeah. Actually I had it working. My, my problem is, I'm unlike any product manager is I had it working within the first three hours I would say, but I just kept adding features to it.

So [00:16:00] fair. And then you like, and then we'll go through this, but you refactor it. There's a whole bunch of stuff that you end up doing as you start building a prod a project,

Ashish Rajan: fair. This is good. So this, I think we have good laid good foundations now, so we have good laid foundations for.

Where to start, what kind of projects now? Maybe we can start with a list if you're keen.

Caleb Sima: Yeah. Yeah. Okay. We've got basically a list. You can find any of these, but there's a list of both tools that I would consider more front end tools like like lovable V zero, although V zero does, they all do backend too, but I just they're extraordinarily good front end.

They make great UIs. And so I, I tend to use Rucode or Claude code for backend, and I make everything APIable and actually the way that I've tended define that I work. And let me give you a little bit of architect. These are some tips of how I structure a project and how I think about it. So before you start doing something, so for example let's [00:17:00] take a web crawler.

I'll just make that as the simple example since we used it. How do you start down the road of making a web crawler? The way that I think about it, first of all is I wanna start in somewhat of the most simplistic way possible. And when everything you do, you wanna start with a V zero.

So what is your MVP? So what do you do first in order to do this? In Ruco Rucode or in OpenAI or in cloud or any just straight up chat agent. Just say, Hey, I want you to be a product manager. In Rucode, there is actually a role that you can pick and you can say product manager and then in the little chat.

So what will happen in Rucode in Visual Studio is you'll get this little window and you'll say Product manager, and then you'll just have a chat box. And then all you do is you put in there and say, okay, I want to write the simplest MVP web crawler that you can have. Give me a product requirements document A PRD for doing this.

And by [00:18:00] the way, I wanna make sure, and the way I always do things is I wanna make it run on the command line. So I want it so that you can run the web crawler on the command line. Why do I do command line? I like command line because it's very simple and it has very definitive inputs and it's a user interface that you can use that later down the road you can just abstract to an API.

And so I like saying, okay, give me a command line version of a web crawler. Give me the PRD of it. Do not do any coding. I always say make it a PRD, no coding. It's the product manager. Just gen it and also generate the user experience of what the user, what are the use cases a user should go through.

And like literally that, just make me a web crawler, MVP product requirements document and what the user experience should be. And it will auto generate a product requirements document alongside with no kidding, like asky [00:19:00] graphics of the way the thing should be the, it'll go through the user experience of running the command line.

What are the use cases that it should, and I would say for 98%, 99%. That's it. It will be very comprehensive and very complete. Wow. And it will generate this PRD and you say, great. I want this to be now my MVP of my web crawler. And so call this m web crawler 0.1 is generally what I start, what I, or one, 1.0 what I start with.

And then what I then say is once I got that product requirements document done, I then in Rucode, there's an architect mode. So I'll switch to architect mode. You can also just enc Claude or whatever. Just say you're an architect. And then I paste in the PRD and I say, based off of this, I want you to break this into dev engineering development sprints.

One feature per sprint and make sure that there is test [00:20:00] cases for each sprint before you call it complete. And keep it simple. Keep it as simple as possible. And then the architect will take the PRD, generate a engineering requirements architecture document and break up each engineering task into sprints.

And they'll call it, like Sprint for, V zero one, V zero two, V zero three, V zero, do four all the way up until it gets enough that it says it's complete to get to your 1.0. And by the way, this is two prompts right now that I'm talking about where you get, and you already have an engineering roadmap.

That's right. And the PRD as well. Correct. I have two documents now. I have a PRD, I've got an engineering roadmap and then that is it. Now I basically save those files. Yeah, these files are done. And then I go, I tell Rucode, I switched engineering mode. I say, okay, now you're in coding mode. Go ahead and reference the product requirements on the [00:21:00] engineering requirements and follow that and build version 0.1.

And then I basically, I personally yolo it. So in every one of these vibe coding, they've got this little window that shows do you want to ask for permission to create files, edit files, all these. And I don't. I turn all of those on. I just say, don't ask my permission for anything. Just go.

And by the way, this works really well, where like Guy was saying, Hey it's good for. Prototypes and V ones like, yeah, you don't ever do this for enterprise. Oh yeah. Code cases. But for, this should be

Ashish Rajan: a warning before this episode is launched as well. Yeah. Things discussed in this episode.

Please do not do this in production.

Caleb Sima: Do not do this in production. But if you're an engineer working in production, then you already know this, so none of this is gonna be new to you. But like I just basically select Yolo it and then it just goes. And I'm no kidding at, usually at these stages [00:22:00] you could just walk away.

I will walk away, go eat lunch or go watch tv. It's like the old days when you used to get your code to compile. Oh yeah. Like you just. Go do whatever you want. Yeah. And you'll come back and that thing will be done and it will go through the sprints one by one, write the test cases.

Although I will be the, to those of you who have been vibe coding before, I will tell you vibe coding and unit test cases can get a little wonky. So you gotta be careful about the way it does tests because it does things like okay, you'll write a test case. And what AI will do. 'cause it cheats if it, for some reason, it can't make a, it'll write a test properly.

And if the test doesn't pass, it'll just rewrite the test. So it just, it passes based on the product. Technically it created the rule so it can break it as well. Exactly. Which is not what it's supposed to do. Which by the way, there is a QA mode that I will switch to in Rucode. I'll create a QA [00:23:00] mode which is dedicated just qa, that it can only write test cases and it cannot modify the main code.

And that is one that I'll use to which is a little bit better tip for doing testing. But this thing will, it'll write it. And so you'll come back and you'll have a fully working web crawler that works, does exactly what the PRD stated. And by the way, when you tell it, update your task. Yeah.

It'll go through that original document and put check marks. On all of the things that it completed on each sprint that it completes. And then what I'll do is then I'll go, okay, I've got a version 1.0. I'll go back and if it works, I'll go back to the product requirements and say, I want to build now a version 2.0.

Here are the kinds of requirements I want in it, and I'll tell it. Here are the, like for example, oh, I want the crawler to be multithreaded. Now I want it to be deployable on AWS I want it to have a vector database so it stores the results [00:24:00] so I can AI chat with any website that I crawl. I wanna be, like, for example I put a feature in L one that says before a crawl is complete, I want to tell it in English language.

The topic or when it sees something complete I'll give you a great example. One of the things that you find all the time is in engineering or development documentation. It's all these webpages and docs, right? Yeah. So sometimes what I'll say is crawl this webpage, but when you see anything that's only crawl the development engineering docs, if you see anything that's not a doc, then call it complete and it will go through every page and then do a comparison to see whether it's an engineering doc or not on how to.

And if it's not, it'll throw it away. And when it thinks it's done, then it'll just stop. Oh I'll add this in my crawling product requirements for version 2.0, it'll do the exact same thing. Build out A-A-P-R-D for web caller 2.0. I go to the architect, I break it into its [00:25:00] sprints, and then I take it the same thing.

I go right back in it and I say, okay, go follow these documents. And it just goes. And by and large, I would say I got to a 2.0 and it took me, yeah, three days to go through a time where you could get all the way to a 2.0. I actually did. A video recording of me doing an incline.

Ashish Rajan: Oh yeah.

Caleb Sima: Yeah. And it took me a total of three hours once I broke it up in

Ashish Rajan: recording. You knew where you were going. So people who are watching or listening to this, if they were to do this I guess maybe on a weekend project, if they want to two, three hours give tops and they should be able to get to a point where they have a V one or V two ready as well.

Caleb Sima: Oh yeah. Oh, totally. Yeah. And then what you'll do is, what you'll find out is as you, as the project starts getting bigger, this is where things start running. So this is why you people say, don't do this. It's great for prototypes, but not for anything else. So as you start vibe coding things like it'll work and it will run.

[00:26:00] I'll give you an example. Once I get the thing running in command line, and it works. So generally, a couple days worth of work you've got a, a pretty small project, but it's definitely not a script anymore, right? Like you've got, I'd say a handful, 30 something different scripts or code files that are there.

And so now when you start doing things, you'll start running out of context, memory, and you have to manage that a little bit, be more specific about what you're doing, and you'll wanna refactor. So now that I've got this as a command line argument, for example, I want to now make this in the cloud and in API.

So I'll tell it. We need to refactor our code. And so same thing, I will say we need to break each refactor into a sprint and test to make sure our functionality works. And this is where things can get hairy which is why in Visual Studio what's nice is you can watch the code being changed. So as vibe coding is happening, you're seeing the code.

Running and being changed and edited. So [00:27:00] you can watch what the AI is doing to it. And if you are any sort of engineer, you can see where sometimes AI will get into what I call rat holes, right? Where it'll get an error or it'll do something like, it'll do something that says oh, we need to write an API to I don't know, pass, like off into a crawler.

And you can see it recycling itself into adding multi inner loops of different auths in the, and you're like, what is it doing? And you have to like, stop it and you have to say, wait a minute, you are adding multiple types of auth on top of itself. This is wrong. And they'll, and of course they'll go, oh, you're right, I'm sorry.

Like I was like, and I have to tell it. Delete all code related auth and start over. And so if you're watching it, this is where things can start getting. Wonky.

Ashish Rajan: I think maybe this is what Guyo mentioned on the recording we did with him as well, that if you only work on a specific task, vibe coding works really well.

[00:28:00] It just does that one particular tiny task. To your point, the prototype that you're looking at works really well, but the moment you start adding features and layers to your prototype, suddenly you're in this world of it can go in any direction and nothing, which is kinda what you're describing with this.

Funny enough at B side SFI was having a conversation with the Anthropic soc team. We did the live panel there and one of the things that was really interesting that was called out by the so person, I think Jackie Bow is the name. Yep. So Jackie was talking about how they're using hallucination as a way to come up with detection rules for like in certain scenarios this.

Rabbit hole, for lack of a better word that it went into, could be good. I think that's where I'm going with this. Where, to your point, maybe not in the authentication context, you don't want the authentication to go wrong, but in, in scenarios where I wonder what other features can I add to make this better?

To, I guess you obviously have a lot of experience, you're able to use that to build on the PRD and engineering roadmap and everything. Oh [00:29:00] yeah.

Caleb Sima: If you don't know what features, you can just ask it and it will tell you all the things that it should add to it. And by the way, again, 98% of the time it's right.

Ashish Rajan: That's what I mean to me. 'Cause I used it to I, you know how a lot of conferences have captured the flag. Yeah. And a lot of the time they're hacky, they're not really the best. They try and make it big enough that they can handle volume. So I went down that rabbit hole, made a cloud security capture the flag.

I didn't end up at the authentication part because there was no authentication for mine, so I didn't end up going as deep as you did. But what was interesting in my little experiment that I did with the AWS pieces it would hit error. It would make up scenarios, which. Looked right, and I would look at it going, that is, that should technically be possible.

There's no policy like that in a Amazon context, right? It sounded like, oh yes, this is definitely it. I'm just gonna apply it. I I went down [00:30:00] the rabbit hole trying to find it, but never found it. Did you have those experiences as well where yeah.

Caleb Sima: It can link libraries where, which do not exist.

I didn't, it does do that to some extent. For example, I'll give you a great example. In my more advanced versions of the crawler, it wrote Terraform to deploy them because then now it used serverless functions. Yep. To do its crawling engines and it wrote a whole bunch of Terraform code to go do the deployments.

And it actually, what really surprised me is it only failed Terraform twice. But it used the, and by the way, just for people to understand also, I don't think people get it. This is not just writing code like Rucode Klein and others isn't just writing code. It compiles it, it runs it, it has access to the command line, like it gets the errors, it gets the feedback.

So it ran Terraform deployed, got the errors, fixed, the errors. Deployed again, got the error fixed, the errors deployed again, and it works. And then it'll run test cases. So then once it deploys, it will run tests to ensure that the actual [00:31:00] instances deployed in AWS work properly. And if it fails, it will see the error fix the Terraform redeploy.

And by the way, all of this is happening within five minutes. So for me, adding Terraform AWS deployment support took me about 10 minutes. Yeah. Like I it did the thing, it wrote, it errored a couple times, fixed it all, deployed it, ran the test, it worked. And by the way, here's what's really interesting.

The permissions on the AWS Terraform were largely least privileged. Oh, really? Yes. It was not wide open PRIs in the Terraform.

Ashish Rajan: Okay. That, which by the way is

Caleb Sima: way better than me on, or any senior engineer by the way on. Yeah. AWS to I would've not

Ashish Rajan: thought

Caleb Sima: it would do

Ashish Rajan: that. Okay. So to which is actually to your point, it's an interesting call out because many people just assume by default it creates security hold.

Yeah. Secure code and security hold by default. But your, in your experience so far, you found the Terraform obviously, [00:32:00] okay. I do want to add a caveat. We did give a credentials and go ballistic as well, so I don't know how many people would do that in their own environment, but in spite of all of that, it still did the right thing in the versions that were created for you in your particular product that you were building.

Prototype that you were building? Yeah.

Caleb Sima: By and large, 98% of all the permissions were least privileged permissions in AWS when it built the Terraform. Wow,

Ashish Rajan: okay.

Caleb Sima: And it, by the way, I didn't have to worry about any of the, it just, it recursively just figured it out, and then deployed it and worked.

Ashish Rajan: I think something worthwhile calling out here.

Because a lot of people may assume this is only for software developers, and I think what you and I just spoke about is not even a software dev, it's a security engineer could do it. A cloud engineer could do it. And anybody.

Caleb Sima: Because you're managing an engineer. Yeah. So what this is now you're managing a quote unquote engineer by just telling it, Hey, don't do it this way.

Do it that way. Yeah. I'll give you a great example. In version two of my web crawler, I converted it [00:33:00] to Rust. Okay. So like you just you just tell it, Hey, convert this and or convert it to go and it'll just do it. Oh my God. And then now you just say, convert it to, to convert it to Rust and make sure all the test cases work.

And it just does, that is just, I mean by, by and large you think I'm oversimplifying it and yes, to some degree I am. But no, I'm not. There's a little bit of oh, you messed up here. Oh, you a little bit. But like maybe two or three times. And the thing that was most amazing to me is I was on the floor of my kids' bedroom.

Yeah. With my laptop and I was playing with my son. Cars and I would just go over to the right when I saw the vibe coding work, and I'd see, and it would, I'd see something, I'd be like, oh no, do it this way. And I'd go back to playing with my time on the left.

Ashish Rajan: You could have it. I guess to your point of this, I think what you're saying that you're managing a person which is just happens.

It has been given a [00:34:00] task, just goes and does its thing. I think it's a good example, but I would also encourage people to not be, putting themselves into a hole that it has to be a software product. It could be anything. You gave the example of Google Drive earlier, obviously giving the example of a capture the flag.

'cause that's what was interesting for me. But people could take it into their cloud environments, their on-premise environments any kind of code. I remember this company that I invested in ages ago, and I remember I. Because of first founding engineers wrote the code in Rust. That's why I laughed at the Rust part.

And we could not find anyone in entire Melbourne to who wrote Rust because everyone had moved on to node.js. Yeah. And this is, this problem would have not existed back then that we had to refactor the entire code to no js just to hire more people.

Caleb Sima: So no, the game and actually I use Rucode and other things just to do interrogation of repos.

So you can download an open source repo and q and a with it like an engineer, right? Or you can [00:35:00] ask it, Hey, generate generate more detailed documentation about how this thing does authentication and it will just go and do it for you. So you can use it like q and a on any repo. You don't actually have to use it to code.

You could just load up your stuff and say, Hey, I want to know how off is done in this repo and that what does the code do here? How does the code interact with serverless or lambdas? And it will tell you it just, you can really do almost, it is an engineer. It is a full-time dedicated engineer for you and quote unquote teams because you can, again, architect modes, product manager modes, debug modes.

Like I'll give you a great example. Many times when you're coding you'll run into bugs. Yeah. And in these bugs it can't figure it out. Like it'll just keep rotating. And this is when. AI code starts getting wonky, like it'll start changing its foundational code to adopt to the error versus fixing the error.

Oh, but [00:36:00] actually, if you change mo, sometimes changing models works. So I have a debug mode in Rucode and in Rucode you can change the setting to change models. So for example, DeepSeek and Grok for some reason I think are really good debuggers. And so I set my debug to Grok model or to. Your DeepSeek model.

And you'll, it will go through and then use that model and look at the code and then help you debug it and say, oh, this is the fix. And then I'll switch it back to code mode. It'll take the fix, implement it, and then run and keep moving. And there's a lot of interesting power in the way you can play with both the models you're using, the settings that you use, the prompts that you use, the configurations that you use.

Yeah. All of these things make big differences around how you.

Ashish Rajan: Now that we have opened the flood gates for being more project manager like, or product manager like, and adding features, how about adding some agents into this as well?

Caleb Sima: How can we, yeah. So yeah, that's a, this is a great transition.

[00:37:00] So let's take the bug and issue problem. Yeah. You wanna know the best way to solve most issues is for your vibe coding session, like Rucode to have access to the internet and search for the issue before trying to fix it. So what you do is you use MCP, and so inside of Rucode or Klein or any of these, there is an MCP registry button.

It literally is like you go, you click on it and it will list you all the top CPS people use, and you find an MCP, like for example, Perplexity. I use Perplexity, MCP, and you just click install. I. And it installs it. And then now your AI coder has access to Perplexity, and then what you do is in your prompt, or you can just do it live in your chat window.

I I tend to put it in my prompt and say, Hey, before you run into fixing a, an issue, I want you to ask Perplexity for the most up-to-date information, and then try to fix the issue. And then what it'll do is, oh, it [00:38:00] runs into an error. It'll then immediately call Perplexity, search for the error, parse the results, take the information, then fix the error.

And so this is way better than it just as the model itself trying to go and figure this out. Oh,

Ashish Rajan: okay. So Stack Overflow is not, it's not going obsolete,

Caleb Sima: I dunno. That's a whole different debate.

Ashish Rajan: No, but I get Right. So we've got the agents, we've got the agent searching for.

The internet to solve the problem. So why MCP? Why not just a Zapier or something or the other?

Caleb Sima: Yeah that's a good question. So you know, there is these how do you vibe code and so what I think, just to recap vibe coding, it's super easy to do. You could go to, by the way, just to note, if you don't wanna download Visual Studio, I highly recommend sort of the model at which I talked about because.

It's, it removes a lot of issues and struggles you might have by building a PRD, building an engineering requirements, doing things in sprints. But let's say you don't want to do any of that. [00:39:00] You can go directly to, lovable V zero. Bolt new without an IDE and just type in something and then just have it go.

But let's say you don't need to vibe code something. Let's just say you want to do something super simple, like you wanna say, oh, what I want to do is any, so anytime someone emails me or let's say, actually this is a real use case I did. If someone adds a calendar an invite to my calendar, take the attendees, take the attendee, look it up in HubSpot, the attendee information, grab the LinkedIn, look up the person's LinkedIn, summarize their information, pasted into your calendar invite.

So before you go into a meeting, you can see immediately who that person is in their background. Wow. Okay. So how would you do that without coding? Actually, I. Any, any of these if Zapier? What's the, there's a bun n eight N. Yep. What's the other ones that that you [00:40:00] do?

Ashish Rajan: I know, I just know the make

Caleb Sima: No, oh, relay.app.

Relay app is good. make.com is another good one. Almost all of these, although I will tell you, Zapier, I will say, used to be, I would say the least configurable out of all of these, but now I think is the most configurable, oh, Zapier is better than relay, better than all of these so far that I found.

They have way more integrations, way more detailed integrations. Like for example, you go to Relay and let's say you wanna integrate with HubSpot Relay may give you eight different ways of integrating with HubSpot versus you go to Zapier and there are like 35 different kinds of. Areas you can use inside of HubSpot.

So and you can write your own as well. You write your own, you write your own flow, so you can then drag and drop and it and zapier's way more than just what people think it is in the old days, which was, oh, do this, then do that, right? [00:41:00] Zapier's entire workflows and agents that will go. So you can then say, oh, take the thing on the calendar.

Anytime a new calendar is invited, grab the participants from that invite. Go ahead and call HubSpot or your CRM, search for that person. Get the results. Pass it to OpenAI or Anthropic. Summarize the results. Edit the existing calendar. Update that calendar, save it done workflow. You can do all of this in almost any one of those apps.

Wow. Okay.

Ashish Rajan: So okay. To, to your point then, you chose MCP because of the kind of tasks that you're looking at. Yep. But people, if, depending on what they're doing, they can naturally go down the path of using zaia with any extension they want. I don't know. Could be like a, Hey, send a Slack message or teams message or whatever because my, I have done coding, my V is ready.

Go find me.

Caleb Sima: Zapier even goes further with agents. Like you can say don't just connect to HubSpot. I want you to do a deep research [00:42:00] using Perplexity on this person. 'cause I want to know about their personal background. So you can then in Zapier, go create an AI research agent. Yeah. And it will go Google or do or call Perplexity using this person's name and any information you have from the calendar invite.

And then say, I only want to research and identify personal information, hobbies, family, something like that. Could be a conversation starter and it will go do the research, find all that information, sum it up, pass it, update the calendar. So like for the everyday person, if you want to use agents, like a lot of these tools will let you do it now without any coding completely.

Just click wow.

Ashish Rajan: And maybe worthwhile saying because. People talk about multi-agent chain multiple agents together and people could obviously make it as complex as they want or as simple as they want as well. In, in the, all the recommendations that we have spoken about [00:43:00] so far, and hey, start here.

Popular tools, things that we enjoyed things we did not enjoy. MCP AI agent. Do people need to I, I guess maybe as they go through this, would you still believe it's not at a point where if you had a few more days, if could make it production ready?

Caleb Sima: You mean? So you're talking about now when we think about vibe coding, let's talk about production level vibe coding.

Ashish Rajan: Yeah, that's what I meant. Yeah. So as in the next level of maturity after this, we've gone through this, spent three days on it. I'm super proud of myself, product manager. I heard the episode, I'm like, went in and did my prototype. And now apart from a few kinks here and there, I think I'm ready to show it to my engineering team or whoever.

But in all the effort you put in what does it take to go from V zero V two to a production ready?

Caleb Sima: Yeah. So let's make a clear distinction between personal fund projects that I'm coding on my own to, oh, I'm a product manager in a company and I want to build a product and then go, [00:44:00] and I. What's the gap between, and just by the way, I just wanna say myPath, empathy to the engineering team who's gonna get in this age, gonna get tons of inbounds from everyone in their company saying, look at my app that I built.

Can we push this on every engineer going, oh my god,

Ashish Rajan: yeah, I playing with my son, that's when I did it. Yeah.

Caleb Sima: And then they're like, can you push this in broad? And engineers being like, just stop. What do you mean it works? Look at all the test cases.

Ashish Rajan: It

Caleb Sima: works. You never gimme a test case. I got 25 test cases here.

But here is what I think is gonna be an interesting challenge. Okay. So let's take this into two pieces. Yeah. First of all. How do you take something vibe coded by a non-engineer and make it engineering ready and production ready? That is what production Ready is very different from company to company.

Every company has a culture of what they deem to be production ready. And thankfully they've got the gates at where people can't, employees can't just randomly bring [00:45:00] things up and then go deploy it. Otherwise it would be an absolute disaster. I think that there's gonna be a lot of people, what is production ready is very different.

I think anything can be production ready. The question is, who is it going to serve? What customers are gonna serve? What product is it integrated to? What are your requirements? There's a lot of this that takes place, which I think it's less relevant to define what is production ready, but more relevant to talk about.

How are engineering teams and enterprises going to somehow manage the influx of people building these apps and saying they want to push them to production? When you have product managers or marketing or sales people building things in a way that help them solve problems, my guess is that the engineering and the enterprise are gonna get enough pressure to say, how do we enable this creativity and the ability for employees to produce applications that solve problems [00:46:00] in a way that is safe and is reliable?

So I think that there is a bigger challenge here, which is how are enterprise gonna manage that? Because the first instinctive response to this is, no, you guys are playing with toys. Which by the way is the right response Today. You are playing what you think is a product may be useful to you, like an Excel macro is useful, but this is not software that we are going to deploy.

However, that is not the right long-term answer. The right long-term answer is how do we embrace the creativity and ability for non-engineers to build these applications, but do it in a way that allows them to do it safely in a reliable way is going to be a really interesting challenge. Because you actually, in an enterprise, want to enable a product manager or a sales person to build a tool or a service that helps them solve the problem that they have on hand, right?

Like when you look at [00:47:00] enterprise applications, like what is all, why do we pay SAP, Salesforce, Oracle, all these mega amounts of money, it's for customized flows that sales and marketing people need to do to write their stuff because they know the problem they need to solve most. So if they start writing their own versions of this, how do you enable the framework that allows them to do this with freedom without getting engineering, having to go through and test and validate it and read it.

That'll be really fun. That'll be a fun problem,

Ashish Rajan: maybe a possible solution. And I this is obviously coming off. There was a si there was a similar challenge in the cloud landscape as well where there was a sprawl of tools, cloud products, people building applications in cloud. And you could had absolute zero visibility into what they were doing.

And maybe this, that could somehow put some light in this point for some people and maybe some people already doing this as well. At least based on the conversation that I had, having a standard paved [00:48:00] road for, Hey, if you are building a prototype. These are things that should comply to, and this is the sandbox environment where you build it in, run it in where you can integrate this in maybe lessons from some of that can apply here as well, where if organizations can come up with a standardized part, and obviously I'm not saying this is the only right answer.

This is something that I saw earlier and seems to work at scale as well. At least it reduces the, for lack of better word bleeding to an extent where you're not bleeding our prototypes left and center. At least, okay, if there's a standard part that the, my AI Governance Council agrees on that should comply with everyone, at least there's a paved path.

People walk down that paved path. You get to go through and go through the different stages to get to production, but if you don't follow the paved path, then you are sandboxed forever. You don't graduate to whatever the next level is.

Caleb Sima: Yeah. It's it is, you are right in the sense that there needs to be some paved path that set. What's interesting is going to [00:49:00] be, this environment is not quite production, right? It's an internal enterprise environment. Or even some product managers are gonna say, you should push my feature into a prod product. But here's the thing is.

The problem isn't the fact that a product manager, let's just take an example. It's not that a product manager can't build with ai, a small utilitarian app that solves a real problem that is able to run in production, right? I firmly believe a PM can do this and do it well. The issue relies on the fact that once he pushes it to, let's say, let's call it internal production, because this is not gonna be something served to customers, but served to quote unquote customers of employees of the company workforce.

Yeah. The workforce who maintains it. The problem has always been hey, when people start using this thing and people have bugs, feature requests, reliability, [00:50:00] then you have, does this product manager now become responsible for maintaining and supporting the small application at which he custom built?

And really that's the major pain is the answer to that is no, he will not be supporting and maintaining that thing. So then who does that responsibility lie to? Then it lies on engineering and engineering is gonna be, yeah. Like literally.

Ashish Rajan: This is actually on the money with the DevOps methodology that everyone's adopted with Agile as well, where whoever makes it is responsible for maintaining it. It's like a plant. You're just watering it. You're making sure it grows and has a healthy life and everything.

I hundred percent support the idea and I think I, maybe people can take that off. And maybe a combination of a pave path, but also if you want a prototype feature to go into production, you are responsible for it moving forward. Every time it goes down, the requires debugging. You leave your position and just go do this.

As long as that is acceptable, you [00:51:00] can go ahead. It would only last as long as a person is in the company, but hey, at least there would be some time. I,

Caleb Sima: Again, you could say at, in this a day and age, you could say AI is the answer to this. 'cause it will both, it, at this stage, ai, is it in a place at where it can manage operations and can it manage and auto update and deal with bug fixes?

If I filed a. A, a support ticket. Can AI automatically take that support ticket, analyze it, update the code, and then repu? It's plausible, right? It's plausible that maintenance and updating of things could be done through ai. Not a backlogs could be cleared by that. Yeah. Backlogs are gonna get cleared.

That, that is, and for these kinds of applications these internal workforce applications, also the definition internally as to when does it become a critical, right? If you as a product manager build this thing, but then all of a sudden 80% of your HR and finance team is now using it in [00:52:00] their workflows, is that now considered?

Critical production service. Yeah. That, it's just, there's a lot of these cultural political role responsibilities that would be interesting because vibe coding something is easy, maintaining and managing it is not. Yeah.

Ashish Rajan: And maybe

Caleb Sima: to

Ashish Rajan: where we started this the motivation for this episode was to encourage more people to do vibe coding because they should understand what's involved in it so that, you can't really protect what you can't see, so at least you need to know how this thing works, even in today's day and age where it may not, it may be V zero for what it is today and maybe become much better tomorrow, but having an understanding of that is essential for people to know what they're up against.

And maybe that brings to the security point as well, in terms of the security leaders listening to this. A, I think we already spoke about the, probably the fire hose of prototypes are gonna come out of from every department wanting people to put [00:53:00] into engineering, coming for security review and maybe asking people to have security people involved with prototyping.

What are some of the other security considerations with vibe coding and I guess maybe now we flip to the CSO hat or to the security leader hat for what would be security things that we should think about beyond just the, Hey, I have a lot of prototypes that have to completely ignore, or unless they manage it, other security thoughts that come to mind as people.

Look at the explosion of vibe coding in their organization or prototyping. Should you consider shadow is, I don't know, there's a shadow prototype thing, but if there was a shadow prototype for, do we give a dedicated environment? Have you thought about that as well?

Caleb Sima: Yeah. So there's obviously in security you have to think about a lot of things, but if you just start from, let's just start from SDLC, starting at left and moving to the right.

Yeah. I actually believe what the security team is probably gonna do first is they're going to create their own security MCP server. Why, what is that going [00:54:00] to do? That means anytime you code or run something in your IDE, there's an MCP server that will automatically analyze the things that you're writing, make calls to ensure that it's giving you the right building blocks.

Here's a great example. Oh, I wanna write this app that will allow an employee to access this field in Salesforce. In your MCP server, you're gonna say any authentication or OAuth that's required to any of these types of services. Call me and I will provide to you the code block that is needed in order for you to do this authentication.

And then the security MCP will deliver these secure code blocks and frameworks automatically while the app is being written so that I. It can automatically give you the right secure frameworks and paved roads to whatever app that you're building. That's probably number one. Number two, obviously it will go through your standard quote unquote static analysis to verify and make sure that there's no other big problems.

[00:55:00] And then I think going, shifting more to the right now, it becomes to this, okay, what am I including in my libraries to ensure that, by the way, which I think in the MCP server, you'll already say these are the libraries that you can include. Everything else you cannot include. You'll go to then deploy.

Deploy is interesting. To me there is definitely going to be this interesting segmentation is my belief between prod and dev. Like right now, if you think of. A normal engineering company, you have your sort of dev environment. You quote unquote, should have a staging environment, although staging is generally a shit show in startups.

And then your prod environment and prod environment is strictly or at least mostly made up of customer facing. Services and dependencies, right? Yeah. However, I think as we get bigger into vibe, coding services being built for the internal [00:56:00] workforce will grow, right? Because people will want to build and customize their own services to solve their own problems, which means that, I wonder if you start having a different environment that is a workforce production environment that is similar to a sense that says, oh, okay, I'm going to split customer prod from workforce prod.

And workforce prod will have its own development pipeline at which then goes through its own set of requirements. So before I can go deploy something that serves my workforce or my internal employee base, it's gonna go through this workforce prod and the security requirements. There will clearly be different than anything that would be required for customer prod environment.

Ashish Rajan: I like the workforce pro analogy, especially for internal facing apps. I was also gonna say in terms of how many companies go down the spot, that would be really interesting to see as well, because as you called out, I. Anyone who goes down [00:57:00] the path of prototyping is probably starting an engineering function in their department, whether it's hr, finance, whoever.

And in a way it's a good thing that every department would have an engineering department as well, which is not a bad thing. And maybe they, it, this creates that space. And we again, have a sprawl of engineers everywhere and we trying to figure out, it could be, it could, because the way I see it at the moment, most people are already say, Hey, my workload is way too much.

I don't have time for this. Whatever. They'll go for a SaaS application or a third party, pay the money and let them solve that. That's what Salesforce ended up doing for a lot at scale.

Caleb Sima: And Yeah. This is already today a common challenge in larger organizations, right?

Yeah. Which is. Do you centralize you know what we call that corporate engineering, right? Which is do you centralize those engineers into a corporate engineering workforce that is run by IT? Yeah. And the sort of corporate workforce, area or [00:58:00] do you let the domains of expertise own them?

So HR will own an engineering HR function. Finance will own a finance engineering function. By and large, I've seen most, most decentralized versions of those fail. Because the problem is the finance team and the HR team have no idea how to hire engineers. No idea what makes a good engineer, no idea how to create an engineering culture, no idea how to follow standard best practices.

And so generally those are centralized as a corporate engineering function under the it, because usually this goes on a workforce area. Yeah. And then, and people listening who work in companies will also debate whether corporate, it understands engineering either well, but that is a different topic for a different discussion.

Yeah. But generally speaking, centralizing that will allow you to enforce the right training, interviews who gets [00:59:00] promoted, who doesn't get promoted, how do you keep leveling the same? A lot of these more corporate policies then have a better way to, to be managed. And then you separate them out and you just dedicate a team for engineering, a team for HR.

Yeah. And that's how it works right now. Yeah.

Ashish Rajan: I think I like where, where we are at towards the end of this episode as well, because we spoke about where people can start today. How can they get to a point where they have a V zero V one, what's the gap between them getting to production and towards the end, we also spoke about the two possible futures this could have in an organization from a change perspective as well.

Hopefully for people who end up doing. Prototyping or vibe coding after this, we would love to hear from you guys in the comment section for what you guys do maybe share your project with us if you're vibe coding one. But otherwise, I hope you enjoy this episode and I look forward to hearing about all the vibe coding you do when we talk to you next. But thanks you so much for your time and talk to you next episode. See you. Thank you so much for listening and watching this episode of AI Cybersecurity Podcast.

If you want to hear more episodes like these or watch them, you can definitely find them on our [01:00:00] YouTube for AI Cybersecurity podcast or also on our website, www.aicybersecuritypodcast.com. And if you are interested in Cloud, which is also assisted podcast called Cloud Security Podcast, where on a weekly basis we talk to cloud security practitioners, leaders who are trying to solve different clients' cloud security challenges at scale across the three most popular cloud wider.

You can find more information about Cloud Security Podcast on www.cloud security podcast or jv. Thank you again for supporting us. I'll see you next time. Peace.

‍