Damian Hasse: [00:00:00] Was there a science

Ashish Rajan: that the first hype cycle would fail before it failed?

Damian Hasse: The difference, I see there was just a few people that could really understand the tech and use it, et cetera, while today that barrier, that level has gone down and literally anybody can use

Caleb Sima: it. There's a universal gel break. I think that just works for everything right now.

And no one's patched it. We can't even make GPT give the same answer twice.

Damian Hasse: AI councils were built quickly. The people there might not necessarily have

Caleb Sima: an ML background there, but then are now releasing and creating laws, rules and ways at which your company or enterprise must use AI without having any understanding of what that means.

My S3 bucket and open to the internet. That's not the right question to ask.

Damian Hasse: I like my performance was, and I will argue it still is a bit of a challenge when we started doing that work. There was no company that had those safeguards. The power of AI is exactly not that, right? There's no perfect solution.

But how do you mitigate it? How do you reduce the risk on those?

Ashish Rajan: If you haven't building a security program on AI, then this is probably a [00:01:00] conversation for you specifically if you are in a world where you have. Ml? Yes. The one before ChatGPT became popular as well as you're building chat bots in the Gen AI space.

In this episode, we had the conversation with the CISO of Move Works, Damian Hasse. In the episode, we spoke about some of the challenges that cybersecurity people face as they build programs or security around ai, how he approaches security in his organization for AI and how he's encouraging his team to start using more AI as well.

And somewhere in the middle, we also got into the whole. I guess this is the fifth wave of ai. I did not know that. So we got into this tiny bit in the middle where we're trying to figure out is this hype cycle that we are in today with ai, is this gonna survive or this is the end of it after another month or maybe two years.

Let me know the comments if you believe the hype will continue, or would this also fade away as all the other previous. AI waved it, and as always, if you are here for a second or third time, I would really appreciate if you take a moment to drop us the review or rating on Apple or Spotify, if that's where you're listening to this.

But if you're watching this on YouTube, on [00:02:00] LinkedIn, definitely give us subscribe and follow. It definitely helps us quite a bit. It takes only a few seconds and I really appreciate your support in helping us grow this channel and be found by more people as well. And if you know someone who's working or wants to know about AI security specifically in that world of ML and Gen AI working together, definitely share this episode with them.

I hope he enjoys episode and I'll talk to you soon. Peace. Hello and welcome to another sort of AI security podcast. What Damian with us today. Hey Damian, thank you for coming on with us. Hello everyone. Thank you for having me. Really appreciate it. Looking forward to the conversation. Damian, just to set things off, just to set the scene, if you don't mind giving us a bit about your professional background.

What are you doing here today and I guess what's your. Professional background, let's get into that.

Damian Hasse: Sure. So I'm currently the CISO at Millworks. I joined over three and a half years ago, literally a year before GPT became A thing. So it's been a lot of fun being able to ride this wave.

One of the core reasons why I joined when I saw demo the product, being able to use ML AI to [00:03:00] actually solve real problems, I was really excited. Before here I was at Amazon for about nine years working on the security privacy for Amazon consumer devices and the services I. I was there when Alex, Alexa Echo, et cetera were created.

So it was a lot of fun seeing that as well as fire tv. When I was there, there were just two devices, the Kindle and the fire tablets. So when I left, I lost track of the number of devices we had, but also worked for a bit on the Ky, which were the satellites that Amazon was putting. Before that I was sure pure VMware also on security.

And before that I was at Microsoft for 12 years doing security before security was a thing. So I joined in 2000 and there was a trust for the computer mill from the gates in 2002, 2003. So anyways, so I've been doing security for quite some time. Awesome. And privacy,

Ashish Rajan: As well. But anyways, I was gonna say you have seen two transitions, I guess it's transition of.

Security emerging and also AI ml before, these days people just say AI willy nilly [00:04:00] and just assume that you're thinking of gen ai. I'm curious as to now that you've spent some time on both sides, the pre G PT era and the GPT era that we are living today. In terms of how you are thinking about ai, how do you describe AI security before that and today outta curiosity?

'cause I don't know how many people are just, do you say AI or do you see gen ai? What do you say?

Damian Hasse: To be fair, I use a little bit of ai, gen, ai, lms, and ml. I mix and match for the most part. I should probably do better, but I just use them. But. The way I see it. Going back to your question, the pre GPT era, I will argue it was more of an explanation and justification to explain people how ML work, how it can add value.

Focusing a lot more on N-L-P-N-L-U, natural language understanding, natural language processing. Which is different from gene ai when it can actually, just generate stuff. You were [00:05:00] just trying to understand in in the world before or if you were doing with video, right? Trying to identify things from a video, which was also, a complex problem to solve. In when I joined Mood Works, one of the key challenges was I would say explaining to people how the technology worked, but for the most part, it's like it was justifying that a bot could work, that there was tech that could solve problems. After that. It was more about education, heavy on how gene AI works.

What are the security. One of the key things people confused was about, hey, if I'm using a large language model, doesn't mean that it's being trained with the data. No. It just depends on what the company's doing with that data. But anyways, so that's for me has been a heavy transition. Also, the one other thing that I will add, a lot of companies today now have AI councils or something to that extent, or the equivalent, some of them I would argue, I think it's good. Others, maybe they're trying to do the right thing, but they still need some education or some [00:06:00] ml expertise to, to help them make the right decisions.

Caleb Sima: So what makes you say that? You sound more careful in your saying of some have counsels.

So I'm not sure if they're good or not, but yeah, like just be direct. Tell Yeah. What's I have

Ashish Rajan: insurance, but I don't really know if it works. Yeah. Yeah.

Caleb Sima: Tell clearly you have run into some issues. Tell me a little bit about what you've seen and why what are the sort of mistakes that are happening there?

Damian Hasse: So from my perspective, a lot of companies like, okay, yeah, we need to embrace ai, but do they really understand what the technology is? Do they understand what's happening with the data? I will say some of the concepts are similar to what we have before, whether it's access control who has the data, what are we doing with the data, access, et cetera.

But at a made a point, this AI councils were built quickly. The people there. Might not necessarily have an ML background. They're trying to learn. So that's why I'm saying they might not, they're trying [00:07:00] to, so I give them credit for that. But sometimes the questions being asked it's okay, let me just take a step back and walk.

You explain to you how the technology works, right? So going to the basics. One example is people think all ml. Is generative ai when in reality is there's deterministic. The way you do search when you need to rank content is not necessarily a generative model. It's just trying to stack, rank, give you scores like, okay, is this the right thing that matches your criteria just to be able to serve back?

Google has been doing that for over a decade. Just there's a thing called

Caleb Sima: machine learning before there was ai.

Damian Hasse: Exactly. But is it the same though, Caleb? Tell me. So Caleb, to be fair, I took an AI class back in the late nineties and I used Lisp. To write a program that ended up solving a puzzle, and that was called AI back in the day.

Caleb Sima: It was yeah. AI has had, [00:08:00] its sort of peaks and valleys right in the history of ai. First, was it two? Maybe there's two peaks, but I feel like I know for sure there's one peak of AI is gonna change the world. And it ended up going nowhere. And so the sort of, it collapsed. And then we are now in our second iteration of AI gonna change everything with LLMs specifically.

Being the new, and then it may have, its collapsed too. 'cause actually I think was, Satya just came out recently this past week saying things around how AI is not as big as what people think it's going to do. And that he, and there is some quote and some sort of like output now of him saying, Hey, AI is not the change at which of which we had expected.

Some people are saying this is just the fight between now OpenAI and Microsoft. But there has been some now. Executive sponsorship saying, oh, maybe this is not all the hype at which we think it's gonna be, ah, [00:09:00] interesting.

Damian Hasse: What you access. Caleb, if you don't mind, what do you think?

I'll tell you my take after. I'm just wondering what do you think? I like

Caleb Sima: Damon, how you just set me up to me to go and say what I think so that you can.

Are you like sacrificial lamb? Caleb, you go first. You go first then I'll share. Okay. Okay, fine. But I think you know, maybe Damon, what you are also saying is I've also seen this similar where, hey, there is obviously this rush to adopt AI every CEO and every board member is saying, you must do this.

Similar to cloud, similar to every other sort of, I think big innovation. And then these AI councils got put together. Consisting of people who have no knowledge, background or technical capability to understand ai, but then are now releasing and creating laws, rules and ways at which your company or enterprise must use AI without having any understanding of what that means.[00:10:00]

And with someone who's more technical, who've maybe had some influence in both machine learning, historics. Capability and Hey, you don't need these safety regulations for an algorithmic model that we've been using for 30 years. We don't need to apply that here. Or, Hey, you're hoping these great things in AI are gonna accomplish these things, but that's not how it works.

That's not how AI works. And you need to have, use cases for these things. Not just say, use it in some specific, is that what you're saying? Is that a little bit around that.

Damian Hasse: Yeah I agree. My question was more about your take on what the value on whether it's gonna drop again.

Oh, okay. Yeah, but I think the way you phrase it, thank you made much better than what I did. I agree. I will. The argument that I have, the AI Councils are trying to do the right thing, but they are learning as they're building. Those capabilities, right? If I look holistic over the last two years, give or take, in the beginning it was all over the place.

People don't really [00:11:00] understanding how LLMs work, what it could do, what it couldn't. Anyway, it's just basics. Today, I'll say. People have level up, which makes it easier to explain. Some of the questions they're asking are legitimate which is good, right? We're having more interesting conversations.

Doesn't mean that happens all the time. We have improved. I look at internally our documentation to better explain, highlight the basics. So then we can have more interesting conversations. And usually when people understand, if we go back into some of the specifics, how do you handle hallucinations?

How do you handle prompt injection? Prompt leakage, right? Going into the access control aspect, it's oh, okay, now I understand what what you're doing, how the product works, how you mitigate those, right? There's no. Perfect solution around some of the stuff, right? But how do you mitigate it?

How do you reduce the risk on those on those areas? But I agree a much

Caleb Sima: more optimistic way than I put it on.

Ashish Rajan: But we, what I'm seeing is because [00:12:00] there's also no. I guess there's no straight answer to a lot of questions as well in AI. I get the very foundational piece of if you have a legal team, if you have a marketing team, if you have sales team, product team, tech team the council is a mix of different people with different agendas all trying to either use it, but then at the same time, we can't even make GPT give the same answer twice.

And it's how, where, because considering we don't even are, we are not have that stable foundation for AI. I think that's where a lot of the councils were struggling, where I've heard this a lot of times where if you have 35 people in the room, there's no one decision coming out of it. For sure.

Like every, there would always be a divide in voting system or whatever. At least that's where, that's how I saw it. And that's maybe my, my viewpoint on that challenge was that, oh, because a HE point, what Damian said, there's not enough knowledge in it. But then to what, Caleb, you're saying we are not looking at use cases.

We are just going, we just need to use AI somehow. Just fix it. Use it. Just do something with it a thing.

Caleb Sima: So if I [00:13:00] were to, if I were to actually merge those together, this goes back to maybe the Satya comment and Damian's, Hey Caleb, you answered this. I want to hear your view. I'm gonna merge all these together and I'm gonna say that I think that everyone is correct.

And here's what I mean by that, which is obviously in any of these things you're gonna get way, you're, you have the hype cycle, right? Yeah. You're gonna get way over hyped. On what this thing can accomplish than what it legitimately probably ends up bringing. And the way you look at any of these is, look at any of these cycles.

Look at cloud. When you look at cloud, did it change everything in our industry drastically? Absolutely. It did in amazing ways. Did it do it in the first two years? No, of course not. Everybody was lifting and shifting into cloud, right? They were copying, what they knew of the, of the world then and trying to replicate that world in this cloud and we didn't really recognize the true [00:14:00] transformational benefits of cloud. Until, what, seven, eight years plus later? Yeah. When people started really building with, re-architecting and building with the mindset of cloud in a different thought and strategy. And I think similar, like with AI, we have super high hopes. I. But it doesn't happen all in the first two years of what happens.

And we're trying, and I think we're also doing this lifting and shift, what, how do we think about what we do today and replicate it in a machine? And so there's a lot of this same sort of copy and there's this expectation level that says, oh, this is what it's gonna be like when in reality that's not gonna be like, and so when do we start seeing true transformations in this is five, six years down the road where we have adapted ourselves to think about it in the new way. Like for example, Ashish, we know that AI is fundamentally, different every time, right? Yeah. Like we get different results. And so we are looking at that today in this mindset [00:15:00] of weakness because we look at things pragmatically, it needs to be like code, and therefore it should be.

I can consistently get consistency out of this when in reality the power of AI is exactly not that, right? It is the power of not being consistent. It is the power of human reasoning and thinking, quote unquote to some extent. And so like I think that, oh, in five or six years, maybe we accept it for what it is and maybe learn how to empower through that and use it.

In those manners. And so do we have transformational change in it? Yeah. There's no question. Like I think it's a given, it's going to come, but I also think we're struggling and trying to outfit the way we think it should be versus the way it is. And as soon as we start making that shift, like similar to cloud, we start.

Rethinking the way we think and the way that it works best. I think you'll start seeing much more great shifts that occur from it. But again, going back to [00:16:00] this, AI council, I think we had high expectations and we're just saying we need to use AI with not really knowing what for or how.

Yeah. Yeah. And that creates a lot of problems, a lot of hype, a lot of overspend, a lot of used resources, hands, no

Damian Hasse: decision made as well. Okay. One thing I can I wanna add Caleb to what you're saying. Yeah. The distinction between cloud, cloud focus on, I would say, specific aspects of the business who could actually do that transformation, who can make that happen?

When I look at GPT Gene AI today, the realities anybody can use it can benefit, right? I personally have been pushing my team to, on each of the areas to focus, what can we do? I. How can we change how we are thinking? How can we look at problems differently, talking to other people in the industry, some startups as well, right?

That, how can we make things better? There's a lot of, I would say, inefficiencies. I look at what is the mundane tasks that we as humans, [00:17:00] quote unquote, are doing today that we don't wanna do anymore, right? Yeah. How can we be more effective? And from that perspective, that's how I see it.

Okay. Let's push the envelope, right? Some stuff will work. Some stuff, maybe the tech is not there today. We just need to wait a bit longer, right? But I see the benefit, the value. I personally use it on a day-to-day basis just to help me. Very simple. Sometimes just summarizing stuff.

I need to just help. This is the idea that I have. Help me explain it better, right? This is the audience. How can I present the content better? Actually very simple.

Caleb Sima: A great question for you, Damian, is when you think about how much you use AI personally, right? Versus how much do you see it being used in your team or at your company that you've had to deal with, how do you what, where do you see that, those meters at and what are good examples of both?

Damian Hasse: I think it, I'll be honest with you here, since we have dedicated ML teams, [00:18:00] right? I will say experts there we use it almost in any way we can. And for me, I also, I'll give you one maybe a specific example. One of the key things that we need to be mindful as we are adopting AI in general, right?

Is, okay, let's go explore test. Does it add value, et cetera. Once you see some value, it's okay, how can we measure that? What is the true ROI. If I look at coding is a good example. Without going into some of the specifics, we ended up collecting stats to just have an understanding how useful are this coding gen AI tools.

That's it really add value, kind of some of the data that we gather. It seems like for more junior developers it added value. For more seniors, not as much. Which I'm like, okay, how would that change in the not so distant future? I think it will add value to more senior developers down the line.

But I can, at least for now, it's just [00:19:00] helping level up those more junior developers. And if I look at personally within my team being able to complete questionnaires, total security questionnaires that, that we all receive and. Not really. It's yeah, gene AI helps a lot.

We actually build those capabilities before gene ai, which wasn't as good, right? Because of some of the responses that, that just the language, the output. But with GenAI it gives you that, what you described a bit of that logic, so the response makes more sense in the context of the question, right?

Obviously. Use the knowledge that we have. Also we've been using it in the security operations aspects. In this case, not so much build in-house, but partnering with startups in the space to help us. So just trying to push the envelope where it makes sense and test things out and prototype POCs, et cetera okay.

And then let's explore where we can go from there. But I think it's worth testing things out and see what works.

Ashish Rajan: Maybe what [00:20:00] doesn't I thought you might say you have an MCP server internally, but thank god you didn't say that.

Caleb Sima: But this is a good question. I was gonna ask a little bit of, like one thing that is clear where AI is making significant.

Impact is coding, right? Is in engineering. Whether it's at a junior level, Damian, to what you're saying versus senior level, that's just a matter of maturity. And where these things go. But even, let's just assume AI models are at their peak right now and don't go any further. The amount of value it's added just in engineering at a junior intermediate level is tremendous.

I agree. Yeah. Yeah. And so there's a lot of discussion around this on. Dam, like how does this change security? Is do you have any thoughts around from an engineering perspective, does this or does this not change the way you think about securing AppSec or prod sec? Or is it just the same stuff but a little bit different?

Like how do you, how are you thinking about it?

Damian Hasse: I look at it, there's some similarities, so I look at it [00:21:00] as a data. Type of that problem, the developing case, right? It's an IP issue, right? So it's my IP, right? The work that we do is not open source. So it's my own IP. So one of the questions that I have based on that, number one, are they using whatever technology we're we are using right to generate code?

Are they using the data for training? In essence, okay, what can happen if my IP is used to train the model and then somebody else ends it up with a quote unquote similar idea. Okay, yes or no. Good to know. Okay. The other aspect is if they need to do any type of debugging or issue that we run into, et cetera.

Okay. What access do they have? Once again, to my data, in this case, my source code if it's a SaaS solution, they're siphoning my code into their into their own quote unquote cloud, et cetera. I'm like, okay, good. Who has access? Thinking about from, once again, a data [00:22:00] problem perspective, right?

Who has access on their side? What are the right security controls? Do, can I ingest the content back into my sim blocks to just look if there's anything suspicious? The other aspect specific to source code, which is also, if I go back to the training, which I think is specific to ai, is how much would I trust this code?

Could this code have malicious intent? Maybe it was poisoning poison, et cetera, right? And it leads to quote unquote bad code back doors, et cetera. Or could it introduce vulnerability, right? I'm like, okay. You can argue how is that risk any different than if you're using if a junior developer, you had a bad actor an insider threat within your organization.

Could you put similar controls in place? I look at the main distinction is the speed I. You might have one insider risk, assuming you have one employee here you have X number of developers, so x number of the exposures [00:23:00] that you might have, how do you truly scale that up? How do you look into those aspects.

So that's how I've been looking at it. And also as I'm thinking about the safeguards, they need to be a little bit different just than the what. The old work looked like better, I'll say. Being able to scale

Caleb Sima: better. One thing that I've worried about that I think is becoming more of a reality is prompt injection problems.

I. Right? Like the one thing that I feel at least, I'm gonna claim this, I won't claim this worldwide about, let's just claim this in tech companies. 80% made up percentage number, by the way, of probably tech companies are using some sort of cursor, so called aI engineering tool.

MCP servers are becoming way more popular in, in all of these usages as well. And the fact now that there's so much open source being consumed in these things and these [00:24:00] AI engines are, we're tossing code into lms, they are gonna be able to, we're gonna start putting prompt injections in this stuff.

Right where we know if I'm an attacker, I'm just saying if I'm gonna, if I'm gonna wear the black hat and I want to target and I go and I can put in malicious code, I can actually now put in malicious code in prompts. That effectively know that they'll run on an engineer's machine and it will go and execute these things.

Sure. And there's nothing you could do about it. And there's nothing that detects this stuff in source code that Cursor is running on. And so this is a whole area that I feel is just this black hole of blindness. It feels like.

Damian Hasse: Can I add two things? Caleb, you touch In my mind, I like to add a distinction to me.

What you just described is what I, the terminology I use is the in indirect prompt injection attack. I think that's how you are what you're talking about, which is basically there is, but using an open source [00:25:00] package, my actually have a prompt in there, right? And I don't know, how it gets consumed, et cetera.

And what the, the LLM might be doing compared to me as a developer trying to using Cursor Live and asking just a direct quote, unquote, from Injection It. I agree with you. That's an very interesting area and hard to solve. What I saw recently, what I read quite honestly, was a very long paper that Anthropic put together to be able to look inside the model and understand how the decisions are being done and how it works on a jailbreak scenario, as well as a bunch of other scenarios.

This was a paper from March of this year, give or take. And I think it was great. We're still on the earlier on the beginnings of that, but potentially if we get that level of visibility, we might be able to make models better and maybe crossing fingers, maybe [00:26:00] prevent this in a more systemic manner if we understand how the models are operating under the covers, right as they're making those.

Those decision. Once again, it's just reflecting on my lessons from the paper that Anthropic book together recently. But I agree with you about the challenge that we

Ashish Rajan: have with what I say in direct common though. Because I guess to, or maybe Caleb, you've said this in a way just before, are we looking at doing security in this new world with the old way that we know?

And is it, I guess I'll give, I'll use an example. I recently, I was talking to one of the AI security companies and they work in the cloud space and I was just asking the question around, oh, so am I asking the question, Hey, is my S3 bucket, I dunno, open to the internet. And then it is and again, kind like yourself, Damian, he came from the ML world, then now he's into the GPT world.

And he's that's not the right question to ask. I'm like, what do you mean that's the question that I'm concerned about. It's no, the question to ask is my, is something wrong with my S3 bucket? And then let it come back to you with Hey by, 'cause it can come back with things that you would not even think [00:27:00] about.

And just because one thing's on the internet just to say, Hey, S3 bucket open to the internet is probably the worst thing that can happen. You're forgetting all the other nine to nine scenarios that could potentially happen. That it will, it would know of, but you just ask the wrong question. Just got you.

Got the answer you want. But it was like you going with the thing that you have been taught and trained to do and I'm like that to and bring bringing it back over here. And I guess there are obviously a lot of unanswered questions in the AI space today.

Damian Hasse: Really like what you just said and that's something that was a bit of a lesson for myself and I can't recall when I started to put things together. But one of the things as I been using a Gen AI is literally, it's what I haven't asked, what am I missing, right? As I'm going back and forth brainstorming an idea or trying to put stuff together, I completely agree with you.

And then you go back and then read what it spits out. More often than not, to be fair, I'm like, okay, yeah, you didn't add much value, but every now and then I get some nugget. I'm like, oh, I missed that. Good. Okay, so I [00:28:00] agree with you. If you just ask that specific question versus. What else should I be looking?

What are my highest risk? Whatever else. I agree with you. Yeah.

Caleb Sima: What am I missing? There's a great, it's a great add to your prompt, right? Yeah, that is a great one.

Ashish Rajan: Yeah. I was gonna say, considering you guys, both of you touched on this earlier, that you guys saw the earlier version of the earlier hype cycle of ai and now you're in the second or third cycle of ai.

I did not see the first one. I'm curious, was there signs that the first hype cycle would fail before it failed? I. Or was it similar to what it feels today where every company out there is Hey, I don't really care about use cases. Just start using ai.

Damian Hasse: I probably be more polite than Caleb building a path.

The example,

Ashish Rajan: only adults listening into this. It's probably failed anyways. We know it. Like we know what the ending is, so it's okay to talk about this,

Damian Hasse: but the difference I see is. Who could really use AI before and for what purposes and what was [00:29:00] it trying to do? From my experience when I interacted with bots in general in the past it was really hard to get a useful type of response.

It was just not there. And also it was, there was just a few people that could really understand the tech and use it, et cetera, while today that barrier. That level has gone down and literally anybody can use it. My daughter was using it. It's okay, she's 14, by the way, so my daughter was using it.

I'm like, okay, that's that, that great. She understands. I don't think she understands all of the issues necessarily but I see that as. The big distinction among the two the two worlds. That's my observation whe whether why that was a failure or not, I don't know, but just my

Caleb Sima: interpretation and I think I was too young for that first phase actually, of it.

I don't know when that happened, but I believe it was, I feel like it was like in the eighties or [00:30:00] 90, or eighties, maybe in the eighties or something like that. Wow. The first sort of AI hype, wasn't it like it was a while ago.

Damian Hasse: Okay. I was thinking like in the last quote, unquote decade, give or take, right?

Oh, yeah. Hold on, let me,

Caleb Sima: you're thinking even before Ask ai. Yeah, ask ai when was the last hype cycle? When was the hype cycles of AI in our history.

Ashish Rajan: But I, while he was doing that, Damian, I agree with you, and we were having this conversation about it's this was the whole vibe coding conversation on how many people were rejecting the idea of vibe coding.

And they said, Hey, that's not the right way or what, or, but whatever. But everyone, it, it could, to me it was a reminder of that whole Apple Blackberry moment where. Going back to the cloud example as well, the reason cloud was slow in its adoption where it took decades because it was only accessible to enterprise, or you need to be able to pay that bill to be able to afford something, putting something in cloud.

But as these days there's a free [00:31:00] subscription, $20 per month subscription, like your daughter has it, everyone else's kids have it. My parents have it. They all, it's even if it's for generating funny images. It's available and accessible to every layer of the society, which is where it, it feels a lot more that it's gonna be here for a long time.

And whether what the stable state becomes is a question mark at this point in time, but. The way, at least it's, that's where it feels a lot more real to me. Where it's I know before this I have to explain to people, cybersecurity is, oh, lemme find it. I gotta

Caleb Sima: read this to you then. Oh yeah.

Go for it. Go for it. Learn. Here's our history lesson.

Ashish Rajan: Oh,

Caleb Sima: Here we go. The first wave, the birth of AI was 1950s through the 1970s. Researchers made bold predictions claiming in 1965 that machines will be capable within 20 years of doing any work a man can do, which is clearly film sound very similar.

The first AI winter came 1974 through [00:32:00] 1980. Criticize AI's failure dramatic funding cuts of AI did not reach the hype second wave 1980s. It's called the Expert Systems Boom. When AI resurged because of expertise in very narrow domains, companies rushed to adopt AI with expert systems market reaching over a billion by 1985.

Second AI winter, 1987 through 1993. Oh wow. High maintenance costs, brittleness of rule-based systems. Collapse of lisp. Damian, collapse of lisp. I use it. The AI winter and then the third wave 1990s through the 2000 tens. When machine learning.

Damian Hasse: Can I interject? Real what? To be fair, when I use list for those of, you're not familiar, it's a recursive programming language and the problem that it has that at some point it will, the machine won't be able to handle anymore.

So I wasn't, I'm not [00:33:00] surprised that it's just the reality. Recursion, go ahead. Yeah,

Caleb Sima: and this is the third wave. I think this is the wave that both Damian and I. Experience, which is the machine learning renaissance. So this was 1990s through 2010. Okay. Statistical approaches in machine learning.

IBM's deep blue. Yep. Vector machines and new algorithms and practical applications and di data mining and web research. And then now we are in the current wave, which is 2000 and tens to present, which is the deep learning revolution. This is Alex net DeepMind alpha Go, and then the generative AI boom in 2022.

Being obviously our LLMs. Obviously the breakthroughs, hype, disappointment, winter progress. Next breakthrough, repeat, wash, rinse and repeat.

Ashish Rajan: Wait so the one that you guys experienced, so what, so is that the, is that. Your, was that your experience as well, Caleb, that it was just basically there were some specialized people who had knowledge about how to use it, [00:34:00] and that's why, you is, was it called the expertise?

Is that what you call it? Yeah

Caleb Sima: I got deep into this when I was at Databricks and even before Databricks actually. And this was more, I started more experimenting around it because of the vision models and NLP models. That kind of came out, right? NLP was the understanding of Symantec, reason, context around what was going on in sort of the language and obviously the vision models because of neural nets and deep learning.

This was the breakthroughs during that timeframe was, oh, I could get train visual data sets and then create something outta that. For example, I, before LLMs existed. I created my own model that detected someone taking a photo of a screen, right? Oh. So I basically, and no, this is actually a really funny story.

There, there was a task around, Hey, if we have a secure iPad device, how do you prevent someone from taking [00:35:00] a screenshot? Taking out their photo or their camera and then just snapping a photo. And so I said this was a specialized device that was meant for very high security things.

So I created a vision model by taking, get this all of the photos I could of people taking selfies of themselves, taking a camera, taking a photo. And I created a neural net, a deep learning net that basically was a model that could detect when people were taking a photo. And I use that as a way of being able to say, yes, you can detect people by having the selfie, front facing camera turned on.

You can then detect when people are trying to snap photos of that screen. This was neural net. This was deep learning. This was the era of taking sort of trained data sets, creating a model out of it and doing something useful. And you could do this as a single person. Which was really the height, the [00:36:00] breakthrough at least what, seven, eight years, 10 years ago?

Yeah. This was the thing that you could go do.

Damian Hasse: What was the compute for doing that? From my experience, the compute becomes expensive depending

Caleb Sima: on the, no not all. I was doing something really simple. I was just taking, I basically took the equivalent of. 150, 200 photos of selfies that, and I trained on it, which was not a lot of compute.

I see. And the model and inference was not that big of a deal, as long as you got access to the stream. I used the model and it was able to detect when. People would put a camera and Oh, interesting. Basically take a photo, either a normal camera like this or a cell phone, camera and pho.

Ashish Rajan: Yeah.

And it

Caleb Sima: wasn't, and that was really the cutting edge at the time, was being able to build this so no rag intense flow, right?

Ashish Rajan: No rag, [00:37:00] no MCP, none of that.

Caleb Sima: No. None of this. Yeah. This was machine learning, right? This was building a neural network. Using TensorFlow, having a trained data set, building this algorithm and then doing it.

And this was before, this is obviously pre LLM, and this was you could you, a single person could build a notebook and go and do this. Yeah. And it wasn't that hard. But it was

Ashish Rajan: not failure because to what Damian said, because there were specialized set of people who could basically work with it and make something useful out of it.

And probably to, to what you said, you had the mission dedicated some time. To do, to go and find a model, find selfies, what would work and all of that. Yeah it had to be specialized people. It wasn't available to say the general workforce.

Caleb Sima: No. You had to be, you had to take time, you had to learn about it.

You had to dig deep into it. It wasn't accessible. Yeah. In the sense that you had to go and figure out, oh, how to use Jupyter Notebooks, how to get the data, how to use [00:38:00] TensorFlow. You know what's the right models to use? Like you had to go and do all of this, and then even once you built a model, you gotta find ways of putting it in production, right?

Like all of this stuff was required. Both a lot of, specialized knowledge, experience. You still had to be an engineer to know how to connect the model to the actual web app or the mobile app itself. Like all of this stuff took a lot of time and effort to get together. Interesting. Yeah.

Damian Hasse: The one thing for me to add, the way you described what I was focusing on computer earlier. Was from my prior experience and I didn't go in the direction you have been Caleb, but it, when I was talking to you the engineers ML experts when I was at Amazon back then it was just the compute from needed to continue fine tune and improve the model.

Yeah. Yeah. And that's where things can get expensive.

Caleb Sima: And also, if you're working on mobile devices. I think AI models and computer are very expensive, right? 'cause you got battery, [00:39:00] limited battery life, right? Limited compute capability. But this was the state of the art machine back then. Yes.

Technically, literally in 2020, right? That was the state of the

Ashish Rajan: art. But this is still there though. It's not gone away. 'cause to Damian, your point, and I guess what everyone. People have been working as data analyst ML people for some time and companies have been doing this. And maybe bringing it back to the topic then the whole security part of it.

Now that we've seen both sides, the ML side and the Gen AI GPT side I. What does security look like today? And I guess you answered the question about how do you see security in the world where, from the workforce perspective, whether it's the coding aspect of it, because I guess a lot of people who are watching are listening to this would also be in enterprise that have I.

All the best softwares for security in the world that they've paid for. Firewalls, proxies, your edge source server. I'm gonna keep going, throwing acronyms off this, but in terms of [00:40:00] where the focus should be, I like the approach you had for, hey, your IP challenge with the code.

If I were to take a step back for people who are listening and thinking about, Hey, what do I add to my security program for this year if I'm taking AI seriously is it. My focus on MCP side rag side. Where do you what's your advice to other people who are trying to build a security program around AI today?

And it could be gen ai, or it could be ML plus Gen ai because concept use and done both sides. You can take whichever direction you want.

Damian Hasse: I'll give you my answer with a bit of an example on what I have done a bit of here. Right when we started to adopt gene ai, I had to take a step back and understand the, how are we really using the technology?

What are the risks that come associated with that technology and based on those risks. How can I mitigate those risks, right? It, and when we think about, to the earlier example around coding, it's okay, what [00:41:00] are, what solution am I using? What are the risks within that solution? If you're potentially contributing to open source versus just closed source is probably, the risks are a little bit different, right? If you're just consuming open source libraries as if nothing, I think you have potential issues in general, whether it's gene AI or not, right? But I think the risks are still there. If you're going all gen AI and just throwing security out the window, I don't think that's just the right approach.

But I just go back as okay, how do I look at myself? I need to enable the organization to do the best work they can on each of the functions that they have, right? And within that context, how can I understand the risks? How can I minimize, mitigate those risks as much as possible, but enabling them to be as.

Effective, as sufficient as they can as they can be as productive as they can be. So is it on MCP do you really need to focus on CPS setting aside that it has its [00:42:00] own set of issues around authorization, authentication, et cetera. But okay. How do you bring it in house? Where, what's the purpose?

I'm I just gonna use it, but publicly available data. Is it private data? Is it legal data? It is a data that is under regulation if you're a financial or a healthcare. So from my point of view, it's really taking a step back and understanding where am I trying to use it? What is the data?

What would this ML be doing with this data. And then based on that, figuring out, okay, what are the right mitigations? Do you need to invest in the space? Yes. Do you need to get familiar with it? Yes. Do you need to have quote unquote AI experts dedicated? Maybe not, but everyone on your team should be familiar and it shouldn't be a checkbox.

So that's just a bit of my framework. I'm purposely not saying one core area just because it could be used in so many different places, and you really need to raise the bar and focus on the core

Ashish Rajan: risk. That's my take. And I guess to your [00:43:00] point, it comes to the use case as well. If you are just looking to apply AI versus, hey, this is a use case, you're able to at least dig down for, what are, I guess the gates or security gates that you wanna put, or verification you wanna put at that point in time as well?

I was gonna say in terms of, I guess the the chat world, 'cause I think most of the use cases today in most organizations, the gen AI use cases are primarily around the whole chatbot space. And in terms of, I, I think when we were talking about this earlier we spoke about the whole in terms of approaching how this security around just specifically chatbot space versus like your LLM space versus a lot more nuance to it.

Is your approach similar in terms of just say focus on access control? I think we talked there is, there's obviously a lot to set about deeper use cases, specific use cases, but I'm curious in terms of. The way you approach security for all these new models coming out, or there's like a new model coming out every week, it feels at least to me [00:44:00] there's new MCP server being released every week.

Everyone has a new MCP client. TB server is the right way to approach, at least the way you approach it without feeling overwhelmed about this. Is what are some of the components that are important for you that people should consider in their program out outside? The hype of the new model came out.

This came out, that came out from a security perspective. What do you see are the core components of AI security that people should care about? If I

Damian Hasse: look in the context, if I go back in time, building upon the prior example, one of the core things that we did when we started to understand what the capabilities of gene ai, how can we deploy that properly within the enterprise, right?

How could our customer trust that tech was doing the right thing? We ended up looking at the risks associated with that and then focusing on, okay, what are the mitigations that we have based on those core risks? So what we did is we ended up building an online assessment that [00:45:00] uses. LLMs to aim to identify malicious inputs, right?

Based on whatever somebody's chatting there. Okay let's just aim to identify they're trying to do something wrong, right? By the way, some of them, something wrong could be something toxic, which shouldn't be appropriate for the workplace. Other stuff, maybe prompt leakage, right? Or a prompt injection, or they're trying to do that, right?

We put that upfront as part of our core work that we are doing as the input in parallel, right? As that thing is running obviously part of the core functionality, trying to answer that question. If we identify there is anything malicious. We stop that right then and there. And then also we ended up checking once the final output is produced, we this is, you can argue it's simple, but if you look at the output, if you don't look at anything else, but you look at the input and you just look at the output, I.

Those that do those two make sense? So if you don't have the context right, do those two make sense? And if they don't, then there's [00:46:00] probably something funky that happened right then and there. And then we also had, going back to your other comment that you just made about there's new LMS being produced on a weekly basis.

What do you do there? How do you cannot keep up? We have an offline set of tests that we run. Aim to identify what a, we call for simplicity's sake regression tests. Okay. We know that this will cause prompt injection, we know this will cause some toxic output, whether it's encoded, et cetera, whatever else.

So that runs on a regular basis to aim, to identify anything that could be done as we potentially ended up bringing a new LLM into into production just as a safeguard. So we have those two checks. When we started doing that work, there was no company that had those safeguards. It was for me, we took a step back as a team.

It's okay, how do we enable this? How can we make this happen? And that's the approach that we ended up coming out today. There is startups that are building or established companies that are [00:47:00] building safeguards around that that area.

Ashish Rajan: Outta curiosity, what were some of the challenges you faced as you were building?

'cause considering there was no startups probably building it in-house. No no vibe coding that could help you at that point in time, unfortunately. So what was I'm curious about some of the challenges you faced that people could probably prepare for beforehand. I guess

Damian Hasse: one of the main challenges that we had, it's, so we already covered one of the, one of them was the outputs could be different, right?

Yeah. So it's okay, fine. Setting that aside, the other bigger challenge that we run into is performance. You cannot wait. If you're interacting with a chat bot, you expect a response right away. You don't wanna wait five minutes. You don't wanna make your coffee. Thinking about other issues in in high tech that we have run into, say, okay, I'm gonna go grab a cup of coffee, come back for the output.

You can't. So how can we make it performing enough? And from there we ended up looking at a specific LLMs that were fine tuned for security, right? That we could leverage or. [00:48:00] And then fine tuning LLMs based on some of the outputs that that we have the inputs and outputs, et cetera, that we have to ensure that they could identify that, but performance was, and I will argue it still is a bit of a bit of a challenge.

And the other one is. Even though I think it may be my, I might end up getting my own words on this one, but I'll just say it as well, is even though I think I see a bit of less of quote unquote jailbreaks happening compared to about a year ago. A year ago, I felt that there was like boom.

New, new ones coming up very regularly. There's still. Cool new type of attacks that are that are happening. And some of them are very complicated. And that for me is like, how do we keep up?

Fortunately, the bulk of this work, it's happening out in the open, so it's easy for us to pick that up, build those safeguards, and then give it back. [00:49:00] I just remember the one that I was talking about, I can't remember who made the paper, but it was about the poem that it ended up spilling out.

Caleb Sima: Yeah. There's a universal gelb brick I think that just works for everything right now, and no one's patched it. So yeah, I, I think that's the case. Although to your point, I'm also not up to date on the jailbreak community. Yeah. But last I heard there was a universal one and it works. Just keeps working everywhere.

Damian Hasse: I. And Ashish. One last thing that I will say on this as well. We don't expose within our product, we don't expose the LLMs directly to the end user, right? We use a variety of different LLMs to or as to say, mls LLMs, et cetera, to produce a response back, right? Obviously on the input it's key to help.

What is the. Person asking us to do. Yeah. And then from there it goes down. But anyway so that's another core aspect as well.

Ashish Rajan: Yeah. I think it's interesting to at least know what the moving parts of a chat, I guess chat bot would be from a security perspective. Also, I guess your [00:50:00] point, access control probably pays the you continuously manage, I guess confirming the right access is being maintained all the way up to the response coming out from the beginning of the request to the end of the response as well.

Damian Hasse: Yeah, it, so on that one, on access control, one of the key things that, that we did, we don't allow ML or LMS, whatever to make a determination. Who are you really Aish? Are you the admin or you're No, we, that, that's deterministic on our system. We, it just comes from the source. So whatever you can do as a shish you'll be able to get it back.

But sometimes the problem is when you're doing on the search page, sometimes the problem has been that. The way the system has been configured and how access has been put in place. It's was more open than what it was really intended. So you ended up finding information that before you just didn't know where to look now because it's index, et cetera, you ended up getting stuff that you're like, oops.

Yeah, why do I see that stuff? And [00:51:00] it's only problem with the LLM necessarily, it's just the reality that you already had access, you just didn't know

Ashish Rajan: where to look for.

Damian, we can we find you and connect with you to know a bit more about what you're doing and in the ML and AI spaceman before we wrap up the call

Damian Hasse: thank you for having me ish. I don't really, and Caleb I don't have much else to ask, say, enjoy the conversation. This has been it has been fun,

Ashish Rajan: For people to connect with you, Damian. Probably is the best place for you. LinkedIn, I'll put your LinkedIn stuff on the internet or, yeah, I think works on the internet, on the social notes.

I guess I'm like, I'm gonna put it on the internet. It's already on the internet.

Caleb Sima: On the internet. On the worldwide web is where I'll place it.

Ashish Rajan: Yeah. On the worldwide web. What

Damian Hasse: is that? K?

Ashish Rajan: No, but I appreciate it jumping on the call and thanks so much for this, man. Thank you so much for listening and watching this episode of AI Cybersecurity Podcast.

If you want to hear more episodes like these or watch them, you can definitely find them on our YouTube for AI Cybersecurity podcast or also on our website. www.aicybersecuritypodcast.com. And if you are interested in cloud, which is also assisted podcast called Cloud Security Podcast, where on a weekly basis we talk to cloud [00:52:00] security practitioners, leaders who are trying to solve different clients cloud security challenges at scale across the three most popular cloud wider.

You can find more information about Cloud Security Podcast on www.cloud security podcast or jv. Thank you again for supporting us. I'll see you next time. Peace.

‍