Caleb Sima: [00:00:00] Hackers aren't gonna give a about being seen at all. They're gonna land a beachhead, they're gonna spawn 500 different agents that are gonna crawl and probe and identify everything in your enterprise as fast as possible.

Andy Smith: In this AI lab we set up, this cloud lab, we found Opus 4.8 was able to hack its way to, to admin in, in 93% of cases.

Andy Smith: By placing a single secret in that same lab, that 93% success rate goes down to 0%.

Ashish Rajan: Anytime someone would ask me going, "Hey man, how good are we?" There was no way for me to guarantee that no one's inside the system.

Andy Smith: Does this thing even work? Why am I paying six figures for this thing when I get nothing out of it?

Andy Smith: They basically told red teamers there was deception in the environment, and even in cases where there wasn't deception, uh, the red teamers would actually be less effective at reaching their goal. Some of the malware authors got here first.

Ashish Rajan: It is actually Terminator style, just goes off and just deletes a bunch of

Caleb Sima: This actually becomes an offensive countermeasure, [00:01:00] not just a detection defensive measure, which I don't think anyone really in the industry today has even started thinking about.

Ashish Rajan: Deception is not the first that comes to most people's mind when they think about implementing a security strategy for an organization.

Ashish Rajan: In this conversation, we had Andy Smith from Tracebit. We speak about what's the point of deception in a world of AI agents floating around the internet and your internal environment, and we are us having both internal agents and external agents and non-human, and the pile just keeps growing. Also, why is it hard to start a deception program?

Ashish Rajan: Do I really need to be mature to start a deception program? Also, how do you stop a frontier model driven AI agent, which is already in your environment, to continue breaching further into your environment? All that and a lot more in this episode of AI Security Podcast. If you have been enjoying episodes of AI Security Podcast and have been here for a second or third time, I would really appreciate if you can take a quick second to drop a follow or subscribe.

Ashish Rajan: Whichever platform you're listening or watching this on, we are on Apple, Spotify, [00:02:00] YouTube, and LinkedIn, and anywhere else you listen or watch your podcasts. I hope you enjoy this conversation with Andy. A huge shout-out to Tracebit for sponsoring this episode as well. I'll see you in the next one. Peace.

Ashish Rajan: Hello, and welcome to another episode of AI Security Podcast. We've got Andy with me. Hey man, thanks for coming on the show.

Andy Smith: Thank you so much for having me. Very excited, uh, to talk with you guys.

Ashish Rajan: May- and to maybe kick things off, if you just wanna share a bit about yourself, uh, and your professional background, man.

Andy Smith: Sure thing. Uh, so I'm Andy Smith. I'm CEO and co-founder of Tracebit. Uh, I've been tinkering with cybersecurity since I was pretty, pretty young kid. Uh, pursued a career, like a technical career in, in software engineering, but most recently I founded Tracebit with my co-founder Sam. So we are a company in the deception technology space.

Andy Smith: We've been going for about three years now.

Ashish Rajan: When deception itself, when we started talking about this, I think, uh, I was doing... People have deception, at least they know what deception is in the traditional world. What is deception in the AI world? So just to get, set some context and people kinda go, "What is this?

Ashish Rajan: Is it still canaries? What are we [00:03:00] talking about here?"

Caleb Sima: Actually I would say w- we should probably just do a, a rehash of what is deception in general, because I think that, when we talk to CISOs and others about deception they very much have You know, old school like, thoughts about where deception- That's a fair point.

Caleb Sima: Basically, they think deception is just honeypots. Like, that's- Yeah ... where they sit. Fair point. Fair point. So we should probably rehash that too. Yeah, yeah. And then go into the, yeah, the growth.

Ashish Rajan: Like 101. Let's do trace l- let's do the, what's it called?

Caleb Sima: Let's do Deception 101. Yeah, that's a great idea.

Ashish Rajan: Yeah, let's do Deception 101, the 2026 edition.

Caleb Sima: Yeah.

Andy Smith: Sure thing. I can jump in and do a bit of Deception 101. So yeah, I think, I think when you hear deception, it can often fall into to two categories in, in terms of use cases. One is deception honeypots. So these can often be, you know, fake servers on the internet.

Andy Smith: Sometimes they're intentionally vulnerable. They're trying to, like, lure attackers into the en- [00:04:00] environment. The use case and the reason a lot of people deploy those is for threat intelligence. And often I think when you hear about s- people say, "Ah, deception, you know, that's really heavyweight," or, "You know, it requires a lot of care and feed," that can often be, like, this honeypot category of deception.

Andy Smith: Because ultimately you have to have someone spinning them up and someone monitoring them and, and analyzing the data that comes out of them. And- Go ahead.

Caleb Sima: And I'd los- also love to add another piece in here. As a, as a CISO, y- your job, you know, a lot of honeypots tend to think, "Oh, attract attackers so that you can see them and watch them."

Caleb Sima: And at the end of the day, actually CISOs just don't wanna attract any attention at all. Yeah. And so putting honeypots is, quote unquote, dumb. Right? Yeah. Because, like, why would I do that? Why... I don't care about what attackers do to a system. Like, I'm not a researcher. What I care about is them attacking my neighbor and not me.

Caleb Sima: Like, you know, why is honeypots even a thing I should think about, right? Yeah, so that, that's- Exactly ... that's I [00:05:00] think what most, I feel today, CISOs think when you hear deception.

Ashish Rajan: Security through obscurity.

Caleb Sima: Yeah.

Andy Smith: Yeah. And it's like e- exactly. It's like, why put that target on my back? Why present myself, like, publicly?

Andy Smith: Why draw attention to myself? And then the other thing is, this is... I've got some, uh, attack surface management tooling. I've got my customers scanning my networks, you know. Now they're gonna start- Yeah ... telling me I've got these vulnerable systems, and that's just, like, another massive headache.

Andy Smith: Like, I just don't want that. So that's, like, one category of, of deception, which is often kind of in this honeypot space. The other space the use case I would say is, is driving towards is high fidelity detections. So those, like, one in a million events that indicate compromise in your environment.

Andy Smith: Uh, that's what my company focus on. Um, and what, what that could look like is something you might have heard of, like canary tokens or, or canary credentials. And this is this idea of a, an event that I know should not happen. So, uh, a- another, another type of deception is deploying decoy resources around your environment.

Andy Smith: So a very, very simple example [00:06:00] could be you have an SSH key sat on a very critical server that you know no one should have access to, or a very limited number of, of people should have access to, and you are monitoring for usage of that SSH key. If someone uses that SSH key, one, you know the server it was deployed onto 'cause you, you took a record of it, two, you know it should not, not be deployed.

Andy Smith: So when that gets used, you know, you have all the information that comes with it, like IP addresses and, and other telemetry about where it is used, and you have the additional context you have. So you have this very, very high fidelity detection that, that likely, like, someone or something has compromised that highly critical asset for you.

Andy Smith: And then you have the additional information about what they, they've done. Uh, and the idea of this is, you know, it goes off very rarely. It, it might go off, it might never go off in your environment. It might go off once in a quarter. But when it does, it's the sort of thing that you can prioritize a high priority action to, or even automation to, in the event that it occurs.

Andy Smith: We would-- This would quite commonly be called, like, canaries, and that's, like, the terminology we use when we're talking about deception that we deploy.

Caleb Sima: And I'd also like to add, you know, a bit of a, [00:07:00] a, a larger overall sort of thought about this, which is, I-- y- Ashish knows this, I have ranted partially in previous episodes about deception and detection.

Caleb Sima: And I, you know, I've also posted some of my posts about this, where I have been a fan of deception for a long time. And I, I just wanna sort of lay some groundwork here because I think it's super important. Deception is one of those things that is truly an assume breach model, right? Where, you know, what, what a lot of people think of today is, I need to prevent, you know, obviously external issues.

Caleb Sima: I need to add prevention, and I need to detect attackers roaming around in my network, and I need to put a SIEM But dis- deception is truly a, to your sort of Andy's point here, it's I'm assuming an attacker's already in my environment. They already have had access to a machine, which they see [00:08:00] either this AWS key or this SSH key sitting there, and they're gonna go use it because they see that as lateral movement capability.

Caleb Sima: And then that flags a detection in order to go say, "Oh, somebody is doing something they shouldn't." And it's super, super, number one, high signal, unlike a SIEM, which basically you're spending most of your time doing work in order just to, just to get signals out of it that are accurate. And then number two, I think the thing that, that it does is not just the fact that you get high signal, but like number two, it m- it raises the cost for the attacker because any step they take could potentially absolutely say, "Oh, this is...

Caleb Sima: This could be potentially a honeypot or a token or a breadcrumb that causes an alert to be sent." And that means they have to be very careful about how they think about their next steps. And most people, I think when you go around, and I've done this many times, I've gone to [00:09:00] CISO events, and you're standing around, having a couple drinks, and I'll...

Caleb Sima: And I literally, this has happened, I'll stand in a group and people will be talking about SIEM and detection and how much they put in detection and response. And I say, "Well, what about deception? Why not just do deception instead of SIEM?" And I make it very adversarial on purpose, right? Like sort of like this, don't do SIEM at all or just do deception.

Caleb Sima: And people are like, "No way, deception is something you do when you're a very mature organization. That's not something you..." Like, why wouldn't you do it first before a SIEM? And the discussion and debate that takes place shows very much, I think, the thought process that I think a lot of the security industry in, is in, that that needs to change.

Caleb Sima: And I'm hoping to see AI do a forcing function here in this area on causing, I think, the industry to recognize this.

Andy Smith: 100%. I mean, I, o- obviously I'm biased, but I, I share a very similar view. I, I think it's very interesting, right? I think, I think the reality is, like [00:10:00] no one ever got fired for building a SIEM.

Andy Smith: It's the sort of thing that you go and you spin up a security program, like no one is ever gonna challenge you for building a SIEM. But we meet security teams, you know, just spinning up, maybe they're resource constrained, they're in a startup, things are changing all the time. You know, you meet them and they're like, "We're building a SIEM, that's the plan."

Andy Smith: And you come back six months later, they're still building the SIEM. Like, they still have those visibility gaps in the organization that they, you know that, that led them to want to build the SIEM in the, the first place, that they're concerned about. And you know, our proposal has been to them, like, "Hey, why don't we just get some deception?

Andy Smith: You know, we could actually cover four or five of your most critical decept- d- critical systems with deception, implement assume breach, get these high fidelity detections. Like, we could do that in a week, we could do that in, in two weeks." The, the times of value on that SIEM is much, much greater. But yeah, the, the reality is that the, the standard, the, the, there's a bit of groupthink going on in the security industry that like you, you must do it, do a SIEM and people won't, won't ever challenge that as the, the first piece that you do.

Andy Smith: Whereas, yeah, I, I, I totally think like deception can be a [00:11:00] quick win, like very early on in the security phase.

Caleb Sima: The, the one thing pointing in the fact that people have to do a SIEM, because every compliance requirement in the world- That is correct ... requires a SIEM, and none of them require deception.

Caleb Sima: So. That is correct. Yeah. That is correct. That is definitely

Ashish Rajan: one. Correct. Yeah. But maybe to- Ashish, you were gonna say something? Yeah, I was gonna say, but maybe to add to why we landed on this as a... I was gonna say maybe the Netflix folks of the world are to be ba- blamed for this a bit. At least when I started doing cloud engineering or cloud security engineering, the whole idea was pretty much the same thing, what we're talking about, the second kind, the canary.

Ashish Rajan: Honeypot was pretty much like the pre-cloud as well. Mm-hmm. But Canary became a lot more popular when the cloud era started. People say, "I can just leave my AWS tokens at places and see who kind of makes, makes a call for it." And it be... I think that's when people started to, oh, it's the Netflix of the world, people who are engineering first.

Ashish Rajan: They are the ones who can do this. They are the ones who kind of set the standard, and they had this thing called the Chaos Monkey or something similar. Yeah.

Caleb Sima: Chaos Monkey was different though. Uh-

Ashish Rajan: Oh, no. I think like what was the suite called? They had like an entire suite of [00:12:00] things that they... But Chaos Monkey was part of that suite.

Ashish Rajan: I can't remember the name of it, but essentially they had created this suite of security and non-security open source tools. One of them used to be around the honeypot thing, and that basically became, oh, I have to be a Netflix engineering first company. And going back to Caleb, what you just said, most people are like, "Hey, I just want my, uh, ISO or SOC 2 Type 2, and I wanna get through that before I-" Yeah.

Ashish Rajan: 'cause I'm just a one-person army." Yeah. I'm more or less... I'm, or it was the Simian army. There you go. That's the word. Simian army was the the Netflix name for that suite of open source products they had, and that's kinda led that part for, oh, you have to be sophisticated to have all these tokens, because someone has to keep a track of where these tokens are, and then someone has to think about, oh, if they do say, raise an alert, where is this going?

Ashish Rajan: And who, what are we doing when this happens as well? There's a whole planning around it. Saying that, both of you mentioned AI should be m- changing this. I'm curious, Andy, how has AI changed that the honeypot versus the [00:13:00] canary conversation, uh, when it comes to deception? And is it a, in many ways, is it the same but with a different skin, or is it a lot more different than people think?

Ashish Rajan: It is actually Terminator style, just goes off and just deletes a bunch of shit.

Andy Smith: Yeah. Yeah, for sure. Um, yeah, I mean, this is a super interesting and obviously very, very relevant topic of conversation. And I think like, to, to Caleb's point, right, we didn't start Tracebit thinking, um, that we, we like, we'll go solve AI, AI security.

Andy Smith: Like we, we just thought deception has a lot of value. Making it really easy for people is g- is like, it's gonna be a bit of a home run. I think what's interesting, if you look at like the history of deception, there was a study 10 or so years ago comparing basically like testing humans and the impact of deception.

Andy Smith: And to Caleb's point, like one of the interesting findings from that study was that, you know, the psychological impact of deception is very, very real. So they found they they basically told red teamers there was deception in the environment, and even in cases where there wasn't deception, uh, the red teamers would actually be less effective at reaching their goal because they had the [00:14:00] perception that there was.

Andy Smith: So what we've done recently at Trace but actually was like run a similar study against AI agents. So, you know, if we start some AI agents in like a cloud range, um, like a realistic cloud lab and ask them to try and like hack the, the environment, is there a difference in their capabilities between them being able to hack that environment when they think there's deception in the environment versus when they don't think there's deception in the environment?

Andy Smith: We found some very, very interesting results. Uh, like Opus, for example it's like ability to hack that environment goes down from like 20% to, to 5% in, in the range because it's more cautious, it's more, uh, it's more likely to skip resources that it believes are canaries even when they're not present.

Andy Smith: And yeah, you'll, you'll

Caleb Sima: see things that you won't try because you'll think, "Oh, that's a trap," right?

Andy Smith: Yeah. Exactly. Even if

Caleb Sima: it's not. Exactly. Yeah.

Andy Smith: And, and then the other thing, right, is that, you know, the beauty of, of deception in canaries is they're attack technique agnostic. You know, they, you know, they [00:15:00] are focused on the resources that you care about.

Andy Smith: They're focused on, you know, you know, I really wanna protect my AWS Secrets Manager secrets for example. They are what matters to me. Yeah. Like I don't care how you get at them, I don't care what IP address you come from, I don't care what series of actions you take to get there. So if we have some- Yeah

Andy Smith: deceptive AWS Secrets Manager secrets, and we focus on those, like we're gonna detect them whether it's an insider, an outsider, an AI agent, like it, it just doesn't matter. Like we're gonna be able to make that, that detection.

Caleb Sima: Let me give you an example of a real world scenario. So in a previous company that I was at, we took our production database. In our production database, we created let's say a file or a table that was clearly something very interesting, but was never queried or touched ever in any production system because that was just never part of the production system at all.

Caleb Sima: And you leave that as sort of your tripwire, right? Mm-hmm. If a query ever hits that We immediately get a very, very high signal to this. And what's [00:16:00] great about this is, going back to Andy's point, is y- you... This is truly assume breach. I don't care if it was an attacker from the outside that somehow used SQL injection or used some app vulnerability in order to gain access to the app, to then gain access to the database in order to query the tables to see what was going on, right?

Caleb Sima: I don't care about any of that. I just assume that already happens, right? I don't care if it's an insider employee who somehow gets access to the system who can then go and query it, right? I can get-- I get signaled any time any of those things ever goes through. And this is super critical, and I feel like the problem space in detection specifically is that we spend so much time and effort and money putting how many logs and visibility can we get, how many rules and detections can we put in place in order to only identify various [00:17:00] certain attack paths without just focusing on the thing that matters and then tripping them up there.

Caleb Sima: You don't have to do any work. You could just create the table, get the right log from there, and put one detection, and it sits there And like that gets you way higher signal, way higher cap- probability of occurring than doing all of this work of trying to detect all of these attacks in the meantime.

Caleb Sima: And so like, I feel like this is something that a lot of people, quite don't grasp in the sense of how deception can be very, very useful in these an- these angles.

Andy Smith: 100%. Like, I think, um, well, what's really interesting, right, is, we're testing and looking at offensive AI agents because of the, the post-Mythos story, right?

Andy Smith: Which is front of mind for everyone at the moment, and, you know, everyone is looking for an answer for, and a right to be concerned of. I would say, like, the truth is what we're actually seeing in customer environments and what our customers are actually experiencing day to [00:18:00] day is the other side of the AI story, which is the internal AI agents.

Andy Smith: You know, the sanctioned AI agents that are maybe overstepping the boundary a little bit, maybe going a little bit deeper into that environment, discovering access that that user didn't even know that they had. And like, we're seeing this in customer environments. You know, we saw this recently where, you know, I think a couple of days into a deployment with a customer, a production Kubernetes pod lit up, and someone had pulled a canary token out of that pod and started trying to use it, and it, it was located to an engineer's laptop.

Andy Smith: Guess what? It was Claude Code. And the, the, the team didn't even know there was a path from that engineer's laptop all the way to that production Kubernetes pod. Uh, Claude Code had, had, had figured it out. And again, you know, the, the beauty of deception, the beauty of canaries is, uh, if you'd have tried to write an alert for that, it would've looked very different for the alert for Mythos hacking that engineer's laptop and then, and then working their way into the pod.

Andy Smith: But because you used canaries, we're actually able to detect both of those cases.

Caleb Sima: So in the AI world, you know, one of the [00:19:00] things is obviously in, in pre-A or actually really today's world to some extent, we quite haven't gotten there yet, is the way attackers work is obviously they gain their foothold or your insider has their foothold, and then you sort of go through and try to figure out how to both lateral move, whether it's just network, whether it's application-based, whe- you know, whe- whether it's cloud-based.

Caleb Sima: They figure out their ways of sort of moving around. And generally speaking, these aren't necessarily AI. They're, they're automated in the sense that today people pop, they put their tool sets, they do their CNC they try to do their stuff in order to sort of probe around and figure out what's going on.

Caleb Sima: And the ability for these little detectors is sometimes hard to graft. So if you, the pro- and so this is where it goes into maybe some of the problem space of deception, which I'd love to sort of hit on. I- in deception, it also requires you to [00:20:00] know and place the right breadcrumbs or the right decoys in the right places- So that when this attacker lands, they're going to go and they're going to hit these things in the right place.

Caleb Sima: Which by the way, requires a lot of both thought, strategy, capability, deployment. And so one of the things that I've always you know, reasoned around why deception has failed is because number one, there is no strategy or no one teaches you how to think this way. Everyone teaches you how to think decept- I mean, detection, right?

Caleb Sima: Oh, you look for the bad things. But no one teaches you how to think in deception. And so no one knows where to put it or how to put it, and they don't understand the strategy of this. So there's this real lack of understanding of saying, "Hey, Bob if you're gonna go implement deception, go do that." Bob's gonna go, "I don't even [00:21:00] know where to start," right?

Caleb Sima: Uh, so that's number one. And then number two, I think even if they knew the strategy Deploying these things is difficult. Oh y- oh, by the way, you need decoys in, on endpoints, on, in browser history, on web apps, in databases, on servers, in containers. And by the way, they can't just be in dev, they have to be in production-oriented assets.

Caleb Sima: And then that immediately becomes a m- you know, red light siren. Oh my God, no way, Yeah. Am I going to do this? So then I look on two hands. I can do detection, which only requires logs and a centralized place that I own and control, or I go somehow deploy all this crap on production things and manage that and figure out how to do that.

Caleb Sima: No way am I [00:22:00] doing this. I'm going straight to detection. So- If

Ashish Rajan: I still want my job, yes.

Caleb Sima: Yeah. Yeah. So how does, you know, how do we either overcome this in the deception, you know, world to make it so that it's not like this, right? How do, how do we overcome these challenges? Yeah.

Ashish Rajan: And can I add one more thing to that, Andy- Yeah

Ashish Rajan: while we are on this topic as well? 'Cause obviously now every person is of the belief that I have Fable 5 or whatever the latest version is that's gonna come out tomorrow. Can I do that on my own if... Because the, we started the conversation talking about the fact that the, the premise or deception has been that, oh, you need to have a certain level of maturity, but then now that we have AI, does that maturity reduces a bit or...

Ashish Rajan: I'm curious if you wanna summarize that into your answer as well. That'd be awesome.

Andy Smith: Yeah.

Caleb Sima: Oh yeah, but also, Ash- Ashish, no way am I letting AI agents touch my production to go deploy stuff either. Well, that's even worse. That's even worse. Oh, that's even- I wouldn't even let the security team do it, and then you're saying the security team's gonna use [00:23:00] agents to go do it?

Caleb Sima: Uh, I don't know.

Andy Smith: Yeah, I, I mean, I think on, on the maturity piece, right, I mean, I think I'm in the Caleb boat here of thinking that actually this is something you, you can do early stage and, you know, I, I think that change is starting to come, and, and I think the kinda why now for that is actually the AI threat both internal, uh, and external.

Andy Smith: Uh, yeah, in, in terms of like getting this to mass adoption, which is ultimately my goal I think like the, you're, you've hit the nail on the head there Caleb, that it's like strategy and deployment. Those are the two blockers. On the strategy side, I think what's great about today, right, is like AI can, can help you with this, you know.

Andy Smith: And this is as simple, right, as, you know, helping a, a security team understand their environment and make suggestions for, for like naming resources in a way that, you know, would be believable for a threat actor. Because e- exactly right, you know, if you're thinking about like detection, where I have to come up with the idea of a few needles- that I might find in, in my big haystack and write those, you know, and I have to write, you know, the, the definition for a couple of detection like needles.

Andy Smith: That's not that much work. Whereas [00:24:00] if I-- the alternative is to go look at a huge environment and deploy, you know, thousands and thousands of bits of hay across all of this massive, massive haystack, right? And, and des- design those and think about them and name them and configure them in a way that's gonna be believable for a threat actor.

Andy Smith: That does take obviously a lot more effort. That said, LLMs are obviously actually really, really good for this, right? You know, simply you could take, all the names of an S3 bucket, all the names of S3 buckets in a, a large like cloud account and say, given this large account, given the context, you know, you can find on the internet about this c- this company, given the, the names of these buckets, give me five that are gonna really stand out to a threat actor.

Andy Smith: And the LLMs from our experience do a really good job at that. I would say like philosophically, our approach when it comes to deception strategies is like breadth of coverage. Let's get these in as many places as possible where they're still gonna continue to produce high fidelity detections.

Andy Smith: Where they're, where they're not going to create a lot of noise in itself, sort of false positives.

Caleb Sima: Yeah. Would you say [00:25:00] like, you know, my answer to that has always been target your crown jewels, right? Don't, you don't need to deploy them everywhere. You just need to deploy them on the important things that you know.

Caleb Sima: Like, the one advantage we have as defenders is we know where the attackers will want to go. Whether it's customer data or, um, you know, whatever that happens to be, you wanna go there, and then you can plant the tripwires appropriately. So you don't have to scatter it across the enterprise. You just have to say, "Well, we've got three primary databases and four primary systems that we know they're gonna go after.

Caleb Sima: If they-- If someone's in here, if there's an insider threat, they're gonna go here. So let's go plant some things strategically."

Andy Smith: I think this, there, there is, there's gonna be trade-offs here, and, and part of that is gonna depend on, you know, if you have a tool that's gonna be able to automate this for you, or if that each new environment you deploy to is gonna require a lot of manual work yourself.

Andy Smith: I personally come from the view that, hey, if, if, you know, in [00:26:00] a dev environment, I can get some high-fidelity detections from Canaries, that has value because the reality is most attacks aren't just hitting, you know, that critical production environment, like from day one, right? They're working their way up from those environments.

Andy Smith: So if you can get the detection and the deterrent effect earlier, I, I think that makes sense. But I think ultimately there's, there is a cost always to doing these things, and there, there is a trade-off there. So yes, if, if I'm gonna put them anywhere, like at all, it's gonna be around the crown jewels.

Andy Smith: But I would say like- So in this new

Caleb Sima: world of automated capability- Yeah ... deploying and strategy, then it's, it's easier to apply that way.

Andy Smith: Exactly. So I think on the deployment side of things, right? I think that for me is like why like cloud is like such a, such an exciting place, right? We've got like a new substrate to play with.

Andy Smith: If you look at the dis- deception companies that like came and went in the past, you know, they were deploying into data centers. Some of them were like shipping you physical racks to go and actually install in, in your data center, and they were super heavyweight. If you contrast that to like [00:27:00] cloud environments where we've got serverless resources, right?

Andy Smith: I've mentioned S3 buckets a few times, right? But it's, it's free to deploy them. You know, and that's both from like a financial point of view, but it's also from like an operational point of view. You've already accepted like, a, the risk from AWS as your software vendor, and AWS are like patching those, those buckets for you under the covers.

Andy Smith: Like you're not having to maintain them, you're not having to care for them. They're not introducing risk, they're not running a vendor's third-party software. So in many senses, these are like as, as free as they possibly could be. So when you have resources like that and, you know, take that and you can apply that to multi- multiple different clouds, many different kinds of serverless resources, you know, and you think about also like the tooling that we now have.

Andy Smith: So we have, you know, Terraform that we're managing the CIC, this, this infrastructure with. We've got Helm charts we're managing our Kubernetes infrastructure with. We, we have these like primitives in, in our environments. We already have these integrations, so like slotting into those to deploy deception, I think is like much more of a, of a no-brainer than it was previously.

Caleb Sima: How do we think about the [00:28:00] new method of lateral movement, right? Like I feel like most attackers now it's not necessarily server side per se, but social engineering endpoint focused. And it's not what CrowdStrike packs. Like, let's take the two that I think come top of mind. I social engineer your employee, I allow them to install a malicious Chrome plugin.

Caleb Sima: I then start roaming around their browser history, see they have access to Salesforce, go and download and generate reports from Salesforce. Right, I now have access to whatever access they have inside their browser history and state. Or number two, I'm an engineer, and clearly I'm using AI to code.

Caleb Sima: It downloads a supply chain library. That supply chain library gives the person a backdoor. They now start rooting around on the user level inside the dev machine, starts looking at their dev repositories, GitHub code, gets access to all of that. Like, how then do we start looking at [00:29:00] what does that look like now from a token deployment canary tripwire perspective?

Andy Smith: Yeah for sure. Um, and you know, we're hearing this from customers as well, right? You know, especially like the malware's AI generated, there's lots of different versions of it, but it's also not just, it's not doing that much on the system, right? It's pulling some cookies or it's pulling some credentials and then- Yeah, yeah

Andy Smith: there isn't time necessarily for CrowdStrike to get in the way of that. Yeah, I mean, I think all of those systems are great opportunities for canaries for, for me. So, we actually launched this recently in our product, which was Chrome cookies as canaries. So, you know, customers can point subdomains at, at Tracebit.

Andy Smith: So imagine if salesforce.customer.com or vpn.customer.com and then we'll actually leverage a Chrome extension to inject those cookies into the browsers, unique ones in every single, in every single browser and monitor for usage. So these are kind of almost like cookie- And this doesn't affect

Caleb Sima: the user's usability of when they're browsing or doing things?

Andy Smith: So these would be like distinct from the real platforms- Mm ... that the, the customer is [00:30:00] using. So, you know, these would be purely deceptive sub-subdomains. And so if someone were to-- so the threat actor would see, uh, and believe that this Salesforce instance is associated with the customer, when in fact it's not, it's hosted by Tracebit, and then, then attempt to use it.

Andy Smith: And you know, you imagine you have like 10, 15, 20 of those, those cookies in, in, in a browser, all with kind of interesting and attractive and, and like distinct looking infrastructure r-running them, running for them. That's, that's a very difficult difficult thing to avoid. Then when you actually get into Salesforce this is something we're actively looking at now.

Andy Smith: It's, it's not in the platform. I think there are actually straightforward things you can do there with deception, right? You can go upload some, some, some PDF documents or some XLS documents into your Salesforce instance. Another one that's front of mind, because I was talking to a customer about this recently, is, you know, those who are using Salesforce for their customer support, I've always thought HAR files are such an attractive target to go, uh, inject a load of credentials and cookies into and, and place around your Salesforce instance.

Andy Smith: On the other use case you mentioned, which was like engineers and [00:31:00] AI coding, yeah, I mean the, the classic access keys we're finding to be really, really effective. So your AWS keys, Azure keys, SSH keys these are really powerful. What I would say, which is, which is interesting there is when you get into the weeds of it here and start thinking about how to configure this for the AI agents, there is some quite, quite interesting nuance that comes out.

Andy Smith: So, one of the things you can do is, uh, you can instruct your AI agents, you know, Claude or Codex, pass through environment variables that are gonna identify them when they do end up hitting those tokens, so that on the defender's side they can say, "Hey, this was a person," or, this was a, was an agent."

Andy Smith: We're also working with customers at the moment to add, you know, instructions into these credentials to discourage agents from using them. So this could be as simple as, you know, a comment above an AWS key that says, "This is for human use only." And you know, guess what? If an, if an AI agent goes and uses that, like that's, that's pretty interesting.

Andy Smith: Like, they may have become misaligned or, you know, a user may be pushing them to do something that they [00:32:00] don't. And what we're having, what our customers are doing at the moment is, you know, they're correlating those events with, the transcripts, with the EDR logs to, you know, confirm like, "Hey, did an agent use this, and what was the, the intention?"

Caleb Sima: It's, it's funny 'cause I actually think what will happen is if I'm an engineer or software developer, and I hit that repo with that comment with the AWS key, my bet is the agent will recognize that there is a, uh, static key in the code, automatically generate a PR to remove this, and send a signal to say, "Hey, you should not have static keys inside your code."

Andy Smith: Well,

Caleb Sima: funny you say that. They, they do

Andy Smith: that

Caleb Sima: very well right now. They do it

Andy Smith: quite

Caleb Sima: well. Yeah.

Andy Smith: Well, that's actually, that's one of the reasons we use, uh, short-lived keys for our, uh, our tokens actually. But yeah.

Ashish Rajan: I was gonna say, maybe, one thing, and I keep going back to where, where we started the conversation.

Ashish Rajan: A, obviously we wanna encourage people for deception 'cause to what Caleb was saying as well, that assume breach. Everyone loves the idea of assume breach, so I don't think anyone disagrees with it. Where it gets a bit difficult, [00:33:00] outside the whole, "Hey, I need to be a certain maturity level," the other thing that people come down to is- Yeah

Ashish Rajan: "Hey, I... What, what's, how do I show an ROI for the investment that I'm doing for spending, I don't know, months, weeks putting all these, uh, canaries across my environment, whether it's in production, not in production?" I'm curious as to people you're working with, uh, A, feel free to, uh, obviously ignore all the names, but, uh, certain industries or certain kind of companies are more open for this, or how are they showing the ROI?

Ashish Rajan: 'Cause a lot of people who would watch listen to this are like, "It's great, guys, but what am I telling my boss to get the budget for this?" So- Yeah ... I'm curious as to what you- Yeah,

Caleb Sima: and, yeah, and also can I add to that, is- Yeah ... generally when you set up deception, most people, hopefully, knock on wood, aren't under active attack.

Caleb Sima: That would be scary. So you see nothing for months- Yeah. ... or hopefully years. Yeah. And then you're like, "Does this thing even work?" Right. And the- Why am I paying six figures for this thing when I get nothing out [00:34:00] of it?

Ashish Rajan: Yeah. And as the budget cup comes in like what are we removing first? Let's remove that deception thing.

Ashish Rajan: Yes. I haven't seen that for a

Caleb Sima: while. It doesn't do anything, right? Yeah. Like, we don't see anything out of it. '

Ashish Rajan: Cause I'm sure you have this battle on your side as well, Andy. So I, as much as I joke about this, but I'm pr- sure this is a real problem for you and- Yeah, yeah ... and the people you work with.

Caleb Sima: It's real.

Caleb Sima: Yeah.

Ashish Rajan: Yeah.

Andy Smith: It's totally real. Um, and it, and it's interesting, right? There is that class of product that just ends up in, you know, without question. You know, EDR is a, a good example of this, right? You're not gonna churn your EDR after 12 months, even if you never got, got an alert from that. I'm not suggesting you should, but you know, I, I think it is, it is curious that, you know, there are these, these products that, that kind of get that checkbox.

Andy Smith: It's absolutely that, like, proof of value. Obviously it's, it's very important. You know, you- If you're buying a product, like you, you wanna ensure you're getting value from that. I mean, what that practically looks like for us I think like coverage and coverage metrics is a big part of that.

Andy Smith: So, you know, what like percentage of my systems and how many different systems am I protecting? Uh, and you know, I think our pitch, has become that, you know, with Tracebit actually, you know, we can cover like such a [00:35:00] large number of systems, you know, where we're on your workstations, we're in your SDLC, we're in your identity, we're in your cloud.

Andy Smith: We're coming for your, you know, your SaaS and your browsers and other places like that. So, you know, it's actually rare if you think about it, for like one security product to give you just such coverage across all those systems. And then, you know, the, the reality is, and this is the, the kind of the industry we're in, you know, there will be, like statistically it's likely within, you know, the course of a year, there will be some incident within, you know, one of those systems that, becomes front of mind.

Andy Smith: And it might not be for a customer that, you know, we make a detection because not everyone is, is getting hacked. Not everyone is getting, getting compromised. But you know, that peace of mind the CISO got, you know, when they woke up that morning and they saw, wow, there's this, there's a big, there's a, there's, you know, yet another supply chain attack going on, and, you know, they know that Tracebit didn't go off for, you know, their SDLC or didn't go off, you know, for their developer workstations.

Andy Smith: That, that peace of mind we've given them in that moment provides a lot of value. There are also obviously more, um, [00:36:00] concrete and, and measurable things like red team exercises. So, you know, I think in a lot of our customer base, you know, they are running a red team exercise, a, a couple of times a year.

Andy Smith: Some will have dedicated i- internal red teams as well. We are very keen with our product and, and generally will always detect those, those red team exercises. That as well, by the way, you know, if you're thinking of running one of these programs, is why it's important to keep it, it fresh. You know, I think, I think these deception programs can fail when the team, they do a deployment for like good for T equals now, red team roll in, red team get detected, red team take their notes.

Andy Smith: The team never, never find time to go update, update that deception, and then the red team are gonna sidestep that, that next time. That's why I do think it's really important, like upfront, ahead of like planning your deception program to think, you know, how are we gonna make this dynamic? How are we gonna make this change so three months later, this is, it's, you know, they're not in the same place.

Andy Smith: They don't, they don't look the same.

Ashish Rajan: The way I, I'm 100% agreeing with you, but I also probably throw another one in there, which worked for me. And the [00:37:00] reason I knew about the Netflix one is because we deployed it in one of the previous companies I was working for. The way I used to justify that to my boss was the CTO, was the fact that I've got all these tools, the SIEM, the

EDR,

Ashish Rajan: EDR provider, everything.

Ashish Rajan: But anytime someone would ask me going, "Hey man how good are we? There was no way for me to guarantee that no one's inside the system. Detections haven't-- I mean, in my mind, the detection haven't triggered is oh yeah, I mean, I guess nothing has happened yet, so maybe that's why no detection. But I think having one of those reception or canary tokens not go off meant that, okay, someone unknown has not come into the environment who pr- clearly went down the wrong route or went down the honeypot or whatever.

Ashish Rajan: And, uh, that seemed to have worked for me, at least in the past. But that- Nice ... that's all I wanted to say, 'cause that, that could be another value add for our people guarant- being able to say, "Yes, I can... The value for this is the fact that I know all the alerts that I have clearly work, but also the fact that this did not trigger double downs on [00:38:00] the value proposition of having all these other things."

Ashish Rajan: It kind of like, for me, it's like they go both, both together, I guess, if that makes sense.

Andy Smith: I think that's a, a great addition. That is something we, we have heard. And I also think actually, you know, this is of- that can often be, you know, a, a CEO, like leadership conversation, right? Which is like, we've bought all these fancy tools, like how do we know they're not- they're even working?

Andy Smith: And like the beauty of deception, it is fundamentally easy to understand, right? It's like, hey, we have these important resources. We put some fake resources alongside them, and we know categorically that no one has hit those resources. So that shows to us that, you know, these multiple layers of defenses that we put in place are doing their job.

Andy Smith: It's an easy thing for, you know, an exec team or a board to reason about, and I think that's a great example.

Caleb Sima: You know, I would just be, I'd be probably a little bit careful here on this, because I wouldn't think about it that way. Like h- here's a great example. Let's just say I have a, you know, an S3 bucket with all my most important information, and then I have a fake S3 bucket with what looks like to be all my important information.

Caleb Sima: And would I be [00:39:00] willing to not put any protections on my primary S3 bucket because I have a fake one, because I would be assured that the attacker would hit the fake one? No. You know, it's like just because I have a fake one does not mean that the attacker will hit that fake one.

Andy Smith: Yep.

Caleb Sima: And so there's still a large, unknown gap here of confidence of s- you know, a- and this is, I think, another challenge for deception, which is deception feels to me a bit of a just in case play, right?

Caleb Sima: Like my primary S3 bucket needs to be well-protected. My primary S3 bucket, I need to understand exact attacks that are going against that S3 bucket if attacks are occurring. And then maybe on the side I have this other S3 bucket that looks juicy, that hopefully the attacker may or may not hit. But I don't have any data or any stats [00:40:00] To, to determine the confidence level of that occurring, right?

Caleb Sima: And so I know I need to do number one first, and then I can add number two just in case. That's why, uh, you know, you have to be careful in that situation.

Andy Smith: I think that's totally fair. Um, a, a good example like that from a CISO I was talking to recently that, that really resonated was like... And, and I think this is like where like, the experienced CISOs like, like really shows, right?

Andy Smith: It's like, they know that at some point, like those controls are gonna fail. And it, you know, it could be completely innocuous. It could be like a complete accident. You know, like the example he gave me was he was in an org once where someone accidentally turned off the EDRs. You know, they, they- Yes

Andy Smith: they pushed the wrong change to Jamf, and CrowdStrike got disabled for a day. By, by the way, happens

Caleb Sima: more often than people would like to admit.

Andy Smith: Yeah. I'm, I'm sure it does. I'm sure it does, right? And, and you know, like, and when that happens, like what do you have in place for that event? And like often the answer, of- the answer for teams is like, is nothing.

Andy Smith: Nothing. Uh, you know, whereas what he wanted was, you know, he wanted like a porcupine basically feeling on those, those [00:41:00] laptops of like credentials everywhere, uh, that, you know, if, if, you know, that EDR did break, that malware is gonna step on something- Which- ... that's gonna produce a

Caleb Sima: detection. By the way, we should transition it here a bit into, I think, how the world is going to change with AI.

Caleb Sima: I've always been here, of course, you know, of this belief that deception will always not make it into a primary spending category, uh, due to many of the things that we've brought up today. But I think that AI has be- is going to become a forcing function for this now deception really sort of becoming a primary spend.

Caleb Sima: And the reasons for that are somewhat obvious, but, you know, we'll restate. Obviously the ability to identify exploits and use them at machine speed has been proven. And so it's no longer about an attacker landing a beachhead and then probing around trying to be, let me say, like hidden in order to go find their things.

Caleb Sima: They don't need to do that [00:42:00] anymore. I actually think it's gonna go the opposite way, which is attackers aren't gonna give a about being seen at all. They're gonna land a beachhead. They're gonna spawn 500 different agents that are gonna crawl and probe and identify everything in your enterprise as fast as possible And it's going to identify these things, and when it finds it, it's gonna exfil it as fast as it can.

Caleb Sima: And so actually the detection team and the team itself is gonna be overrun with massive amounts of things that are going through from a, from sort of an AI agent attacker perspective, and that's gonna be the next sort of level of how these attacks occur. What do we do here, and how does deception really play a role that becomes very fundamental?

Ashish Rajan: Uh, if you have- Or maybe even if you have a different perspective of the future of, uh, deception as well.

Caleb Sima: Yeah, yeah. Or you have a different perspective, yeah,

Ashish Rajan: for sure. Yeah. I mean, and you completely disagree with, uh, Caleb. Like, no, man. Yeah. Like, way worse.

Andy Smith: I'm, I'm happy to agree with Caleb that deception is gonna be kinda, [00:43:00] gonna become a line item for all security teams.

Andy Smith: Yeah. No, I... But like, but-

Caleb Sima: But why?

Andy Smith: But

Caleb Sima: why? Yeah, you have to-

Andy Smith: Yeah, for sure ... is, is the reasoning correct?

Caleb Sima: Yeah. Yes.

Andy Smith: No, I, I think that reasoning is spot on, you know. And I think the, a scary image that you've just summoned up, right? The idea of all these, these agents ripping through our systems.

Andy Smith: I think, like, that's where, like, the deterrent effect of deception is very powerful. You know, the fact that, hey, you know, I, maybe I prompt these AI agents to say, "Hey, I saw this company blog about using deception in their security stack recently try and avoid the canaries," right? You know, that is actually, and we've shown this, like, that is actually gonna hamstring these agents.

Andy Smith: It's actually gonna slow them down. It's gonna cause them to miss attack paths that are valid. It's just gonna cause them to, to be less effective. So that's, that's a key part of it. And then, yeah, exactly like the speed at which this is happening I think is ultimately going to drive us to more automation, right?

Andy Smith: It's like we, we should just be creating like, as much confusion as we possibly can for these AI agents, right? Like, we want them in an environment, you know, like every [00:44:00] agent like hacking your environment, like needs to be like assuming there's deception this, in, in this environment and acting accordingly.

Andy Smith: And like, we need to be like, like, and we can leverage like these lightweight resources to just sow like sheer confusion, waste those tokens, right? Like how, like what like token cost can I incur on these agents by like just by, by putting ultimately like junk canaries in the environments that they, they're gonna have to consume, they're gonna have to interact with, uh, to waste, waste their time.

Caleb Sima: Or I find it fascinating to think about because it's true, it's like how do I stop the flood? How do I stop the army of agents in the sense of where you feed it misinformation, Yeah ... so that you can actively prevent the agents from actually working versus... Now this actually becomes an offensive countermeasure, not just a detection defensive measure, which I don't think anyone really in the industry today has even started thinking [00:45:00] about.

Caleb Sima: You're about to publish a, a paper around this. I'd love to can you talk a bit more about that?

Andy Smith: Yeah. So, so we, we're just publishing some, some research here, which is actually some of the malware authors got here first. And what they've been noted as doing is in placing in some of their, their malware, they've been placing it- strings that are h- highly likely to trigger safeguards in these agents.

Andy Smith: So think, biological warfare instructions, or think references to, for the Chinese models, like references to some, like, controversial, like, political events. These agents are trained and set up to shut down the moment they interact with that, that material. These are the, the guardrails that are placed in them.

Andy Smith: When we, when we started pulling on this thread and then exploring that, what we realized was, like, canaries and deception are actually uniquely placed as a, as a defensive mechanism to take advantage of this. So the question we asked was like, what if we placed some of these, you know, references to, you know, certain political events or, or [00:46:00] biological warfare m- m- material into, you know, a secret in a cloud environment?

Andy Smith: Like, how does that affect the actual agents and their, their effectiveness and a- and ability to go through the environment? There were some very, very interesting results we got. I think the, the, the best headline number is you know, in this AI lab we set up, this cloud lab we set up, We found Opus 4.8 was able to hack its way to admin in, in 93% of cases.

Andy Smith: By placing a single secret in that same lab that had some content in it that triggered Opus's guardrails, that 93% success rate goes down to zero. So that is just like an example of, of just how, you know, you can, you can use some of the guardrails a- like that these agents have in them, like against them, you know, in a very, a very like asymmetric way, right?

Andy Smith: Like a single secret that is, you know, it's like-

Caleb Sima: Yeah ...

Andy Smith: 50 characters of text has actually shut down this like incredibly capable piece of technology.

Caleb Sima: Man, that's pretty... It's, that's fascinating to think about. Especially as they, if they [00:47:00] use one of the things that, you know, everyone thinks you're using frontier models, but of course they're gonna be like, "Well, hackers aren't gonna use frontier models."

Caleb Sima: What I really liked is they're gonna use things like DeepSeek or GLM or Chinese models, and you go, "Yeah, well, Chinese models got the same thing." You just talk about negative things about Chinese culture and politics, and it does the exact same shut down. And it's, it's amazing to think about- Yeah, and you see- Yeah, go ahead.

Andy Smith: You pull on that thread, right? And, you know, you have like when you, when you're like doing the response and you see, okay, well, we, we saw this agent shut down on like the, you know, the anti-Western model string or the anti-Chinese model string, and you can like distinguish- Yeah ... and like fingerprint the models based upon what they actually did.

Andy Smith: Like it's actually pretty, pretty fascinating.

Ashish Rajan: So is it, would it f- would it be fair to say at least one action people can take as they walk away from this conversation, where at least we do be able to consider a To what you were saying, Andy, or and Caleb as well, identify what your crown jewels are, either third party or internal, whatever that production database or whatever that looks like.

Ashish Rajan: 'Cause you at le- people should at least be aware of what their [00:48:00] absolute crown jewels are, where if it's not part of the assume breach for how, whether it's for detection or deception, you probably wanna have the same consideration for it. And the next, next one was the research that you guys did, Andy, which is basically having some kind of instructions where it at least it turns into anti-Western or anti-Chinese, uh, re- comment, or it turn, turns into like a, "Hey stop this right now and report into whatever the instruction may be."

Ashish Rajan: Those are some of the... At least it's like what I'm taking away, it's possible to start doing the right thing even if you're not a mature organization, or at least quote unquote mature to where, where we started this conversation. Most people are in a state that they can start considering deception AI or no AI.

Ashish Rajan: Would that be a fair statement?

Andy Smith: I would say so, yes. I think the, quite recently, the Cloud Security Alliance they put out their, like, CISO guidance on, you know, the post Mythos strategy and, you know, they flagged deception as something that now has become a, a high priority item. So I, I do think, you know, this isn't just a vendor saying it, like, there are, there are third parties out there saying this.

Andy Smith: You know, this is something you should seriously consider. Yes, I mean, the, I, I would say your, you know, [00:49:00] your crown jewels, your most critical environments are definitely the place to start. I would totally direct you to our research. If you check at agentic.tracebit.com, you can, you can go read on that and see, you know, how we're able to use some of these techniques by triggering, like, the safeguards and the models against them.

Ashish Rajan: Yeah. Awesome. That's most of the questions we had, unless, Caleb, you have anything last that you wanted to ask before we wrap up?

Caleb Sima: Yeah, you know, I, I feel like if I were to also tell, you know, our listeners things, it's, I would pay attention to deception And I would learn and educate myself a bit more about deception strategy.

Caleb Sima: Um, I think this is... it's already important, but I think it's gonna become more and more important as we move forward, and to think very similarly about deception as you think about detection and response today. It's just a key part that I, I feel has a lot of value that is being missed. Uh, that's my walkaway for everyone is, don't just blow away deception as the honeypots of the '90s, right?

Caleb Sima: [00:50:00] This is definitely becoming a way different technology for a way different time.

Ashish Rajan: Soun- sounds like you should put it in your mail list, uh, Caleb, at least one strategy.

Caleb Sima: Well, yeah, I, you- I post a lot, uh, about

Ashish Rajan: deception. Well, I, I would say that- I don't know ... for if people wanna-

Caleb Sima: That's how this, that's how this conversation came around That

Andy Smith: is exactly-

Andy Smith: how this conversation came around, yeah.

Ashish Rajan: Uh, where can people find Andy, your- yourself to connect with you and know more about Tracebit and everything else you guys are working on?

Andy Smith: Yeah, if you, if you check us out on, on tracebit.com, that'll have all the details you need and our research as well.

Ashish Rajan: Awesome. I would also put your LinkedIn link in there as well for people to connect with you- Yeah,

Andy Smith: yeah ...

Ashish Rajan: to kind of have a conversation. Would love

Andy Smith: to. Love to connect.

Ashish Rajan: But thank you so much for spending your time with us, Andy. Really appreciate the conversation.

Andy Smith: Thank you so much. Really enjoyed it.

Andy Smith: Appreciate the time.

Ashish Rajan: Thanks everyone. Thank you for watching or listening to that episode of AI Security Podcast. This was brought to you by Techriot.io. If you wanna hear or watch more episodes of AI Security, check that out on aisecuritypodcast.com. And in case you're interested in learning more about cloud security, you should check out our sister podcast called Cloud Security Podcast, which is available [00:51:00] on cloudsecuritypodcast.tv.

Ashish Rajan: Thank you for tuning in, and I'll see you in the next episode. Peace.

‍