Security Cryptography Whatever
Summertime Sadness
Are you going to be in Vegas during BlackHat / DEF CON? We're hosting a mixer, sponsored by Observa! We have limited capacity, so please only register if you can actually come. Location details are in the confirmation email. Tickets will be released in batches, so if you get waitlisted, there's a good chance you still get in. Looking forward to seeing you in Vegas!
Ticket Link: https://www.eventbrite.com/e/scwpod-vegas-2024-tickets-946939099337
We talk about CrowdStrike in this episode, but we know we made some mistakes:
- The sys files may be code in addition to data.
- The bug might be bigger than "just" a null pointer exception.
Luckily, none of that is actually relevant to the main issues we discuss.
Show page: https://securitycryptographywhatever.com/2024/07/24/summertime-sadness/
Other Links:
- https://csrc.nist.gov/projects/post-quantum-cryptography/post-quantum-cryptography-standardization
- https://dadrian.io/blog/posts/pqc-signatures-2024/
- https://dadrian.io/blog/posts/cto/
- https://www.blackhat.com/us-24/briefings/schedule/
- https://terrapin-attack.com/
- https://www.youtube.com/watch?v=-AqayGm0_pw
More like ClownStrike, amirite?
"Security Cryptography Whatever" is hosted by Deirdre Connolly (@durumcrustulum), Thomas Ptacek (@tqbf), and David Adrian (@davidcadrian)
Are you going to be in Vegas during BlackHat / DEF CON? We're hosting a mixer, sponsored by Observa! We have limited capacity, so please only register if you can actually come. Location details are in the confirmation email. Tickets will be released in batches, so if you get waitlisted, there's a good chance you still get in. Looking forward to seeing you in Vegas!
Ticket Link: https://www.eventbrite.com/e/scwpod-vegas-2024-tickets-946939099337
We talk about CrowdStrike in this episode, but we know we made some mistakes:
- The sys files may be code in addition to data.
- The bug might be bigger than "just" a null pointer exception.
Luckily, none of that is actually relevant to the main issues we discuss.
Show page: https://securitycryptographywhatever.com/2024/07/24/summertime-sadness/
Other Links:
- https://csrc.nist.gov/projects/post-quantum-cryptography/post-quantum-cryptography-standardization
- https://dadrian.io/blog/posts/pqc-signatures-2024/
- https://dadrian.io/blog/posts/cto/
- https://www.blackhat.com/us-24/briefings/schedule/
- https://terrapin-attack.com/
- https://www.youtube.com/watch?v=-AqayGm0_pw
More like ClownStrike, amirite?
"Security Cryptography Whatever" is hosted by Deirdre Connolly (@durumcrustulum), Thomas Ptacek (@tqbf), and David Adrian (@davidcadrian)
Hello, Welcome to Security Cryptography, whatever I'm Deirdre.
Speaker 2:I'm David.
Speaker 3:My wife Amy is sick.
Speaker 1:Thomas isn't going to talk very much and it's just us today. We wanted to visit before we all head off to Vegas in a couple of weeks, because we have some very important news we're having our first in-person, in-the-flesh event in Vegas for Black Hat USA this year, or Black Hat slash DEF CON, or whatever it's going to be Thursday, august 8th, 730 to 930, at a place on the Strip. We would love to see your friendly faces in the flesh from across the internet, but we have a capacity limited spot, so there will be an Eventbrite link in the show notes and please go to that link and fulfill our call to action and actually sign up so you can actually be part of the capacity limited crowd. And that's where you'll find out the exact location on the strip. And we are very excited to have a sponsor for our event. Observa is sponsoring our little event. Thank you, observa.
Speaker 3:Observa is pretty cool. Observa is Rob Picard and Luca Caratoni and John Villanelle all Montessano people. So Observa does security programs for startups. I don't think you could find three people that you would rather have as your security team for your startup than those three people, so Observa check them out. They're very cool.
Speaker 2:Yes, we are all very excited to have the event in Vegas. Thank you to Observa for sponsoring the event and please keep an eye on either the episode notes or our social media to get tickets to the event.
Speaker 1:Looking forward to seeing you all there. David did all the legwork to make this happen, so if it goes well, he gets all the credit, and if it doesn't, we will never speak at the event again.
Speaker 2:Moonlighting as a marketing event coordinator has always been my dream.
Speaker 3:It cannot possibly go worse than the Microsoft gathering, where they bust us out into the middle of the desert to wait in an hour long line for one drink and then, another hour to get bust back. It was the most amazing meetup I've ever been to.
Speaker 2:We can say this event is on the strip, it is easy to get to.
Speaker 1:Yes, we can say this event is on the strip, it is easy to get to, yes, and it is near some frequent locations of the kind of DEF CON affiliated hotel universes.
Speaker 2:And it's not far from the In-N-Out Burger.
Speaker 3:Oh good, we will not kidnap you At no point in this. Will you wonder whether this was an elaborate scheme to get all the security researchers out of the desert to kill them?
Speaker 1:Hopefully it will go better than the event. I don't remember who hosted it by, but it was at the Hustlers Club and I got to see John McAfee there before he died. That was fun, speaking of McAfee and things that are supposed to make your computer safer that's never a good transition.
Speaker 2:Speaking of McAfee that's never a good transition.
Speaker 1:Speaking of McAfee. There was a major international information technology event.
Speaker 3:I think y'all are taking this a little too seriously, this particular event.
Speaker 2:I woke up to a text from my mother about this outage.
Speaker 1:Excuse me, this affected me personally. I tried to order a Starbucks and their entire mobile ordering system was broken across the United States, as far as I could tell, because of CrowdStrike, aka CloudStrike.
Speaker 2:Important question what's your Starbucks order?
Speaker 1:Now it has evolved into a Trenta Berry Summer Refher, which is it's just the blue one. They got it. They introduced a new flavor. It's effectively blue raspberry refresher drink with caffeine in it, no bubbles, light ice. And then I usually get the feta cheese wrap with egg in it, or I get an egg white egg bite, or I've discovered that they have a cheese danish and I'll get the cheese danish if I don't want something protein.
Speaker 3:Anyway, is the Trento like the big gulp?
Speaker 1:Yes, it is like 32 ounces of drink, yes, and so I get like a. I get a big gulp worth of blue flavored like vaguely grape drink with a hundred milligrams of caffeine in it or something like that, because it's large anyway. So if you were under the rug of most of uh yesterday, we're recording on saturday the 20th yesterday, friday, july 19th, um crowd strike. Who operates uh, uh exploit detection and remediation system. Is that what edr?
Speaker 2:stands for endpoint, detection and response.
Speaker 1:So endpoint security, which is corporate antivirus and sometimes corporate configuration management yeah, I got one of the words right um, they are extremely popular on a whole bunch of operating systems but apparently they were uh, they were particularly uh hit on their windows systems for everybody on latest and probably latest minus one or something like that.
Speaker 1:I saw that somewhere. I don't know if that's true. But a lot of their deployed windows uh fleet, which is companies, organizations across the world all of them received an update not to the software but to a content, piece of content that gets parsed by their software and their parser is in a kernel module for the Windows kernel. And when your parser panics inside your kernel module loaded into the Windows kernel, guess what? Your fucking computer panics and reboots and reboots and reboots until you can get to it and say stop that. Please unload any foreign kernel drivers or kernel plugins or whatever, which is called safe mode for Windows, so that you can try to boot into Windows, remove the offending files, because it's just a file that a existing parser was trying to parse because it got pushed. You can remove that and actually try to restart Windows again safely. For a lot of people you had to go manually to these machines.
Speaker 3:As somebody who will never be able to run a deploy again without having a small panic attack, which is a consequence of spending four years now working on a public cloud. I have nothing but sympathy for the CrowdStrike people and also whoever ran that build is my favorite person in the world for a variety of reasons. People are nuts about what happened here and the consequences of this, and nobody can imagine how a company like this could have let this happen. This is the natural state of all software yeah.
Speaker 3:What's weird is that it hasn't happened 100 times before.
Speaker 1:Yeah, because it does seem like this is a parser bug that's been there for a length of time and it was just finally triggered by this. You know, whatever new rule, new like signature that they were trying to parse with their parser and it got triggered and the all zeros content thing they were trying to parse is one instance of something that can trigger this parser bug, which is like a null pointer dereference in their parser. Well, number one why is this parser living in a kernel plugin in the first place? A kernel driver? It feels like we've learned not to do that sort of thing. Living in a kernel plugin in the first place? A kernel driver? It feels like we've learned not to do that sort of thing, especially for less privileged stuff. But that's exactly what's happening here.
Speaker 2:I think, generally, you're right. There's a question. I mean, I'm no expert on what CrowdStrike's product is specifically trying to achieve, but you know the the issue here is less on like oh, automatic updates are bad. And more on like, why is there so much privileged code running in the kernel? If it could have been in user space, right. Like, how big is this kernel generally and could it be made smaller?
Speaker 2:Um, in this case, you know, I don't know what the threat model and the trust boundaries are around the CrowdStrike's driver. So, like you would think, oh, you know, just parse those malware definitions, you know, not in the kernel. Like, why would you parse it in the kernel? But what happens? Like, is there some way that you can verify that it was parsed correctly, right? Like if part of your threat model, is there some way that you can verify that it was parsed correctly, right? Like if part of your threat model is there's something running at user privileges, like do you want that thing at user privileges to be able to somehow then like, alter the output of that file and, like you know, replace the malware signatures with zeros or just delete them? Right, there's probably a way to accomplish this where the parsing is not done in the kernel and you can still reach that trust boundary. But I'm no expert on Windows.
Speaker 3:But your CrowdStrike. Here you're operating at a scale where malware is built specifically to defeat CrowdStrike right. Most security products, your adversaries, security products in general, things that you pay money for to make you more secure or whatever. No adversary in the world gives a shit about anything that you do. Crowdstrike is the rare exception to this right when, like, it is worth somebody's time to write like a couple hundred lines of code to defeat CrowdStrike. So if there's a thing you can do where you can like jam up you know a user land Windows service and thus keep it from getting updates or keep it from detecting or something like that Like this is not me vouching for the model of, you know, hoisting all of your security stuff into the Windows kernel.
Speaker 3:Like I don't think that's a great thing to be doing, but like you can see why they would do it that way, right, like they actually do have adversaries that are you know.
Speaker 2:Yeah, I mean the windows model. There's like user admin. Lots of people just run as admin. Um, that's becoming less common on windows 10 and windows 11, um, but more or less there aren't really like a lot of per process protections and those that exist are still like very hard to reason about in a way that, like other platforms kind of offer more. But windows is in a little bit of a bind here because there's such an old platform and they're so big, like if they say you know, oh we don't actually want things to work like this.
Speaker 2:We're gonna like shrink some of what you can do in a kernel module or change it like oops, like not letting third parties do something is an antitrust, and continuing to let third parties do things is, while Microsoft gets blamed when a driver crashes. So it's a bit of a rock and a hard place there.
Speaker 1:But also Linux doesn't have these anti-competitive hawks watching it, because they're Linux and not Microsoft Windows.
Speaker 2:They also don't have users.
Speaker 1:That's why they don't have anti-competitive locks. But CrowdStrike also works on. They have a macOS version but it doesn't seem like it's as capable and it seems to be slower on macOS because, I'm assuming, because of the way that the macOS kernel is architected.
Speaker 2:But I guess it's just not the same competitive base as microsoft and windows and maybe it has the same bug, and they just didn't push out a bad definitions file to mac, like or maybe it does and it just doesn't cause a panic.
Speaker 1:Or maybe the panic is caught, or me or whatever it is. It doesn't cause a kernel, a full system kernel panic, or something. If I were a kernel, I would simply not panic if I were a null pointer, I would simply not panic. If I were a null pointer, I would simply not be dereferenceable.
Speaker 2:What do you think, Deirdre? Does Rust fix this?
Speaker 1:It actually does not, because if you did everything, if you wrote this parser and you're still like pointing at pieces of memory via kernel boundaries and blah, blah blah, you're still dereferencing uncontrollable pieces of memory. I don't think Rust would actually help this.
Speaker 3:It totally would fix this. It has option types. You would never have a null to begin with.
Speaker 1:Oh, so it's not the memory safety, it's the nullness.
Speaker 3:We have lots of languages that don't have the stupidity of null, so yeah, you would have like a none where you expected a sum, and there's no way to dereference a none in Rust Aside from in rust, aside from the specific keyword used for doing that. Like explanation point, right like what happens when you panic in rust inside the kernel? Oh, it's a. Is that a kernel panic? It's a kernel module written in rust and it's all unwraps instead of matches.
Speaker 1:Yep, yep, excellent, yep. It's like, oh, we did it, we did the right thing, it's no.
Speaker 2:I mean, presumably you would like not do that a ton, but like I think the point is that a crash in the kernel module is still a crash, yes, kernel module, regardless of it's actually a null pointer, deref yes. Or if it's just like oh, the language, the language has saved me by asserting on this bounds check or this null check for me me, but oh well, once it started, like yeah, still crashed.
Speaker 3:Yeah, this has been our two minutes of rust humor. You're reading a lot of stuff about like what CrowdStrike is, um, and like what EDR things are, and it's like it's all very simple, right? This is just what people run in 2024 instead of antivirus. So you know compliance regimes and like IT security practices. Like you know, if you look at a fortune 500 company, they've had like like a large scale IT security practice for the last 30 years, right, some of this is just logic that is handed down from generation to generation that something has to be running on every endpoint.
Speaker 3:It used to be, you know McAfee, you know AV or Symantec AV, and now it's CrowdStrike and honestly, you're better off now by a lot. You would much, much rather be running CrowdStrike, where they every once in a while push out a build that crashes every Windows machine on the planet, than running AV engines that were all written in the late 1980s. There's so much chatter going on this is the biggest thing that's ever happened for itinerant in the last several years, so people can't shut up about it and everyone keeps saying that this is what y2k was gonna be like, if y2k actually happened, and I don't know if that is actually so extremely mild I don't know if this is quickly.
Speaker 1:Everyone say your age at y2k um, unfortunately, like, maybe the impact like one time impact is at the large scale that we were afraid of for Y2K, but the remediation is quite easy, like quite easy compared to Y2K. Like Y2K basically required the most critical pieces getting rewritten, the code getting rewritten versus this is just like there is a step-by-step thing you can do. It's not fun. You have to go find your host and go into safe mode, remove a file and then reboot it 10 times perhaps, but we know what to do. You have to rewrite all your software. So I don't know.
Speaker 3:I have a bone to pick here and I think David will appreciate this particular bone I have to pick here. I'm again dragging the orange site into this conversation, although this particular orange site drama links back to Twitter, so it's also Twitter drama. But some enterprising souls have uncovered the interesting detail that George Kurtz, the CEO of CrowdStrike, was at one point the CTO of McAfee the quote unquote CTO of McAfee during an era where McAfee also pushed out a bum updater that broke everybody that was running McAfee. And my problem here is that no one knows what the hell a CTO does. So there's like a. I can't get the picture out of my head that these people are imagining George Kurtz sitting there on his computer writing software updaters. So, david, what does a CTO do?
Speaker 2:I mean it depends. But, um, I would say CTO is mostly like not a real title and it is actually just like a title that you have when you're in a different role but you need to sound cool. Those roles being technical co-founder, which is a perfectly valid form of CTO and arguably the most valid form of CTO. It's like you have someone at the company who has a lot of equity, a lot of institutional knowledge and is kind of driving the technical product direction. Um, that's you know, your technical co-founder. There is like cto as in is actually just a vp of engineering, in that they're like in charge of engine execution. I would argue like you should get your title to be vp engineering in that case again, if you are a vp of you're not writing a software updater, no.
Speaker 1:And then there's like sufficiently large company, or a parser, or whatever.
Speaker 2:Yeah, absolutely. The CTO that is actually a VP Eng is like not writing parsers right, they're looking at Jira tickets and they're writing OKRs. And then the other case is like you're a big-ish public company and you kind of need someone at roughly the C-suite level to go around and talk about technology to other people at the C-suite level, and that person is usually someone who used to be a VP-Eng at somewhere big. So, for example, like the CTO of Microsoft, I believe used to be the VP-Eng at Salesforce. I have a simpler breakdown of CTO of Microsoft, I believe used to be the VP at Salesforce.
Speaker 3:I have a simpler breakdown of CTOs than you do. They're equally valid breakdowns. They're both worth hearing right. You have the OSI model of CTOs and I have the IETF model, and I want to caveat this by saying that none of this applies to the CTO of Flatio Jerome, who I adore and is amazing and also spends all day and all night rewriting every line of code that I've ever written. So this is definitely not him. But my general breakdown of CTOs is that there's three types. There is the type where it is the consolation prize that the founder gets when they take away his commit privileges. Finally, when there's a real engineering team. That's the first kind. And then there's the kind of cto you are when you are the ceo of a large company that gets bought by another company, where they're not making you the ceo, they have to give you a c something title, so it's cto, which basically means the seat, the ceo of the product line that is now part of this bigger company. And the third kind of cto is the sucker who took the cto title instead of pushing for more compensation at a small company that shouldn't have a CTO at all. Those are the three kinds of CTOs there are. Right and like.
Speaker 3:If you ask me what the most important job skill of a CTO is, it is identifying themselves as the CTO on a call with a customer prospect. That is it. That is the CTO skill. That is what they do. The idea that a CTO is doing any kind of technical work whatsoever People think they're like. It's like they're the lead programmer in the company, it's like chief scientists, like they're doing the most science. It's the same thing. Right, it's a title that has absolutely nothing to do with what the, in my experience, has absolutely nothing to do with what the title sounds like. So, no people, I don't think that there is much of a George Kurtz connection with the Broken Up Daters.
Speaker 1:Now we have to do CISOs and CSOs.
Speaker 2:Our advice on that one is never accept that title even if that's your role. Just get any other title.
Speaker 1:Yeah, the connection between these two. Any incident with McAfee and any incident with Krautsark is like coincidental at best.
Speaker 3:I mean, if you do get the CSO title, just remember, like the Mesoamerican cultures where, like before, they sacrifice you you get like a couple of days of everyone just treating you really awesome, and you get to eat whatever you want Just make sure you're getting all of the stuff that you can possibly get before the day of sacrifice.
Speaker 1:Absolutely, you are there to be fired when there is a security incident. That has traditionally been the rule. Updates, whether they are actual code, patch updates to you know your deployments, or configuration updates, that can have almost as much or equally as much impact, because they tell they are executable instructions by the executing software or whatever, which is basically what happened in this case. Um, but like how this thing, as far as we can tell, the configuration update or signature, whatever it is, got pushed out, uh, between 4 am and about 5 30 am UTC yesterday and it was all zeros as far as we can see, and it like people have claimed to replicate uh, the panic with you know, uh, content that's being parsed, it's not just all zeros. How do we prevent problems like this? Well, the problem is that we don't have the procedures in place and people in organizations that allow this problem to occur, not now there's a meme about this, yeah it takes a long drag on a cigarette.
Speaker 1:It's like, yes, the root cause analysis, your five whys, whatever you want to call it, the blameless postmortem will eventually zoom in on. There was a pipeline procedures, pressures, checks that either were in place and not followed, or they were all in place and followed and were insufficient. Or there was a bug in the pipeline, in either the CI testing pipeline or the CD deployment and packaging pipeline. And you know, or you know, someone claimed on the internet like oh, this was supposed to only go to, you know, the client staging environments and it overrode that. Or you know, we don't know enough now there's a lot of speculation, but you know the client staging environments and it overrode that. Or you know we don't know enough now there's a lot of speculation.
Speaker 1:But you know, having a thing that's basically all zeros get pushed out feels like something that we should be able to flag and it's unclear whether it was built and tested and passed all the tests with all zeros, or if it passed all the tests and then it got built and after that it turned into all zeros and nothing would catch that or whatever. So I'm very curious, I want to learn more and, like we'll see if there's anything that we'll learn publicly about this. But I'm very interested in sort of like the pipeline, the procedures of how this gets out into the wild in general. Um, because that's where I was zeroing on and trying to figure out how to prevent stuff like this from happening.
Speaker 2:The pipeline procedures question is somewhat interesting, I think, and we'll definitely have to do it for regulatory or legal or even just probably soft price reasons. But I actually don't think it's the the thing that matters most, in the sense that, um, uh, you have this question with any system that like has various checks in it of just like, oh like, if this one thing like isn't working, like what is the expected behavior? What a failure rate of like some given part of the system and like, just because it was caught, like he had a failure over in step one but it gets caught in step three, like how bad is that failure in step one? Um, and so like there's shortcomings there and that this failure wasn't caught. But there's another question.
Speaker 2:I'm just like parsers are the easiest thing to fuzz and test right like so there's one side of like don't parse in the kernel if you can avoid it. But there's another side of like if you're writing a parser like, we know how to do that.
Speaker 2:Well, like the cryptographers are pretty good at this, like parsing, like major browsers still parse X509 certificates in the browser process because like have to cryptographers have got, because one they have to, there's nowhere else to do it and two, like the cryptographers have realized like how to spend the right amount of effort to make a parser that is like up to a standard that is kind of higher than the parser standard might be in other situations, but the reality is like of the code that you could write at a like very high level of reliability. Parsers are like the most straightforward thing to test and to fuzz and like sure you can go into all of these policies and procedures about the deployment process.
Speaker 1:But like I would almost rather see them skip that and just be like we have fixed our parser, which is why I want to give them the benefit of the doubt that they did take this change of this config, tested it against their parser and then it passed. And then they built it for deployment. And then they built it for deployment and that's where. And then something went wrong in the build, the CD process, the packaging process, the release pipeline, not the testing pipeline. But that also means that you have to test twice or you're testing in the wrong place, because if you're testing before build and it passes, and you test after build and it doesn't pass, then you've got a problem. Um, but I don't know that much. But I generally agree with like we know how to make, we know how to test and write parsers better now and it feels like this one is like an extremely non-risk, tolerant parser. So, yeah, I don't know. I'm very curious.
Speaker 1:There's been debate about like why this is written this way anymore. Because Windows, in later versions of Windows, they have made safer hooks via the kernel or via that surface available to software like crowdcirc. I think this was like their falcon sensor, um to use in a safer way. Um, and like why aren't they doing that? And it's like well, I have a feeling that this is deployed. This is deployed to hospitals. This is deployed to airline systems. This is deployed to airline systems. This is deployed to fucking Starbucks, like. This is deployed across a lot, a lot, a lot of diverse Windows instances. I think that there's a lot of them that aren't up to speed in making those hooks available and so they're doing it like kind of the tried and true and available way.
Speaker 2:Yeah, in terms of, like, diverse deployments, though um I I moved recently and um I'm trying to get my car title. Um, because I need to like register at a new place, and so I went to log in to my car loan portal and it was down. I was like man, I all the things to go down because of Crab Strike. I did not expect Mazda Financial to do that.
Speaker 1:It wasn't even like the local municipality, like you know Secretary of, you know Department of Transportation.
Speaker 2:No, it was not the DMV, it was like the car loan portal, where I wanted to go to pay off the loan so that they would send me my title, Because that just felt easier than like requesting they send a title to the DMV.
Speaker 1:So one last thing before we move on from this is like why, like I thought Windows has their own built in like AV security thing, like Windows Defender or you know equivalent for maybe something? I know there is one on Mac OS and I'm forgetting their name right now so why don't we just all use like Windows Defender? And why is that not fine and sufficient? And I don't have a good answer, except that CrowdStrike does a lot more and is a lot nicer to use on the backend as like an operator of an EDR solution, and like Windows Defender is just not enough. And so there's a reason that crowd strike has a market, because this podcast does not endorse any individual antivirus product no, I don't like no, but like.
Speaker 1:There's a reason that markets have like found like a solution, and it's that windows defender did not just completely be like solved it. Just use this for free. It comes with your windows and, like what thomas says, like you run it, you run CrowdStrike in 2024. Instead of back in the day you would use McAfee antivirus, because that's what you do. And when the built-in stuff into your OS is not sufficient, as a real operator and instead of talking about compliance, let's talk about literally anything else. Do you want to talk about post-quantum standards compliance?
Speaker 2:I'd like to talk about how there still aren't any post-quantum standards. It is difficult to comply with compliance guidelines that provide deadlines in terms of when communication needs to be post-quantum secured, if the algorithms that are post-quantum have not even been standardized yet yeah, but let alone whether or not they're suitable for use in the real world yeah, we have a draft.
Speaker 1:We've had a draft for a year. We're talking about the NIST, the United States National Institute of Standards and Technology, the post-quantum cryptography competition that they've been, or whatever. They don't call it a competition, whatever the thing, the thing that they've been running for eight years now at least. Um, the standards, the official fips standards for ml chem, which is a new kind of key agreement based on lattices. Mldsa, which is a new signature based on lattices, and slh. Dsa, which is stateless, hash based signatures. Uh, which is signature based on lattices, and slh. Dsa, which is stateless, hash based signatures. Uh, which is not based on lattices, it's based on signatures, are supposed to drop any day now and it's all just like rumors and whispers and just sort of like people sending emails to anyone that might have an in at nist.
Speaker 1:I've been refreshing their website every fucking day and just like watching and preparing and just waiting, because they signaled publicly that they might be ready by end of summer, slash September, and then I think someone from NIST was on stage somewhere and they were like no MLChem document is done. We finished it and we've sent it up the chain and we didn't hear anything about the other two, but we think they're all going together in one fell sweep. Falcon is not done yet. They've told us that falcon is another lattice related signature scheme. That's not done yet because it's harder. I think that's coming later.
Speaker 1:But these three it's very possible that they will drop any day now and as soon as they drop it's like a starting gun for, like lots of people in industry, especially anyone that has to be compliant with, like these memos from the United States executive branch about being compliant with the systems that you sell and deploy to the federal government. But also there's all these other systems of like. If you want to be FIPS compliant, if you want to deploy the software in a national security system they've already outlined like, you must have these xyz things it's and they have to be done by like 2030. Some of these deadlines are like real soon for the like big infrastructure and organizational deployment lift yes and no, so the part of the problem here also is the number on the page that.
Speaker 1:Yeah, there's the number on the page.
Speaker 2:Yeah, there's a number on the page, but the page isn't actually a compliance document. So the one that people cite a lot is the DoD that has this document that says this is what the new CNSA guidelines are, the algorithms for computer network security, or something like that, which is basically like For national security systems.
Speaker 2:Yes, yeah, which is basically what are you allowed to use to secure top secret data, and they're, like you know, we are going to switch this to require a post-quantum, which if you buy the you know the post-quantum threats. Then like that's a very reasonable thing to issue a document saying the next version of this 2.0 will look like this and a little bit of guidance on like prioritize, you know, uh, store now to encrypt later over signatures and like here's how like this interacts with, like web browsers and here's how this interacts with like long-term storage and software and firmware signing. And then it provides some some like suggested dates, but like one this document is non-normative.
Speaker 2:It literally says this document is describing how we will decide the process.
Speaker 1:And they're like.
Speaker 2:It will be based on standards from NIST. It will be based on these threats and needs.
Speaker 2:That are different priorities and so on, but it had some years in it, including things like 2025 for uh browsing traffic and it's like well, if nist hasn't released a single final version of the standard in mid 2024, like you're not gonna by default have like uh, post quantum signatures, um, anywhere in 2025? Um, but notably, the 2025 year is just a recommendation in this document. And then it says 2030 just a recommendation in this document. And then it says 2030 is the deadline. But this document isn't even the deadline because it's a non-normative document. So all this to say, the only thing that matters for actual migration to post-quantum is that NIST standardizes algorithms and then from there people can work on solutions. And from there people can work on solutions. But as long as these algorithms are not standardized, there's like very limited cases where they can be used in anything outside of like an experimental context.
Speaker 1:Yep, and I will. I will point to yet another document, that's the OMB memo, which does say goal of my mitigating as much of the quantum risk as is feasible by 2035. Quantum risk as is feasible by 2035. So that's like different, because that's for different systems in the us government than these national security systems that the the thingy from cnsa20 is talking about. But that gives it a little more oomph, but it is still like try to mitigate years.
Speaker 2:It's like there's a big difference between 2030 and 2035 when it comes to like trying to migrate the web.
Speaker 1:Yeah, yeah but what you ended on is is absolutely true is like we can't we literally can't like make progress until we have a final standard of like the thing that everyone is pointing to, um, including all these things in the us government, but also countries around the world are watching this, you know process, this NIST PQ process like proceed and they are keeping an eye on it as sort of like, yeah, we're going to like, you know, take, take suggestions from that and maybe we'll add a little bit more to our stuff.
Speaker 1:Like a lot of the governments in other parts of the world are much more comfortable with hybrid solutions for like their systems than this NSA, cnsa document for national security system, which is basically like no hybrid. If you can help it, we want it to be pure PQ, which is a big vote of confidence for things like MLK, because they're saying, like MLK, m10, 20, 10, 24 will be like by itself, sufficient and required to protect a key exchange for, you know, top secret, you know communications or storage or whatever. Um, other governments are like eh sure, mo cam and frodo or mo cam and I don't know, and true, or something like that. I don't want to spend the whole summer just sort of waiting, waiting, waiting, especially because I thought we would get it at the end of the summer and like no, it's gonna be like any day.
Speaker 2:Now it's like fuck, just do it already there's another memo from omb, m2302 I believe, that says um to like. It requests that agencies basically inventory their post-quantum risk.
Speaker 2:Um in terms of like what systems would be affected by a quantum computer and if this memo percolates down to you, if you're like at an agency, deputy cio or whatever, my advice to you is to ignore the condition and just send back a list of all of your computer systems. I think, uh, the practical matter is that it will affect all of it and nothing is standardized yet. So, instead of trying to ask people whose job is not to think about quantum computers and cryptography, if you get this memo, just ask for a list of all of your computers.
Speaker 1:And then it's like cool, we have an inventory and now what? Now, what do we do? And like people are not asking that question, they're sort of like what, what's the next step? How do we remediate this thing?
Speaker 2:that's a good question that not a lot of people have a straightforward answer to I think we have talked I don't want to say extensively, because I don't think we've done a full like episode on this but I feel like we talk about this every other episode because it's like related to the day jobs of several of us here, in form where we're just like oh, all of these transitions are hard and there are no good solutions and these algorithms are very big, yep, but like, a bunch of that discussion is even just predicated on like can we get a final version?
Speaker 1:of the standard? Yes, especially because, like there's the ongoing signature on ramp, so like they're going to drop MLDSA, which is like a digital signature that one could theoretically replace a bunch of stuff in the web PKI with, but it's so big, the keys are big and the signatures are big. Slhgsa is probably good for signing firmware or things like that. It's not good for online signatures and putting them on the wire, for example, or in your like end-to-end encrypted communications or anything like that. Mldsa is only one of these three drafts. Falcon might land later, but it's got some sharp edges that it's unclear if people will be happy with. Even that is still kind of big. Will be happy with. Even that is still kind of big.
Speaker 1:Um, the on-ramp is supposed to find more signature algorithms that will give different security guarantees. So maybe not based on lattices, it'll be based on something else, but all of them, except for ski sign in the on-ramp, are still too big. They're still too big and like that's gonna take more time. So, like one, we want these things to land period, but two, when they do land, we still have a lot of hard work to do to figure out how to make these things work in things like the web pki, because none of our options are very good and it's going to be some hard architectural design, organizational migration decisions, designs and planning and coordination to figure out how to make any of this work. But that doesn't significantly degrade the experience that that users of the secure Internet have come to expect for the past decade.
Speaker 2:I think let's go back to Vegas for a second and talk about the schedule. Black Hat, I don't know, maybe you will stick around for DEF CON. I don't believe in DEF CON.
Speaker 1:Okay, why don't you believe in DEF CON?
Speaker 2:I like conferences where I can get coffee and water in the hallway. That's not really a thing at DEF CON and that's not really a thing at DEF CON. And also, you know I'm on the review board for Black Hat, as is everyone else on this podcast and so we kind of have a little bit of a connection there.
Speaker 2:So I tend to go for Black Hat if I go at all, I think we'll all be there for Black Hat, and so let's I don't know maybe highlight a few things that we saw on the schedule for 2024.
Speaker 1:We wanted to talk about this more in depth earlier in the year on the podcast, but this Terrapin attack on SSH, breaking SSH channel integrity by sequence number manipulation this is pretty cool. This is very cool.
Speaker 3:This is the best this attack is. It's not Drown level for me. Drown is still my favorite applied crypto attack, but this is it's. It's not drown level for me. Drown is still my favorite applied crypto attack but, this is still as one, so like when I started, like when I first started engaging with cryptography stuff, maybe like the late 90s or whatever you run into like um, like ssl3 and like the ssl transcript hash and I was in my like my 20s or whatever, or my early 20s, when this was I'm old, but like, so like if you you're a normal programmer your first experience of dealing with a serious cryptographic protocol, and Paul Kocher designed SSL 3.
Speaker 3:SSL 3 is terrible, but that's mostly because we didn't know as much as we know now.
Speaker 3:But, a serious cryptographer designed most of SSL 3, right, your first experience of it is like okay, so you're doing a handshake, but at the same time you have to do a running hash over all of the components of the handshake, which is a giant pain in the ass and it's like at the time it was hard for me to even get my head around the level of complexity of running a hash over multiple it wasn't chained, it wasn't what we have in TLS 1.3 now, which is sort of like updated chained hashes.
Speaker 3:I'm saying these things out loud and so, like you know, I can throw together like a noise implementation and, like you know, in a couple of hours now, right, but back then it seemed like just moon science to me.
Speaker 3:Right. So Terrapin, right, so there's, you can think of like two lines of major transport, major encrypted transport protocols, right, like there's the tls line and there's the ssh line of protocols and sometimes they have the same vulnerabilities and sometimes they don't, right, so in the original ssh 2.0 design, the first serious, like sl3 is the first serious tls or the first serious sl protocol, and sh2 is the first serious ssh protocol. Right, they do not do a transcript hash like a full transcript hash over all the messages that happen, right, um, so instead what they do is it's kind of the same deal where you're just building up to doing a dh, right, you're just going to do a diffy helman and do a key exchange. So they look at this I'm putting myself in their heads, but I wasn't in the room, but I'm pretty sure I'm right about this they're running a handshake to do a DH. So they look at this whole thing and they say, well, here are the things that contribute to the DH that we're going to do here's all the things that matter.
Speaker 3:So what we're going to do is, instead of hashing every message that happens in the transcript, we are going to just hash the things that could ever hit the key exchange, right, which is like what could go wrong? Right, and it turns out like so you can't influence the key exchange, but what you can do is throw off the sequence numbers now right, because they don't go into the key exchange anymore which?
Speaker 3:means you can like pad out like you can, you can eat messages.
Speaker 3:Basically, you can like, you can snip out a message from and like the. The only problem with this is that like and I haven't looked at this in a little while, so I could be wrong about this and maybe they're going to get up on stage and show like a really, really awesome outcome for this. The only like this is like as, as cryptographic protocol things go, this is I think this is a pretty devastating attack. Like this is not a great property for a cryptographic protocol to have, but I think in practice and deployments it doesn't matter a whole lot. So SSH has, after the handshake, like option messages that you can send to like ratchet up security or add restrictions or whatever.
Speaker 3:I forget what the options are, right, so you can snip those out, as I remember, but like nobody uses them, so it doesn't matter, I think is the outcome. But still, like that is the thing you could do, because they're only um, they're only hashings, they're only doing authenticity for the things that matter for the key exchange, but not for, like, the meta stuff. For that runs the protocol. You can fuck with that stuff. It's a really neat attack.
Speaker 1:Mm-hmm.
Speaker 1:And this smells like things that were optimizations in protocol design back in the day in the 90s, early 2000s, where they're like we only try to hash or authenticate or do extra work over the things that absolutely need quote, need to be done, such as, like just the, the diffie-hellman, the things that will influence the value of the diffie-hellman handshake, but the teal, like especially in tls1-3, like they've basically been like fuck it, like we, it's really hard to let some things be manipulable by an attacker and other things not be.
Speaker 1:Like the whole transcript needs to be like chained together in a, like you know, a hash tree. Basically like you do, every, every handshake, every piece of the agreement is getting hash, and then every new thing is also getting hash and also getting hash. And you know they've even gone farther just recently with encrypted client hello, so that there's even more information that is not even visible, let alone manipulatable by an adversary when you're trying to set up a session in TLS 1.3. And this attack smells of like we learned these lessons the hard way and fixed them in TLS 1.3 and other places in TLS and they haven't quite percolated down into SSH.
Speaker 3:What gets me here is that this is just such an obvious difference between the TLS protocol and SSH, right, so it's like this was discovered relatively recently, right yeah, I think at the end of 2023 is when that preprint came out, or whatever, right yeah, but that huge distinction between the two protocols, betweenaci and TLS, has been there for decades, right, and it took until now for someone to look at this and say wait a minute, these are not the same protocol. What is the consequence of these not being the same protocol? And the consequence was that the security of the other protocol fell apart. Right, so I just it makes me happy to be in a world where, at any moment, something like this might come out of somewhere else.
Speaker 2:Can I say something slightly embarrassing, which is that when we were working on the weak Diffie-Hellman stuff in grad school like the logjam attack.
Speaker 2:So we were looking at TLS, we were looking at Diffie-Hellman group parameters and other protocols and I was tasked with figuring out what root parameters SSH uses. And also go look at the SSH protocol and figure out if it has the same protocol vulnerability that logjam represented in TLS. And I looked at that handshake for a while I was like no, this is a good handshake because, unlike in TLS, it's got a signature over the full um set of key agreement parameters and I missed, like what turned into terrapin.
Speaker 2:I was like, oh, it's got the signature, so like we can't strip the thing that didn't have the signature in tls and then moved on, but I just I missed it, um, but I don't know. I also wasn't. You know not to like come up with excuses, but I was primarily interested in implementing the handshake to get a scanner out of it because, at the end of the day, as mentioned repeatedly, Thomas isn't a tax person, Deirdre is a real cryptographer and I am just a network engineer who happens to like reliable systems.
Speaker 3:They should credit you on stage for leaving that bug there for them.
Speaker 1:I'm the cryptographer that likes to build the good thing. I'm still not very good at writing proofs yet, but I'm okay with that for now. Yeah, I mean, you were also looking at weak DH and you were looking at these things that influence the actual Diffie-Hellman value, which is protected. It was the part of this that is protected, and the sequence values and these other optional messages were not protected. So I don't blame you at all. You were looking for something quite specific and it was protected against that. But if you were just sort of looking around at what else you could notice in it, maybe you would have saw something like that. But it's OK.
Speaker 2:Our approach we were. Sometimes we collaborated with people who did like actual formal modeling and proofs. But the way that we got into ground, for example, was just one day someone mentioned that sslv2 was like still enabled by default on a bunch of stuff and we were like that's probably buggy. And then we just applied the feynman method to open ssl's code. We're like there's probably something here. We just stared at it then we were like, ah, we have found like logic bugs, um, not cryptography bugs oh, yeah, and we were like.
Speaker 2:So then we like contacted open ssl and they were like have you met these other people that found related cryptography bugs? I think you might. Your stuff might help them so and we took a quadratic attack down to a linear attack because we just stared at the code and found bugs with pointers okay, so you found, did you find, a buggy implementation of a good protocol, of a good we?
Speaker 2:found a buggy implementation of a bad protocol and that's lv2 with drown, but yeah, okay, but we found the bug and we found the bug in the code, in the implementation, and then, um uh, like nimrod avaram found the bug in and his collaborators found the bug in the cryptography. And then when you put them together it made the attack on the cryptography much more efficient.
Speaker 1:Oh God, this is the shit that keeps me up at night, because all of this shit is so hard. It's literally like writing down something that's non-buggy and then translating it into code in a non-buggy way, and then you might have a different bug over here and another different bug over here, and then you combine them and you get something that is completely just, and I hate it. This is such hard shit and I hate it. It's all hard and I don't know.
Speaker 2:If you're familiar with the Matt Blaze attack on like physical keys, where you target one cut at a time. That is basically the bug that we found in the open SSL implementation, which let you target the key one byte at a time as opposed to all of it at once.
Speaker 3:Remind me of the Matt Blaze attack.
Speaker 2:So in a master keyed system, if you have access to a like a, just an individual key for a single door.
Speaker 2:This isn't quite accurate, but more or less the master key will have one cut and your key will have another cut, meaning the like various points would be at different heights and you know the height of your key. So that means that for every individual cut you can, you just need to find the cut that's at the other position. So if you can duplicate, if you have there's 10 cuts in a key and you duplicate your key and you make nine of them the same, and then you try all 10 values for the one cut that you've changed, you'll learn oh the master, my key is at height four, the other valid cut is at height eight. Therefore the master key must have height eight. And then you do that for each, for each um cut in the key, and so it takes what was like an exponential attack yeah, like let's just guess all the cut heights to becoming linear in the number of cuts so tai dong called that the hollywood attack.
Speaker 3:Right, but that's my mental model of most actual cipher attacks. Is Tai Duong's Hollywood thing? Which is it's Hollywood because if you do printfs while you're debugging the attack or whatever, you'll see the string of gibberish like the actual ciphertext and at the end you'll just see the rotating letter as you figure out letter by letter with the last one. So that's how Beast works. For instance is how the CBC padding oracle works. If you do printf there it'll look like it's working it out one byte at a time, like in the movies, because that's what you're doing. Right is bringing the attack down to guessing one out of 256 values, instead of, you know, one out of two to the 128th right.
Speaker 2:So okay, cool.
Speaker 1:That's very cool. Anything else on Black Hat's schedule we want to highlight? Well, just note.
Speaker 2:I think the network security track is quite good this year. There's several interesting things in it. There's attacks on DNS servers I don't know if that involves DNSSEC or not, thomas, but if it does, I'm sure you'll be there to laugh there's exploiting VPNs, which is always a fun reminder that your security appliance might make you less secure. And there's stuff about vulnerabilities and rpki validation, which you know might actually start to become relevant as rpki deployment becomes more of a thing, so I think it's interesting to look at it from that standpoint what is rpki?
Speaker 2:rpki is the pgi used to secure BGP router communication so that you can't spoof routes.
Speaker 1:Yeah, that's important. And yeah, why is it different? Why is it Never mind?
Speaker 2:Well, I mean, you're not authenticating domain names, so like something. The authorities and things have to be different, but it's still just. You know, x.509 certificates.
Speaker 1:I think, and signatures. It's different, but not significantly different.
Speaker 2:It's another PKI, because it's another youth case.
Speaker 1:Yeah, okay, all right, there's some keynotes. Jen Westerly, who's running SISA, is the primary keynote.
Speaker 2:Jen Easterly, not Jen Westerly.
Speaker 1:Fuck, perfect, perfect.
Speaker 2:Yeah, we're keeping that one in. That's an incredible fifth. Keep it in that's perfect. Um, yeah, the wall luigi uh, they're the primary keynote.
Speaker 1:They're like the whatever main, whatever main stage, whatever you call it um, there's a michelob ultra arena is it really is this?
Speaker 2:at least that's one of the keynote arenas. It's still.
Speaker 1:Yeah, it's still mandalay bay, it's all mandalay bay. Yeah, the michelob also.
Speaker 2:Yeah, that's the one that's like the fight stadium right like, if you think about like oceans 11 and the fight night, right, it's like that, and that's a different casino but oh, something's there is, is, uh, is um.
Speaker 1:What's his fucking face? Is Mike Tyson going to kill one of those YouTubers in a fight there in the near future? They'll draw.
Speaker 2:No, but if you go one door over to the Luxor, you can go to the eSports Arena and you can also see America's Got Talent live.
Speaker 1:I do have a soft spot for the Luxor just because I like pyramids.
Speaker 2:Anyway, not pyramid schemes.
Speaker 1:I like pyramids. Anyway, not pyramid schemes, just pyramids. There's another keynote, a fireside chat with Moxie Marlinspike, with Jeff Moss, who's the guy that runs DEF CON and all that stuff. What's his handle? I forget his handle.
Speaker 2:DarkTangent. Yes, darktangent, I don't know if he still runs it, but he definitely started both DEF CON and then Black Hat as like the corporate version of DEF CON.
Speaker 1:Yes, def CON was absolutely first, because they needed a place to send their people with budgets to learn about all the tips and tricks from the hackers at DEF CON, and they couldn't do it via the DEF CON name, so they created Black Hat and that's why it's very different in vibes, even though DEF CON is massive now. And yeah, I guess they're going to talk about privacy and security and other things like that, and Moxie's moved on from stuff with Signal and he's been doing some other cool adventures, so I'm interested to see if he's going to talk about that sort of stuff.
Speaker 2:But let's see my other advice for black hat is well. So, like the first time I went was when I talked back in like 2016 about drown and um cool I was like oh, I don't want to go to this vendor floor.
Speaker 2:That's for. You know these, like I, like I do real work, I'm not going to the vendor floor. Um, uh, now it is. I I strongly recommend that if you're there, you do go to the vendor floor. Um, if not, not to buy anything, that's like a terrible idea to buy something, like at one of these. But it's just interesting to see, like, what is going on. Like how are people marketing things? Like? What are people, what are the things that people are trying to sell? Because we're a very tactical-focused podcast and we have perhaps a different perspective than if you're actually responsible for securing an organization. What are the people that are being sold to there? How are these being marketed? What are they going?
Speaker 1:after.
Speaker 2:A good chunk of it doesn't make sense. Um, a chunk of the people in the booth like won't actually know anything about, like what it is that's going on, but it's still very interesting. And also like you can get pictures with f1 cars because it's ridiculous and so I do strongly recommend that you actually go and walk around the vendor floor at least a little bit.
Speaker 1:Strongly recommend that you actually go and walk around the vendor floor at least a little bit um, let's remember, um, you know, all those people are, in theory, making money, that's true, oh god, yeah, I've avoided it like the plague, but that is a very good idea. Um, I, I should definitely canvas for, like, if people are, if people are shilling for PQ migration or just quantum or quantum key distribution and like those things are very, very, very different.
Speaker 2:I also recommend that you be careful about if you do go to the vendor floor, like consider what you put on your badge about. If you do go to the vendor floor, like consider what you put on your badge, um, because if you're not representing your employer on the vendor floor um, like in a booth or whatever, but you do want to go talk to other things, you don't necessarily want to like reveal that who you work for, especially if you could be considered a competitor someone that you go talk to um or just because, like, if you have a sufficiently large company on your badge, everyone will try to sell you something and you probably don't want to buy it or don't have buying permission for it so street smarts those are some black hat street smarts.
Speaker 1:I don't know. I'm like that, um, anything else. I'm going to defcon. I'm going to defcon for at least like a day and a half, so I'll see you around the defcon, um, probably. I like to go check in on the Crypto and Privacy Village, and maybe there's also a newish Quantum Village that's been around for one or two years and they do a lot of post-quantum stuff sometimes, so maybe I'll see you around DEF CON. Awesome, defcon Awesome.
Speaker 2:Alright, well, once again, keep an eye out on our social media or the episode notes for details on the get-together in Vegas. Thank you again to Observa for sponsoring it. We're excited to see everybody first or second week of August, depending on how you count, and have a great summer.
Speaker 1:Yeah, security, cryptography, whatever is a side project from Deirdre Connolly, taz Tachek and David Adrian. Our editor is Nettie Smith. You can find the podcast online at scwpod and the hosts online at Durham Crustal Loans, at TQBF and at DBC Adrian. You can buy merch at merchsecurityphotography or whatever dot com If you like the pod. Give us a five-star review on Apple Podcasts or wherever you in Vegas. Bye.
Speaker 2:Also. We're on YouTube now.
Speaker 1:Yeah, we are on YouTube.
Speaker 2:But not with video of us, just audio. We're on YouTube.
Speaker 1:Hello, welcome to security cryptography, whatever I'm Deirdre.
Speaker 2:I'm David. My wife made me sick.
Speaker 1:Thomas isn't going to talk very much and it's just us today. We wanted to visit before we all head off to Vegas in a couple of weeks, because we have some very important news. We're having our drumroll first in-person, in-the-flesh event in Vegas for Black Hat USA this year, or Black Hat slash DEF CON, or whatever USA this year, or black hat slash DEF CON, or whatever we have. It's going to be Thursday, august 8, 730 to 930 at a place on the strip. We would love to see your friendly faces in the flesh from across the internet, but we have a capacity limited spot. So there will be an event bright link in the in the show notes and please go to that link and fulfill our call to action and actually sign up so you can actually be part of the capacity limited crowd. And that's where you'll find out the exact location on the strip. And we are very excited to have a sponsor for our event. Observa is sponsoring our little event. Thank you, observa, so they don't have to pay for it out of our own pocket. Yay.
Speaker 3:Observa is pretty cool. Observa is Rob Picard and Luca Caratoni and John Villanelle All Modestano people. So Observa does security programs for startups. I don't think you could find three people that you would rather have as your security team for your. I don't think you could find three people that you would rather have as your security team for your startup than those three people. So Observa check them out. They're very cool.
Speaker 2:Yes, we are all very excited to have the event in Vegas, and so thank you to Observa for sponsoring the event, and please keep an eye on either the episode notes or our social media to uh, get tickets to the event. Um, and we know that you're very excited to tell us all the ways that we've been wrong over the last three years and so this is your opportunity to go do that in the middle of the desert with drinks.
Speaker 1:Uh, happy, looking forward to seeing you all there. David did all the legwork to make this happen, so if, if it goes well, he gets all the credit and if it doesn't, we will never speak it up it again.
Speaker 3:Um, please go to a moonlighting as a marketing event coordinator has always been my dream it cannot possibly go worse than the microsoft gathering, where they uh bust us out into the middle of the desert to wait in an hour-long line for one drink and then, another hour to get busted back it was a it was the most amazing meetup I've ever been.
Speaker 2:Yes, we can say this event is on the strip, it is easy to get to.
Speaker 1:Yes, and it is near uh some frequent locations of the kind of defcon affiliated hotel universe and it's not far from the in and out burger, oh good we will.
Speaker 3:We will not kidnap you at no point in this. Will you wonder whether this was an elaborate scheme to get all the security researchers out of the desert to kill them?
Speaker 1:I mean, isn isn't that Black Hat and DEF CON period? Hopefully it will go well better than the event. I don't remember who hosted it by, but it was at the Hustlers Club and I got to see John McAfee there before he died. That was fun.
Speaker 3:It better be solved after he died, yeah.
Speaker 1:It's like how, speaking of McAfee and things that are supposed to make your computer safer, that's never a good transition. Speaking of McAfee. There was a major, major international information technology event.
Speaker 3:I think y'all are taking this a little too seriously.
Speaker 2:This particular event. I woke up to a text from my mother about this outage.
Speaker 1:Excuse me, this affected me personally. I tried to order a Starbucks and their entire mobile ordering system was broken across the United States, as far as I could tell, because of crowd strike, aka clown strike sorry, important question what's your starbucks order?
Speaker 1:um, now it has evolved into a trenta berry summer refresher, which is it's just just the blue one. They introduced a new flavor. It's effectively blue raspberry refresher drink with caffeine in it, no bubbles, light ice, and then I usually get the feta cheese wrap with egg in it, or I get an egg white egg bite, or I've discovered that they have a cheese danish and I'll get the cheese danish if I don't want something protein-y.
Speaker 3:Anyway, is the Trento like the big gulp?
Speaker 1:the yes, it is like 32 ounces of of drink, um, yes, and so I get like a. I get a big gulp worth of blue flavored like vaguely grape drink with 100 milligrams of caffeine in it or something like that, because it's large, anyway. So if you were under the rug of most of yesterday, we're recording on Saturday, the 20th, yesterday, friday, july 19th, crowdstrike who operates Exploit Detection and Remediation System. Is that what EDR?
Speaker 2:stands for Endpoint, detection and Response.
Speaker 1:So endpoint security, which is corporate antivirus and sometimes corporate configuration management.
Speaker 1:Yeah, I got one of the words right. They are extremely popular on a whole bunch of operating systems but apparently they were particularly hit on their Windows systems. For everybody on latest and probably latest minus one or something like that I saw that somewhere, I don't know if that's true. But a lot of their deployed Windows fleet, which is companies, organizations across the world, all of them received an update not to the software but to a content, piece of content that gets parsed by their software and their parser is in a kernel module for the Windows kernel.
Speaker 1:And when your parser panics inside your kernel module loaded into the Windows kernel, guess what? Your fucking computer panics and reboots and reboots and reboots until you can get to it and say stop that, please unload any foreign uh kernel drivers or kernel plugins or whatever. Um, which is called safe mode for windows, so that you can try to boot into windows, remove the offending files, because it's just a file that a existing parser was trying to parse because it got pushed. You can remove that and actually try to restart Windows again safely, and sometimes it takes between three and 15 reboots once you've done the fix Before you can reboot your Windows system again. For a lot of people, you had to go manually to these machines, which is like I get the safe mode bit.
Speaker 3:I get that they have to boot into safe mode and remove the all zeros content update that got put there by whatever bad uh you know.
Speaker 2:Build process they had, but why?
Speaker 3:do you have to boot multiple times after you do that?
Speaker 2:I know I think the boot multiple times is there's some sort of deterministic aspect to the book or non-determinism aspect to the bug, where for whatever reason, the thing won't get parsed.
Speaker 2:A very small chance chance at the time, and then you might be able to boot normally without it going away At which point, if you do boot normally, the update process will kick in again and remove the invalidly formatted content file, and then you'll be good. So if you can manage to boot normally, for whatever reason, it'll fix itself, but you probably won't. At which point you boot into safe mode and then you remove the um content file is full of zeros.
Speaker 3:That is causing the parser to crash in the kernel as somebody who will never be able to run a deploy again without having a small panic attack, which is a consequence of spending four years now working on a public cloud. I have nothing but sympathy for the the crowd strike people and also whoever ran that build um is my favorite person in the world for a variety of reasons, but like that's, that's, that's right.
Speaker 3:People are like people are nuts about like what happened here and like the the consequences of this and like nobody can imagine how like a company like this could have let this happen. Like this is the natural state of all software yeah it's.
Speaker 1:What's weird is that it hasn't happened a hundred times before yeah, because it does seem like this is a parser bug that's been there for a length of time and it was just finally triggered by this. You know, whatever new rule, new like signature that they were trying to parse with their parser and it got triggered and the all zeros like content thing they were trying to parse is one instance of something that can trigger this. This parser and it got triggered and the all zeros like content thing they were trying to parse is one instance of something that can trigger this, this parser bug, which is like a null pointer dereference in their parser. Um, but yeah, I mean when? Well, number one, why is this parser living in a kernel uh plugin in first place, a kernel driver? It feels like we've learned not to do that sort of thing, especially for less privileged stuff, but that's exactly what's happening here.
Speaker 2:I think, generally, you're right. There's a question. I mean I'm no expert on what CrowdStrike's product is specifically trying to achieve, but you know, the issue here is less on like oh, automatic updates are bad. And more on like why is there so much privileged code running in the kernel? If it could have been in user space, right, like how big is this kernel drive generally and could it be made smaller space, right, like how big is this kernel driver generally and could it be made smaller In this case?
Speaker 2:You know, I don't know what the threat model and the trust boundaries are around the CrowdStrike's driver. So, like you would think, oh, you know, just parse those malware definitions, you know, not in the kernel. Like why would you parse it in the kernel? But what happens? Like is there some way that you can verify that it was parsed correctly? Right, like if part of your threat model is there's something running at user privileges, like do you want that thing at user privileges to be able to somehow then like alter, uh, uh, the output of that file? And there's there's, presumably and, like you know, replace the malware signatures with zeros or just delete them? There's probably a way to accomplish this where the parsing is not done in the kernel and you can still reach that trust boundary. But I'm no expert on Windows.
Speaker 3:But your CrowdStrike. Here you're operating at a scale where malware is built specifically to defeat CrowdStrike. Most security products, your adversaries, like security products in general, like things that you pay money for to make you more secure, or whatever. No adversary in the world gives a shit about anything that you do. Crowdstrike is the rare exception to this right when, like, it is worth somebody's time to write like a couple hundred lines of code to defeat CrowdStrike. So if there's a thing you can do where you can like jam up you know a user land windows service and thus keep it from getting updates or keep it from detecting or something, like that like this is not me vouching for the model of, you know, hoisting all of your security stuff into the windows kernel.
Speaker 3:Um, like I don't think that's a great thing to be doing, but like you can see why they would do it that way, right, like they actually do have adversaries that are you know I'm trailing off here that's okay yeah, I mean the windows model, um, for like, uh, cross-process kind of protections is more or less that they're like isn't one you had?
Speaker 2:there's like user admin. Lots of people just run as admin. Um, that's becoming less common on windows 10 and windows 11, um, but more or less there aren't really like a lot of per process um protections, and those that exist are still like very hard to reason about in a way that like other platforms kind of offer more. But windows is in a little bit of a bind here because there's such an old platform and they're so big, like if they say you know, oh we don't actually want things to work like this.
Speaker 2:We're gonna like shrink some of what you can do in a kernel module or change it like oops, like not letting third parties do something is an antitrust, and continuing to let third parties do things is while microsoft blamed for, you know, when a driver crashes. So it's a bit of a rock and a hard place.
Speaker 1:But also, like Linux, doesn't have these anti-competitive like hawks watching it, because they're Linux and not Microsoft Windows.
Speaker 2:They also don't have users.
Speaker 1:That's why they don't have anti-competitive locks, um. But mac os also. Crowd strike also works on. They have a mac os version, but it doesn't seem like it's as capable and it seems to be slower on mac os because, I'm assuming, because of the way that the mac os kernel is architected, um, but I guess it's just not the same competitive base as Microsoft and Windows.
Speaker 2:And maybe it has the same bug and they just didn't push out a bad content file a definitions file to Mac.
Speaker 1:Or maybe it does and it just doesn't cause a panic, or maybe the panic is caught, or whatever it is. It doesn't cause a kernel, a full system kernel panic or something like that If I were a kernel, I would simply not panic. If I were a null pointer, I would simply not be dereferenceable. I don't know Something like that.
Speaker 2:What do you think, Deirdre? Does Rust fix this?
Speaker 1:It actually does not, Because if you did everything, if you wrote this parser and you're still like pointing at pieces of memory via kernel boundaries and blah, blah blah, you're still dereferencing uncontrollable pieces of memory. I don't think Rust would actually help this.
Speaker 3:It totally would fix this. It has option types. You would never have a null to begin with.
Speaker 1:Oh, so it's not the memory safety, it's the nullness.
Speaker 3:We have lots of languages that don't have the stupidity of null, so yeah, you would have like a none where you expected a sum, and there's no way to dereference a none in Rust. Okay.
Speaker 2:Aside from the specific keyword used for doing that like explanation point right.
Speaker 3:What happens?
Speaker 2:when you panic in Rust inside the kernel. Oh is that a kernel panic?
Speaker 3:It's a kernel module written in Rust and it's all unwraps instead of matches. Yep, yep, yep, excellent.
Speaker 1:Yep, it's like oh, we did the right thing. It's like you didn't no.
Speaker 2:I mean, presumably you would like not do that a ton, but like I think the point is that a crash in the kernel module is still a crash in the kernel module, regardless if it's actually a null pointer. Deref module is still a crash, yes, kernel module, regardless if it's actually a null pointer, d ref yes, or if it's just like oh, the language, the language has saved me by asserting on this bounds check or this null check for me. But oh well, once it asserted like yeah, still crashed yeah this has been our two minutes of rust humor.
Speaker 3:I'm you. You read a lot of stuff.
Speaker 3:Oh my god, I feel so bad right now you you read like a lot of stuff about like what CrowdStrike is and like what EDR things are, and it's like it's all very simple, right? This is just what people run in 2024 instead of antivirus. So you know compliance regimes and like IT security practices. Like you know, if you look at a Fortune 500 company, they've had like a large-scale IT security practice for the last 30 years. Some of this is just logic that is handed down from generation to generation that something has to be running on every endpoint. It used to be McAfee AV or Symantec AV and now it's CrowdStrike. Honestly, you're better off now by a lot. You would much, much rather be running CrowdStrike, where they every once in a while push out a build that crashes every Windows machine on the planet, than running AV engines that were all written in like the late 1980s. There's like so much chatter going on. This is like the biggest thing that's ever happened for IT nerds in the last several years, so people can't shut up about it.
Speaker 1:Everyone keeps saying that this is what y2k was gonna be like, if y2k actually happened, and I don't know if that is actually so extremely mild. I don't know if this is quickly.
Speaker 1:Everyone say your age at y2k um, unfortunately, like, maybe the impact, like one-time impact, is at the large scale that we were afraid of for y2k, but the remediation is quite easy, like you, quite easy compared to y2k. Like y2k basically required, like, the most critical pieces getting rewritten. Uh, the code getting rewritten versus this is just like, oh, just just go like there is a step by step thing you can do. It's not fun. You have to go find your host and go into safe mode, remove a file and then reboot it. You know 10 times perhaps, but like that's, we know what to do. It's not. You have to rewrite all your software. So I don't know.
Speaker 3:I have a bone to pick here and I think David will appreciate this particular bone I have to pick here. I'm again dragging the orange site into this conversation, although this particular orange site drama links back to Twitter, so it's also Twitter drama. But some enterprising souls have uncovered the interesting detail that George Kurtz, the CEO of CrowdStrike, was at one point the CTO of McAfee the quote unquote CTO of McAfee during an era where McAfee also pushed out a bum updater that broke everybody that was running McAfee.
Speaker 3:You know, broke everybody that was running mcafee and my problem here is that no one knows what the hell a cto does. Um, so there's like a. I can't get the picture out of my head that these people are imagining george kurtz sitting there on his computer writing software updaters, as if that was what the cto at mac like. The quote, unquote, ct. So, david, what does the cto do?
Speaker 2:um, I mean, it depends, but um, I would say cto is mostly like not a real title and it is actually just like a title that you have when you're in a different role but you need to sound cool. Um, those roles being technical co-founder, which is a perfectly valid form of cto and arguably the most valid form of cto. It's like you have someone at the company who needs, who has a lot of equity, a lot of institutional knowledge and is kind of driving the technical product direction. That's, you know, your technical co-founder. There is like CTO, as in, is actually just a VP of engineering, in that they're like in charge of NG execution. I would argue like you should get your title to be VP engineering, engineering. In that case, again, if you are a vp of engineering, you're not writing a software updater no um, and then there's like sufficiently large company or whatever
Speaker 2:yeah, absolutely the cto um, that is that is actually a vp eng is like not writing parsers, right, they're looking at jira tickets and they're writing okrs and they're you know um like the, the highest level okrs, right um. And then the other case is like you're a big ish public company and you kind of need someone at the roughly the c suite level to go around and talk about technology to other people at the C-suite level, and that person is usually someone who used to be a VP Eng at somewhere big. So, for example, the CTO of Microsoft, I believe, used to be the VP Eng at Salesforce. I would need to double-check that exactly.
Speaker 3:I have a simpler breakdown of CTOs than you do. They're equally valid breakdowns. They're both worth hearing right. You have the OSI model of CTOs and I have the IETF model, and I want to caveat this by saying that none of this applies to the CTO of Flatio Jerome, who I adore and is amazing and also spends all day and all night rewriting every line of code that I've ever written. So this is definitely not him. But my general breakdown of CTOs is that there's three types. There is the type where it is the consolation prize that the founder gets when they take away his commit privileges. Finally, when there's a real engineering team. That's the first kind, right. And then there's the kind of CTO you are when you are the CEO of a large company that gets bought by another company, where they're not making you the CEO. They have to give you a C something title. So it's CTO, which just basically means the CEO of the product line that is now part of this bigger company. And the third kind of CTO is the sucker who took the CTO title instead of pushing for more compensation at a small company that shouldn't have a CTO at all. Those are the three kinds of CTOs. There are right and like.
Speaker 3:If you ask me what the most important job skill of a CTO is, it is identifying themselves as the CTO on a call with a customer prospect. That is it. That is the CTO skill. That is what they do. The idea that a CTO is doing any kind of technical work whatsoever People think they're like it's like they're the lead programmer in the company. They're like it's like chief scientists, like they're doing the most science. It's the same thing, right? It's a title that has absolutely nothing to do with what the, in my experience, has absolutely nothing to do with what the title sounds like. So, no people, I don't think that there is much of a George Kurtz connection with the broken updaters.
Speaker 1:Now we have to do CISOs and CSOs.
Speaker 2:Our advice on that one is never accept that title even if that's your role. Just get any other title.
Speaker 1:Yeah, yeah, just get any other title. Yeah, yeah, it's uh.
Speaker 3:the connection between these two and any incident with mcafee and any incident with crowd psych is like coincidental at best I mean, if you do get the cso title, just remember, like the mesoamerican cultures where, like before, they sacrifice you, you get like a couple of days of everyone just treating you really awesome, and you're gonna eat whatever you want just make sure you're getting of the stuff that you can possibly get before the day of sacrifice.
Speaker 1:Absolutely. You are there to be fired when there is a security incident, or that has traditionally been the rule.
Speaker 2:Just don't have security incidents. What could it cost $10?
Speaker 1:So, like we did, kind of, you know, do our little hug ops bit about like this is like the, the fear of anyone that pushes out updates, whether they are actual code, patch updates to you know your deployments, or configuration updates that can have almost as much or equally as much impact because they tell they are executable instructions by the executing software or whatever, which is basically what happened in this case. Um, but like how this thing, as far as we can tell, the configuration update or signature, whatever it is, got pushed out, uh, between 4 am and about 5.30 am UTC yesterday and it was all zeros as far as we can see, and people have claimed to replicate the panic with content that's being parsed. It's not just all zeros.
Speaker 1:But, how do we prevent problems like this? Well, the problem is that we don't have the procedures in place and people in organizations that allow this problem to occur.
Speaker 2:Not now there's a meme about this Takes a long drag, yeah it takes a long drag on a cigarette.
Speaker 1:It's like, yes, the root cause analysis, your five whys, whatever you want to call it, the you know blameless postmortem, will eventually like zoom in on.
Speaker 1:There was a pipeline procedures, pressures, checks that either were in place and not followed, or they were all in place and followed and were insufficient, or there was a bug in the pipeline, in either the CI testing pipeline or the CD deployment and packaging pipeline.
Speaker 1:And you know, or you know someone claimed on the internet like oh, this was supposed to only go to, you know, the client staging environments and it overwrote that. Or you know, we don't know enough. Now there's a lot of speculation. But you know, having a thing that's basically all zeros get pushed out feels like something that we should be able to flag and it's unclear whether it was built and tested and passed all the tests with all zeros, or if it passed all the tests and then it got built and after that it turned into all zeros and nothing would catch that or whatever. So I'm very curious, I want to learn more and like we'll see if there's anything that we'll learn publicly about this. But I'm very interested in sort of like the pipeline, the procedures of how this gets out into the wild in general, because that's where I was zero, zero in on and trying to figure out how to like prevent stuff like this from happening figure out how to like prevent stuff like this from happening.
Speaker 2:The the pipeline procedures question is is like somewhat interesting, I think, and we'll definitely like have to do it for like regulatory or legal or even just probably stock price reasons, but like I actually don't think it's the most sort of the, the thing that matters most in the sense that, um, uh, you have this question with any system that like has various checks in it of just like, oh, like, if this one thing like isn't working, like what is the expected like behavior? Like what a failure rate of like some given part of the system and like just because it was caught, like he had a failure over in step one but it gets caught in step three, like how bad is that failure in step one? Um, and so like there's shortcomings there and that this failure wasn't caught. But there's another question.
Speaker 2:I'm just like parsers are the easiest thing to fuzz and test right like so there's one side of like don't parse in the kernel if you can avoid it. But there's another side of like don't parse in the kernel if you can avoid it. But there's another side of like if you're writing a parser like, we know how to do that. Well, like the cryptographers are pretty good at this, like parsing. Like major browsers still parse X.509 certificates in the browser process because like they have to.
Speaker 2:Cryptographers have got because one they have to, there's nowhere else to do it and two, like the cryptographers have realized like how to spend the right amount of effort to make a parser that is like up to a standard that is kind of higher than the parser standard might be in other situations, but the reality is like of the code that you could write at a like like very high level of reliability. Parsers are like the most straightforward thing to test and to fuzz and like sure you can go into all of these policies and procedures about the deployment process.
Speaker 1:But like I would almost rather see them skip that and just be like we have fixed our parser skip that and just be like we have fixed our parser, which is why I want to give them the benefit of the doubt that they do. They did take this change of this config, tested it against their parser and then it passed. And then they built it for deployment and that's where. And then it something went wrong in the build, the, the c process, the packaging process, the release pipeline, not the testing pipeline. But that also means that you have to test twice or you're testing in the wrong place, because if you're testing before build and it passes and you test after build and it doesn't pass, then you've got a problem. But I don't know that much. But I generally agree with like we know how to make, we know how to test and write parsers better now and it feels like this one is like an extremely uh, non-risk, tolerant parser. So, uh, yeah, I don't know.
Speaker 1:I'm very curious or you know yeah use after free um, I'm trying to see if there's anything else.
Speaker 1:Um, there's been debate about like, why, why this is written this way anymore?
Speaker 1:Because windows, in later versions of windows, they have made safer hooks to in the like via the kernel or via that surface, available to software like crowdcirc. I think this was like their falcon sensor, um to use in a safer way. Um and like, why aren't they doing that? And it's like well, I have a feeling that this is deployed, this is deployed to hospitals, this is deployed to airline systems, this is deployed to fucking Starbucks, like. This is deployed a lot across a lot, a lot, a lot of diverse Windows instances. I think that there's a lot of them that aren't up to speed and making those hooks available, a lot of them that aren't up to speed and making those hooks available, and so they're doing it like kind of the the tried and true and available way for a lot of these heterogeneous windows safer hooks is a good thing when it's the operating system, but when chrome does safer hooks for, you know, intercepting network traffic and ad blockers and things like that the whole fucking world lights on fire.
Speaker 1:Well, it seems like another flavor of the anti-competitiveness like like, like for Microsoft and Windows. Like if they try to make it harder to do a thing for safety reasons, everyone freaks out about anti-competitiveness. And if Chrome tries to do something to make it make it available to people in a safer way, people go privacy.
Speaker 3:I'm just saying all the anti-manifest V3 people out there, this is the future that you are asking for is what happened with Crowdster?
Speaker 1:Something like that.
Speaker 2:Yeah, of like diverse deployments though. Um I I moved recently and um I'm trying to get my car title. Um, because I need to like register at a new place, and so I went to log in to my car loan portal and it was down. I was like man, I have all the things to go down because of, uh, crab strike. I did not expect um maz Mazda Financial to do that.
Speaker 1:It wasn't even like the local municipality, like you know Secretary of, you know Department of Transportation.
Speaker 2:No, it was not the DMV, it was like the car loan portal, where I wanted to go to pay off the loan so that they would send me my title, because that just felt easier than like requesting they send a title to the DMV. Yeah, lol.
Speaker 1:So one last thing before we move on from this is I thought Windows has their own built-in AV security thing like Windows Defender or equivalent for maybe something.
Speaker 1:I know there is one on macOS and I'm forgetting their name right now and everyone's like LOL, everyone forgets that this exists because mac apple doesn't like advertise around it, but it does exist. Um, why, like having the thing built into the os which has privileged access to all the things that you theoretically want to get your, your hooks into, to do the things that crowd strike is doing or another EDR solution might be trying to do, the OS is already doing it for you. So why don't we just all use like Windows Defender? And why is that not fine and sufficient? And I don't? I don't have a good answer, except that CrowdStrike does a lot more and is a lot nicer to use on the back end as like an operator of an EDR solution and like Windows Defender is just not enough. And so there's a reason that CrowdStrike has a market because this podcast does not endorse any individual antivirus product no, I don't like no, but like.
Speaker 1:There's a reason that markets have like, found like a solution, and it's that Windows Defender did not just completely be like solved it, just use this for free. It comes with your Windows or uses it for effectively free. It comes with your Windows license or whatever it is. So I guess that's why, and like what Thomas says, like you run it, you run CrowdStrike in 2024 instead of like back in the day, you would use McAfee antivirus, because that's what you do, and when the built in stuff into your OS is not sufficient as a real operator, it's not sufficient for the feature set and the usability.
Speaker 2:And it's not sufficient for compliance. I think think too for a lot of stuff like that, but uh, instead of talking about compliance, let's talk about literally anything else do you want to talk about post quantum standards compliance? Uh, I'd like to talk about how there still aren't any post quantum standards. It is difficult to comply with um compliance guidelines that provide deadlines in terms of when communication needs to be post-quantum secured If the algorithms that are post-quantum have not even been standardized yet.
Speaker 2:Let alone whether or not they're suitable for use in the real world.
Speaker 1:Yeah, we have a draft. We've had a draft for a year. We're talking about the Nist, the united states uh, national institute of standards and technology, not science and technology standards and technology, the post quantum cryptography competition that they've been or whatever. They don't call it a competition, whatever the thing, the thing that they've been running for eight years now, at least the standards, the official FIPS standards for MLChem, which is a new kind of key agreement based on lattices, mldsa, which is a new signature based on lattices, and SLHDSA, which is stateless hash-based signatures, which is not based on lattices, it's based on signatures are supposed to drop any day now, based on last days, is based on signatures are supposed to drop any day now. And it's all just like rumors and whispers and just sort of like people sending emails to anyone that might have like an in at NIST and like I've been refreshing their website every fucking day and just like watching and preparing and just waiting. Because they signaled publicly that it might, they might be ready by end of summer, slash september, and then they signaled somewhere that uh, oh, no, I think someone from this was on stage somewhere and they were like no, ml chem document is done. We finished it and it's, it's, we've sent it up the chain and we didn't hear anything about the other two. But we we think they're all going together in one fell sweep.
Speaker 1:Falcon is not done yet. They've told us that Falcon is another lattice related signature scheme. That's not done yet because it's harder. I think that's coming later. But these three it's very possible that gun for, like lots of people in industry, especially anyone that has to be compliant with, like these OMB memos from the United States executive branch about being compliant with the systems that you sell and deploy to the federal government. But also there's all these other systems of like if you want to be FIPS compliant, if you want to deploy the software in a national security system they've already outlined like you must have these X, y, z things and they have to be done by like 2030. Some of these deadlines are like real soon for the like big, like infrastructure and organizational deployment lift.
Speaker 2:Yes and no, so the part of the problem here also is that.
Speaker 1:It's the number on the page.
Speaker 2:Yeah, there's a number on the page, but the page isn't actually a compliance document. So, like the one that people cite a lot is the DOD that has this document that says this is what the new CNSA guidelines are, the algorithms for computer network security, or something like that, which is basically like For national security systems.
Speaker 2:Yes, yeah, which is basically what are you allowed to use to secure top secret data? Yeah, and they're, like you know, we are going to switch this to require basically like post-quantum, which if you buy the, you know the post-quantum threats. Then like that's a very reasonable thing to issue a document saying the next version of this 2.0 will look like this and a little bit of guidance on like prioritize, you know, store now to encrypt later over signatures, and like here's how like this interacts with like web browsers and here's how this interacts with like long term storage and software and firmware signing. And then it provides some like suggested dates, but like one this document is non-normative.
Speaker 2:It literally says this document is describing how we will decide the process.
Speaker 2:And they're like it will be based on standards from NIST. It will be based on these threats and needs. There are different priorities and so on, but it has some years in it, including things like 2025 for browsing traffic. Including things like 2025 for browsing traffic, and it's like well, if NIST hasn't released a single final version of the standard in mid-2024, you're not going to, by default, have post-quantum signatures anywhere in 2025. Oh yeah, but notably, the 2025 year is just a recommendation in this document. And then it says 2030 is the deadline, but this document isn't even the deadline because it's a non normative document. So all this to say, the only thing that matters for like actual migration to post quantum, is that NIST standardizes algorithms and then from there, people can work on solutions. But as long as these algorithms are not standardized, there's very limited cases where they can be used in anything outside of an experimental context.
Speaker 1:Yep, and I will point to yet another document, that's the OMB memo, which does say goal of mitigating as much of the quantum risk as is feasible by 2035. So that's like different, because that's for different systems in the us government than these national security systems that the the thingy from cnsa20 is talking about, but that gives it a little more oomph, but it is still like try to mitigate years.
Speaker 2:It's like there's a big difference between 2030 and 2035 when it comes to like trying to migrate the web PKI.
Speaker 1:Yeah, but what you ended on is absolutely true. It's like we can't we literally can't like make progress until we have a final standard of like the thing that everyone is pointing to, including all these things in the US government. But also countries around the world are watching this, you know process, this NIST, pq process, like proceed, and they are keeping an eye on it as sort of like, yeah, we're going to like, you know, take, take suggestions from that and maybe we'll add a little bit more to our stuff. Like a lot of the governments in other parts of the world are much more comfortable with hybrid solutions for, like their systems than this NSA, cnsa document for national security system, which is basically like no hybrid. If you can help it, mlchem 1020, 1024 will be like by itself sufficient and required to protect a key exchange for, you know, top secret, you know communications or storage or whatever.
Speaker 1:Um, other governments are like sure, mlchem and frodo or mlchem and I don't know, and true, or something like that. Um, but yeah, I don't want to spend the whole summer just sort of waiting, waiting, waiting, especially because I thought we would get it at the end of the summer and I'd just be like, yeah, yeah, yeah, I have a whole summer to not worry about that. And they're like, no, it's going to be like any day now. It's like, fuck, just do it already.
Speaker 2:There's another memo from OMB M2302, I believe that requests that agencies basically inventory their post-quantum risk in terms of what systems would be affected by a quantum computer and if this memo percolates down to you, if you're at an agency, deputy CIO, or whatever my advice to you if you're like at an agency, deputy cio or whatever my advice to you is to ignore the condition and just send back a list of all of your computer systems. I think, uh, the practical matter is that, like, it will affect all of it and nothing is standardized yet. So, instead of trying to ask people that whose job is not to think about quantum computers and cryptography all day, if you get this memo, just ask for a list of all of your computers yeah, uh, maybe talk to my employer, we can help with that.
Speaker 1:I don't know, we sort of help with that, with crypto inventory, um, we, we, we do some of that stuff. But yeah, uh, it's hard. It's hard to just get all the software and all the computers and just be like find me all the cryptography in it. So lots of people are trying to attack that problem. And then there's the part of like. And then it's like cool, we have an inventory and now what? Now, what do we do? And like people are not. They're asking that question. They're sort of like what's the next step? How do we remediate this thing? And it's just sort of like that's a good question that not a lot of people have a straightforward answer to.
Speaker 2:So I think we have talked I don't want to say extensively, because I don't think we've done a full like episode on this but I feel like we talk about this every other episode because it's like related to the day jobs of several of us here in some form where we're just like oh, all of these transitions are hard and there are no good solutions and these algorithms are very big, yep, um, but like a bunch of that discussion is even just predicated on like can we get a final version?
Speaker 1:of the standard. Yes, especially because, like there, there's the ongoing signature on ramp. So like they're gonna drop mlSA, which is like a digital signature that one could theoretically replace a bunch of stuff in the web PKI with. But it's so big, the keys are big and the signatures are big. The SLHGSA is probably good for signing firmware or things like that. It's not good for online signatures and putting them on the wire, for example, or things like that. It's not good for online signatures and putting them on the wire, for example, or in your like end-to-end encrypted communications or anything like that.
Speaker 1:Mldsa is the only one of these three drafts. Falcon might land later, but it's got some sharp edges that it's unclear if people will be happy with. Even that is still kind of big. The on-ramp is supposed to find more signature algorithms that will give different security. You know guarantees. So maybe not based on lattices, it'll be based on something else, but all of them, except for ski sign in the on-ramp, are still too big. They're still too big and like that's going to take more time.
Speaker 1:So, like one, we want these things to land period, but two, when they do land, we still have a lot of hard work to do to figure out how to make these things work in things like the web PKI, because none of our options are very good and it's going to be some hard architectural design, organizational migration decisions, designs and planning and coordination to figure out how to make any of this work. But that doesn't significantly degrade the experience that that users of the secure internet have come to expect for the past decade. So it's our so. So you're the, you're the browser man. You can talk more about that.
Speaker 2:I think we can. We can hold off on any more discussion of that, but we'll drop a couple links and the show notes. I think let's go back to Vegas for a second and talk the schedule. Uh, black hat, I think. I don't know. Maybe you will stick around through defcon I?
Speaker 1:I don't believe in defcon, so okay, why don't you believe in defcon?
Speaker 2:um, I like conferences where I can get coffee and water in the hallway, and that's not really a thing at defcon. Um and also, um, you know I'm on the review board for for black hat, as is everyone else on this podcast, and so we kind of have a little bit of a connection there.
Speaker 2:So I tend to go for black hat if I go at all, um, and so let's, let's I think we'll all be there for black hat, and so let's I don't know maybe highlight a few things that we saw on the schedule for 2024.
Speaker 1:Some of the things we reviewed include this cool like we wanted to talk about this more in depth earlier in the year on the podcast. But this Terrapin attack on SSH, breaking SSH channel integrity by sequence number manipulation this is pretty cool, like basic. It's not even like a cryptography thing. It's about the sequence numbers If you can manipulate. Well, I don't even remember if there's any cryptography thing that allows you to manipulate, but this is very cool. This is one that I'm going to go to, and we wanted to talk to these um, to these uh researchers earlier, but we didn't make it to um, yeah, the they identified two root causes um. Ssh handshake supports optional messages. Optional messages the best.
Speaker 3:This is the this. This, this attack is it's. This is it's. It's not drown level, for Drown is still my favorite applied crypto attack but this is still, it's one of so like when I started, like when I first started engaging with cryptography stuff, maybe like the late nineties or whatever you run into like um, like SSL three and like the SSL transcript hash.
Speaker 1:And.
Speaker 3:I was in my like my twenties or whatever, or my early twenties, when this was I'm old, but like, so like if you're like a normal programmer and you're like your first experience of dealing with a serious cryptographic protocol, and paul kocher designed ssl3 um like ssl3 is terrible, but that's mostly because we didn't know as much as we know now but a serious cryptographer designed most of ssl3 right.
Speaker 3:your first experience of it is like okay, so you're doing a hands, but at the same time you have to do a running hash over all of the components of the handshake, which is a giant pain in the ass. At the time it was hard for me to even get my head around the level of complexity of running a hash over multiple it wasn't chained, it wasn't what we have in TLS 1.3 now, which is sort of like updated chained hashes.
Speaker 3:I'm saying these things out loud and so I can throw together a noise implementation in a couple of hours now, but back then it seemed like just moon science to me. So, terrapin, so you can think of two lines major encrypted transport protocols, right, like there's the TLS line and there's the SSH line of protocols, and sometimes they have the same vulnerabilities and sometimes they don't right. So in the original SSH 2.0 design, the first serious like SSL 3 is the first serious TLS or the first serious SSL protocol, and SSH 2 is the first serious SSH protocol, right, and SH2 is the first serious SH protocol, right, they do not do a transcript hash like a full transcript hash over all the messages that happen, right. So instead what they do is it's kind of the same deal where you're just building up to doing a DH, right, you're just going to do a Diffie-Hellman and do a key exchange.
Speaker 3:So they look at this I'm putting myself in their heads but I wasn't in the room, but I'm pretty sure I'm right about this they're running a handshake to do a DH. So they look at this whole thing and they say, well, here are the things that contribute to the DH that we're going to do, here's all the things that matter. So what we're going to do is, instead of hashing every message that happens in the transcript, we are going to just hash the things that could ever hit the key exchange. Right, which is like what could go wrong, right, and it turns out like so you can't influence the key exchange, but what you can do is throw off the sequence numbers now right, because they don't go into the key exchange anymore.
Speaker 3:They're effectively attack-controlled, which means you can like pad out, like you can eat messages. Basically, you can like, you can snip out a message from it and like eat messages. Basically you can like, you can snip out a message from and like the. The only problem with this is that like and I haven't looked at this in a little while so I could be wrong about this and maybe they're going to get up on stage and show like a really, really awesome outcome for this the only like this is like as, as cryptographic protocol things go, this is I think this is a pretty devastating attack.
Speaker 3:Like this is not a great property for a cryptographic protocol to have, but but I think in practice and real SSH deployments, it doesn't matter a whole lot. So SSH has, after the handshake, like option messages that you can send to like ratchet up security or add restrictions or whatever. I forget what the options are right, so you can snip those out, as I remember, but like nobody uses them, so it doesn't matter. I think is the outcome, but still, still, like that is the thing you could do, because they're only um, they're only hashings, they're only doing authenticity for the things that matter, for the key exchange, but not for, like the meta stuff. For that runs the protocol. You can fuck with that stuff. It's a really neat attack.
Speaker 1:And like this, this smells like things that were optimizations in protocol design back, quote, back in the day, and like the 90s, early 2000s, where they're like we only try to hash or authenticate or do extra work over the things that absolutely need quote, need to be done, such as, like just the Diffie-Hellman, the things that will influence the value of the Diffie-Hellman handshake.
Speaker 1:But the TLS like, especially in TLS 1.3, like they've basically been like fuck it, like we, it's really hard to let some things be manipulable by an attacker and other things not be. The whole transcript needs to be like chained together in a, like you know, a hash tree. Basically like you do, every, every handshake, every piece of the agreement is getting hash and then every new thing is also getting hash and also getting hash. And you know they've. They've even gone farther just recently with encrypted client hello, so that there's even more information that is not even visible, let alone manipulatable by an adversary when you're trying to set up a session in TLS 1.3. And this attack smells of like we learned these lessons the hard way and fixed them in TLS 1.3 and other places in TLS and they haven't quite percolated down into SSH.
Speaker 3:What gets me here is that this is just such an obvious difference between the TLS protocol and SSH, right, so it's like this was discovered relatively recently, right yeah, I think at the end of 2023 is when that preprint came out, or whatever, right yeah, but that huge distinction between the two protocols, betweenaci and TLS, has been there for decades, right, and it took until now for someone to look at this and say wait a minute, these are not the same protocol. What is the consequence of these not being the same protocol? And the consequence was that the security of the other protocol fell apart. Right, so I just it makes me happy to be in a world where, at any moment, something like this might come out of somewhere else.
Speaker 2:Can I say something slightly embarrassing, which is that when we were working on the weak Diffie-Hellman stuff, in grad school, like the logjam attack.
Speaker 2:so we were looking at TLS, we were looking at Diffie-Hellman group parameters and other protocols and other protocols and I was tasked with like figure out what root parameters ssh uses and also go look at the ssh protocol and figure out if it has the same protocol vulnerability that logjam represented in tls. And I looked at that handshake for a while I was like no, this is a good handshake because, like, unlike in tls, it's got it's got a signature over the full set of, like key agreement parameters. And I missed, like what turned into a terrapin.
Speaker 2:I was like, oh, it's got the signature, so like we can't strip the thing that didn't have the signature in TLS and then moved on, but I just I missed it.
Speaker 2:But I don't know, but I don't know. I also wasn't. You know not to like come up with excuses, but I was primarily interested in implementing the handshake to get a scanner out of it because at the end of the day, as mentioned repeatedly, thomas isn't a tax person, deirdre is a real cryptographer and I am just a network engineer who happens to like reliable systems they should credit you on stage for leaving that bug there for them.
Speaker 1:I'm the cryptographer that likes to build the good thing. I'm still not very good at writing proofs yet, but I'm okay with that for now. Yeah, I mean, you were also looking at weak DH and you were looking at these things that influence the actual Diffie-Hellman value, which is protected. It was the part of this that is protected, and the sequence values and these other optional messages were not protected. So I don't blame you at all. You were looking for something quite specific and it was protected against that. But if you were just sort of looking around at what else you could notice in it, maybe you would have saw something like that else you could notice in it.
Speaker 2:Maybe you would have saw something like that. But it's okay. Yeah, our, our approach. We were, we were, you know, sometimes we collaborated with people who did like actual formal modeling and proofs. But the way that we got into like drown, for example, was just one day we were like we should look at that. Like someone mentioned that sslv2 was like still enabled by default on a bunch of stuff and we were like that's probably buggy. And then we just applied the feynman method to open ssl's code. We're like there's gotta be something here. We just stared at it.
Speaker 2:Then we were like, ah, we have found like logic bugs um, not cryptography bugs oh yeah and we were like so then we like contacted open ssl and they were like have you met these other people that found related cryptography bugs? I think your stuff might help them. We took a quadratic attack down to a linear attack because we just stared at the code and found bugs with pointers.
Speaker 1:Did you find a buggy implementation of a good protocol? We?
Speaker 2:found a buggy implementation of a bad protocol and. Sslv2 with Drown. We found the buggy implementation of a bad protocol, and that's LV2 with Drown. But we found the bug. We found the bug in the implementation, and then Nimrod Avaram found the bug and his collaborators found the bug in the cryptography and then, when you put them together, it made the attack on the cryptography much more efficient.
Speaker 1:Oh God, this is the shit that keeps me up at night, because all of this shit is so hard. It's literally like writing down something that's non-buggy and then translating it into code in a non-buggy way, and then you might have a different bug over here and another different bug over here and you combine them and you get something that is completely just, and I hate it. This is such hard shit and I hate it, and I don't think there's any. It's just like how do you? It's all hard and I don't know.
Speaker 2:If you're familiar with the Matt Blaze attack on physical keys, where you target one cut at a time. That is basically the bug that we found in the OpenSSL implementation, which let you target the key one byte at a time as opposed to all of it at once.
Speaker 3:Remind me of the Matt Blaze attack.
Speaker 2:So in a master keyed system, if you have access to just an individual key for a single door, this isn't quite accurate, but more or less the master key will have one cut and your key will have another cut, meaning the like various points would be at different heights and you know the height of your key.
Speaker 2:So that means that for every individual cut you can, you just need to find the cut that's at the other position. So if you can duplicate, if you have there's 10 cuts in a key and you duplicate your key and you make nine of them the same, and then you try all 10 values for the one cut that you've changed, you'll learn oh the master, my key is at height four. The other valid cut is at height eight. Therefore the master key must have a height eight, and then you do that for each, for each um cut in the key, and so it takes what was like an exponential attack yeah, like let's just guess all the cut heights to becoming linear in the number of cuts so tai dong called that the hollywood attack, right, but that's my mental model of most.
Speaker 3:Like actual crypt, like cypher attacks. Is tai dong's hollywood thing which is it's hollywood because if you, if you do printfs while you're debugging the attack or whatever, you'll see like the string of gibberish, like the actual, like the cipher text and at the end you'll just see kind of like the rotating letter as you figure out, letter by letter, um, you know, with the last. So that's like that's how beast works. For instance, is how, like the cbc padding oracle works. If you do printf there it'll look. It'll look like it's working it out one bite at a time, like in the movies. Because that's what you're doing. Right is bringing the attack down to guessing one out of 256 values, instead of you know, one out of two to the 128th, right so okay exactly that's very cool.
Speaker 1:Anything else on black hat schedule we want to highlight? Um well, just know.
Speaker 2:I think the network security track is quite good this year. There's several interesting things in it. There's attacks on dns servers. I don't know if that involves dns sec or not, thomas, but if it does, I'm sure you'll be there to laugh. Um. There's um exploiting vpns, um, which is always a fun reminder that your security appliance might make you less secure and there's stuff about vulnerabilities and RPKI validation which might actually start to become relevant as RPKI deployment becomes more of a thing, so I think it's interesting to look at it from that standpoint.
Speaker 1:What is RPKI?
Speaker 2:RPKI is the PGI used to secure BGP router communication, so that you can't spoof routes.
Speaker 1:Yeah, that's important. And yeah, why is it different? Why is it Never mind?
Speaker 2:Well, I mean, you're not authenticating domain names, so like something. The authorities and things have to be different, but it's still just. You know, x.509 certificates.
Speaker 1:I think, and signatures it's different, but not significantly different.
Speaker 2:It's another PKI because it's another use case.
Speaker 1:Yeah, significantly different. It's another PKI because it's another youth case. Okay, alright, there's some keynotes. Jen Westerly, who's running SISA, is the primary keynote.
Speaker 2:Jen Easterly, not Jen Westerly.
Speaker 1:Fuck, perfect, perfect.
Speaker 2:We're keeping that one in. That's an incredible fit. Keep it in, that's perfect.
Speaker 1:Yeah. A W we're keeping that one in. That's an incredible fit. Nowhere that's perfect, yeah, a Waluigi to her Luigi, they're the primary keynote, they're like the whatever main stage, whatever you call it, the.
Speaker 2:Michelob Ultra Arena.
Speaker 1:Is it really?
Speaker 2:At least that's one of the keynote arenas.
Speaker 1:Yeah, it's still Mandalay Bay, it's all Mandalay Bay. Yeah, the Michelob also to be seen.
Speaker 2:Yeah, that's the one that's like the fight stadium right Like if you think about like Ocean's Eleven and the fight night, right, it's like that.
Speaker 3:And that's a different casino. Do they have UFC there?
Speaker 2:Oh, something's, there.
Speaker 1:Is what's his fucking face? Is Mike Tyson going to kill one of those YouTubers in a fight there in the near future? They'll draw.
Speaker 2:No, but if you go one door over to the Luxor, you can go to the eSports arena and you can also see America's Got Talent live.
Speaker 1:I do. I do have a soft spot for the Luxor just because I like pyramids. Anyway, not pyramid schemes, just pyramids. There's another keynote, a fireside chat with Moxie Marlinspike, with Jeff Moss, who's the guy that runs DEF CON and all that stuff. What's his handle? I forget his handle.
Speaker 2:Dark Tangent. Yes, dark Tangent, I don't know if he still runs it, but he definitely started both defcon and then black hat, as like the corporate version of defcon yes, um, it def.
Speaker 1:Defcon was absolutely first because they needed. They needed a place to send their people with budgets to learn about all the tips and tricks from the hackers at defcon. And they couldn't do it via the defcon name, so they created black hat and that's why it's very different in vibes, even though defcon is massive now. Um, and yeah, I guess they're going to talk about privacy, uh, and security and other things like that, and moxie's moved on from stuff with signal and he's been doing some other cool adventures, so I'm gonna I'm interested to see if he's going to talk about that sort of stuff, but let's see my other advice for Black Hat is well, the first time I went was when I talked back in 2016 about Drown Cool.
Speaker 2:I was like, oh, I don't want to go to this vendor floor. That's for these. Like, I do real work. I'm not going to the vendor floor. Now it is. I strongly recommend that, if you're there, you do go to the vendor floor. Um, if not, not to buy anything, that's like a terrible idea to buy something like at one of these. But it's just interesting to see, like, what is going on. Like how are people marketing things? Like what are people, what are the things that people are trying to sell? Because, like, we're a very tactical, focused podcast and we have perhaps a different perspective than, like you know, if you're actually responsible for securing an organization. Like, what are, like, the people that are being sold to there? Like, why, how are these being marketed? Like, what are they going after? And you know, know, a good chunk of it doesn't make sense. Um, a chunk of the people in the booth like won't actually know anything about, like what it is that's going on, but it's still very interesting.
Speaker 2:And also, like you can get pictures with f1 cars, because it's ridiculous and so I do strongly recommend that you actually go and walk around the vendor floor at least a little bit. Um, as I remember, um you know, all those people are, in theory, making money that's true.
Speaker 1:Oh god, yeah, I've avoided it like the plague, but that is a very good idea. Um, I, I should definitely canvas for, like if people are, if people are shilling for pq migration or just quantum or quantum key distribution and like those things are very, very, very different.
Speaker 2:So I also recommend that you be careful about, if you do go to the vendor floor, like consider what you put on your badge, um, because if you're not representing your employer on the vendor floor um, like in a booth or whatever but you do want to go talk to other things, you don't necessarily want to like reveal that who you work for, especially if you could be considered a competitor someone that you go talk to um or just because, like, if you have a sufficiently large company on your badge, everyone will try to sell you something, and you probably don't want to buy it or don't have buying permission for it.
Speaker 1:Street smarts those are some black hat street smarts. I don't know I'm doing that Anything else. I'm going to DEF CON. I'm going to DEF CON for at least a day and a half, so I'll see you around DEF CON Probably. I like to go check in on the Crypto and Privacy Village and maybe there's also a newish Quantum Village. It's been around for one or two years and they do a lot of post-quantum stuff sometimes, so maybe I'll see you around DEF CON, anything else.
Speaker 2:There's a Google event on Friday, the 0x0gG. They run it every year. I will show up to that. I don't actually know anything about what they're running this year, but I will be at that. So I will be around on Friday. So I'm staying one day longer than I normally do, but awesome Thomas.
Speaker 1:Do you have anything else?
Speaker 2:I have nothing cool. Yeah, let's go go take a nap. Nap time, cool, all right. Well, once again, keep an eye out on our social media or the episode notes for details on uh on the get together in vegas. Thank you again to observe, if you for sponsoring it. We're excited to see everybody first or second week of August, depending on how you count, and have a great summer.
Speaker 1:Yeah, security Cryptography Whatever is a side project from Deirdre Connolly, taz Tachek and David Adrian. Our editor is Nettie Smith. You can find the podcast online at SCWPod and the host online at Durham Crustalum, at TQBF and at scwpod, and the hosts online at durham crust alone, at tqbf and at david c adrian. You can buy merch at merchsecuritycryptographywhatevercom if you like the pod. Give us a five-star review on apple podcast or wherever you rate your favorite podcast. Thank you for listening. See you in vegas.
Speaker 2:Bye also we're on youtube now yeah, we are on youtube but not with video of us, just audio. We're on youtube.