Episode Transcript
Matthew: [00:00:00]
Welcome to the security Serengeti where your hosts, David Schwendaker, Matthew Keener, stop what you're doing. Subscribe to our podcast. Leave us a lovely five star review and follow us at Serengeti Sec on Twitter, where Twitter posts occur approximately three times a year.
David: Quite frequently. We're here to talk about the past and future of the SOC, and hopefully provide some insight, analysis, and practical application that you can take back in the office to help protect your organization.
Matthew: The views and opinions expressed in this podcast are ours and ours alone, and do not reflect the views or opinions of our employers.
David: And we're not here to darn the SOC. We're here to create new SOCs.
Matthew: All right. So, as mentioned we're talking about sock, talking about old style sock and modern sock. And what kicked this off is my favorite psych op spirit animal, Anton Chevakhin has been on a roll for the past few articles.
And a few weeks ago, he released an article summary on what he calls modern sock. So the, I started, I wrote the one line summary in [00:01:00] our notes, but really this is like the three line summary. He and others have been iterating on the modern sock since 2016, but very few companies have actually created a modern sock.
Actually in one of the after I wrote this I was watching one of the interviews, I think down the security rabbit hole or one of the other ones. He said, he thinks it's about 0. 1 percent of companies have actually moved to what he calls modern sock. And they're mostly tech companies. Instead, most companies seem to just be iterating on the classic sock.
So I thought it'd be useful to kind of summarize a lot of these thoughts both for listeners and for David and I, as both of us believe in the modern sock philosophy, and this is an exercise for myself to try and put together all my thoughts on this and hopefully, you know, an act change. So in this discussion reminds me of the security poverty line discussions I've seen in the past.
I actually haven't, haven't heard that term in quite a few years. Have you heard it recently, David?
David: No, I haven't. But it hasn't gone away.
Matthew: No, we just stopped talking about it.[00:02:00]
David: It's like the homeless problem.
Matthew: Yeah. Yeah. And this refers for those of you that may not know, this refers to companies who can't or won't pay for the minimum level of security that is required to be on the internet. There was somebody, some famous security person a decade or so ago coined it. The, the law of the internet, you must be this tall to ride this ride.
David: I Think that's that what they were talking about, though, at the time, and I think this is still true today, is you have small companies that can't afford to, to pay for a lot for protection. And yet they house extremely large volumes of sensitive information, like small doctor's offices and things of that nature.
Matthew: yeah. And they can't not be online. Like you as a company, you cannot choose to be not online.
David: Right.
Matthew: So now I feel like we're starting to see an additional divide between what I'm calling the security rich and the security middle class hopefully, you know, hundreds and thousands of people will hear this and I'll be credited. They can call it [00:03:00] the Keener law of the security middle class. And the security middle class is kind of the sock of the nineties.
We'll talk in more detail about what it is, but you guys, when I say that, you all know what I'm talking about. You're, you're seeing big boards and PPU maps and lines of desks, all facing a wall of monitors. Then there's the security upper middle class. It's the same sock of the nineties, but they've got nicer toys
David: And more of them.
Matthew: and more of them.
Yeah. They've got the, the, the security middle class has their EDR and their. Application where firewalls and their proxy, and then maybe a SIM and then the upper middle class, like now you're getting into the fancy stuff you know, SSL inspection. And I can't remember what's the name of that, what's the name of the tool that looks into packets and like does malware detections on the wire and like, you know, full packet capture and stuff like that.
David: Yeah. I think one of the things though, but what you're calling security rich here related to the modern SOC is I think if you start from ground zero with the concept of [00:04:00] modern SOC, though, I think you could be a small organization and still be security rich. If you start with this idea in mind, this concept in mind that we're going to talk about versus transitioning to it.
If you start there, it's possible for you to actually be a modern SOC. and be security rich without necessarily having to be a large expensive company.
Matthew: That's fair. Yeah. I mean, it's not a perfect analogy. I was just. Taking the security poverty line analogy and running with it,
David: No, I think, I think it's still true though because, you know, you could be one of the one percent security rich you know, or one of the sub one percenters but that doesn't necessarily equates to security wealth and not, you know
Matthew: actual cash outlay.
David: actual cash.
Matthew: Yeah, that's fair. All right. So let's start with a summary of what is the sock of the nineties? Everybody kind of knows what this is, but I felt like it was useful to kind of lay out simply in terms of contrast. So we'll talk about tools. We'll talk about methods. We'll talk about people and we'll talk about [00:05:00] organization and location.
In terms of tools, every sock in the nineties is built around a SIM. I remember working on ArcSight over a decade ago but now it tends to be Splunk or any number there's, you know, a dozen different SIMs out there that all kind of constitute the center of one of these socks. And in each of these SIMs, you have a whole bunch of vendor provided tools.
With little to, you know, moderate, maybe customization. They're all pouring their logs into the SIM. Depending on your ecosystem, you may have an XDR instead of a SIM, especially in a smaller company, I think, although I could see value in doing it even in medium sized companies as well. You might have a SOAR platform with a few simple, small playbooks.
David: I don't, I don't think so. The SOAR of the 90, you're not gonna have any, you're not, you're gonna have zero soar.
Matthew: You're right. I mean, SOAR didn't really come out until 2007 or so.
David: Yeah. I mean
Matthew: it was later than that, wasn't it?
David: yeah, it was, yeah. I don't know. I'd have to go back and look at it, but [00:06:00] no, I don't, I don't think the, the sock of the nineties had any soar and I think there's still a lot of companies that we would say are still relics of the nineties that don't have SOAR either.
Matthew: So searching for when did sore come out as May 21st, 2021 is sour. The debut studio album by Olivia Rodrigo. Thank you. Google.
David: good to know.
Matthew: Yeah. So actually you're right, I think. And I think there's some discussion in the podcast and articles about some companies who see some of the success of modern sock and they look at the characteristics of those sock and try to get there backwards by buying tools like a SOAR platform.
David: Yeah. I think it's one of those, it's, it's one of those things like the seven habits of, you know, so successful people kind of
Matthew: yeah. If you, yeah, if you just mimic those habits, then you too will be successful, but maybe those habits came because they were successful.
David: right.
Matthew: Yeah. so That's fair. Cause that does seem to be the buzzword really recently, if there are custom tools or scripts, they tend to be fairly simple.
David: Yeah. And [00:07:00] there's going to be very few of those. They're generally created by one person. They're completely unsupported. If that person leaves the whole, they all collapse and fall into disrepair.
Matthew: Yeah. The hero model, right.
David: Yeah.
Matthew: In terms of methods these places are big on detect everything. iF you talk to leadership here, they'll tell you that they detect everything, they watch everything. They'll be proud of the number of alerts they review. They'll say things like, Oh yeah, we, we review thousands of alerts a year as a badge of pride, instead of as a mark of, you know your detection logic is poor, if you've got thousands and thousands of alerts and tons of false positives.
David: Yep. They will have a slide that says we detected a billion events. We filtered that down to a thousand, 10, 000 alerts. You know, and it filtered down into five incidents or something like that. You'll have, they'll have a slide on one of their briefings. That's has, that has that diagram on it almost
certainly,
Matthew: Yeah. It's, it's the metric of busyness. Look how busy we are. They're not talking about how effective [00:08:00] they are. They're talking about how busy they are.
David: to know.
Matthew: Yeah. Yeah. They'll have separate teams for everything. There'll be an engineering team who reports to one silo. There'll be a threat Intel team who maybe reports directly to the CISO.
There'll be an engineering team who reports to a different director than the SOC manager, and then they'll have the SOC all by itself.
David: Yep. And when something happens, they're like little kids playing soccer.
Matthew: Everybody chasing the ball.
David: Yep.
Matthew: In terms of people you'll see large groups of analysts in a tiered format. You know, the largest group will probably be the tier one analysts. There will be small to mid sized groups of engineers and they'll frequently be unit tasked to a tool. You know, you support the EDR, you support the firewalls, etc, etc.
David: Well, depending on the organization, it might be, you might, you're tasked to these five tools,
Matthew: Yeah, yeah,
small small organizations don't have enough folks to, I get that,
David: Yeah. So you, so you're going to be overworked and nothing, no one, no [00:09:00] tool is going to be well, configured, designed, maintained, because there's, there's too many for the engineers to keep up with.
Matthew: that's fair. in Terms of organization and location remember we're talking about the drive from the network operation centers in the 80s, there'll be a big room with lots of TVs in front. One might be a pew pew map one will be a national news station, one will be a local news station, et cetera, et cetera.
Might be co located with a network operation center. It's rows of desks with everybody facing the same direction. So let's contrast that with some characteristics of the modern sock. It may have many of the same tools as the classic sock. The key difference here is the level of customization and how they are connected together.
And the classic sock, they are all separate tools that are all, it's almost a star configuration. And it's a one way star configuration. They're configured independently and they're all sending their data into the sim in a modern sock. They are configured with bi [00:10:00] directional connections and they may be configured connecting to each other even depending on your ecosystem and level of advancement, whether you can customize that
David: It's a lot like a mesh,
Matthew: in an ideal world, you'll be able to bounce alerts off of multiple pieces individually before it even gets to your, your dashboard, but they may still use a SIM, but frequently they will use one in combination with a data lake, or they might use a data lake exclusively. Because they've run into issues.
They've run into the limits of the set. There will be a SOAR or a custom automation platform. It doesn't have to be a vendor SOAR, but there'll be some method of automated automation that is consistent and everybody uses and is supported. And there'll frequently be lots of custom tools and scripts. In terms of methods, the primary differentiator of a modern SOC is their automate first reduce manual actions that provide little or no additional value. First example I thought up looking up [00:11:00] the manager of an associate. If you have an alert come in for a username. That doesn't add any value over and above what an automation would do.
So typically they start with automating enrichments, such as, you know, who does the, what group does this user belong to? What have they logged into? Who owns this IP address? Has it been detected in malicious stuff? Then they will move to other automations we've talked about in the past, such as blocking IP addresses isolating systems, running malware scans, stuff like that. They also try to never perform an action manually twice, which I think is kind of an exaggeration, but it is a good lodestone. There's definitely a point where if you perform a manual action twice a year, it's probably not worth automating. But if you perform that manual action once a day it's definitely worth automating.
David: or for every incident,
Matthew: Yeah, or for every incident. Yep. So these modern socks are engineering led detection engineering is software engineering. This is a quote from a linked Google cloud security podcast. And the [00:12:00] article Anton's article is basically, he summarizes a lot of it, but he provided links to like 10 other articles, papers, and even a LinkedIn discussion on modern sock.
So you'll definitely want to go check out that article and start pouring through those links too. If you like this. As opposed to a sock where there are thousands of alerts, the modern sock focuses on the critical few items, or they find a way to either automate the response to those high quantities or to correlate multiple alerts together so that they only have to look at a few items or all these things together.
David: right? And they also know the business. They know how the organization functions and why it functions as it does.
Matthew: Money.
David: Primarily, hopefully.
Matthew: Yeah. In terms of people. This is a big differentiator. Like we mentioned before in the classic sock, there are a ton of analysts, a lot of chair, one analysts, and the engineers are separate and in a separate group and do different things [00:13:00] in a modern sock. Everyone is a combination analyst and engineer.
And this actually reminds me of when I interviewed with Facebook and Google several years ago. I don't remember which one actually tried to fly me out, but one of them was telling me that I would be spending one week on as an analyst and then one week creating, modifying, improving tools and automating. And at the time I didn't fully understand it, but I've actually been taking that and trying to spread that with me as I've worked other places, because I think that's a brilliant idea. bUt unfortunately it's a little tougher than just telling your analysts now you must work on projects to improve.
We'll talk a little bit more about them in a bit. So one of the comments about why having analyst engineers combined is that if you're not an analyst and you, for example, are responsible for reviewing the output of an alert, are you motivated to make sure that alert is the best it can be it was mentioned in one of the podcasts, they thought those who created the detection and those who monitor the detection to be the same people that aligned your incentives to [00:14:00] make sure that if that alert goes off at, you know, midnight.
That it's a good alert. If you're the one who's got to pick up the phone, answer the call and validate it.
David: Yeah, might say you have skin in the game for the alert.
Matthew: Yeah. That's actually one thing I've seen as well. I've been working a lot on automation over the last two years and it can be, there's definitely a disengagement when you're automation engineer is working on building stuff and then your analysts are just doing stuff. And nobody's really talking about how well the automations are working.
And the automation engineer thinks everything's going great. And the analyst is like, ah, this is so frustrating. Why doesn't this work the way I want it to?
David: Well, that's also a failure to have a good feedback loop.
Matthew: Yeah. But it's still easier if they're all in the same group or the same people. If the automation engineer also has to work alerts.
David: Right. But I'm saying there's a middle ground if you're not there.
Matthew: Yeah. And that's something we'll talk about later. Like you can't just flip a switch and suddenly say like, now you're an analyst and an automation engineer. Congratulations. That's, [00:15:00] that's like, well, you
David: not say it's a good idea.
Matthew: yeah.
David: You can certainly do it.
Matthew: oNe of the, one of the things they talked about at Google was that analyst engineers are not just managing cases and performing investigations.
They're also architects and project managers. For the change they want to see, if they see something that can be improved, if they see some way that things are going badly, it's expected that they put together a solution for and create a solution for and manage the solution. So that does require some pretty individually empowered and.
David: yeah, I think that might be a. Bit of a bridge too far because really what you're saying is you have to be a specialist in all things versus saying, Hey, you can have this concept and then you can work with others to flush that out and then work on others. You still to implement. So you could be an analyst engineer, come up with a good idea, pitch it to the architects.
And have the architects work that out. And then when they've got it worked out, they'll turn it over to a project manager, implement, and you could be along for the ride through that whole process, [00:16:00] but to say, you're going to be an analyst, engineer, architect, and project manager, that's, that's way, that's a bridge way too far.
People have limits.
Matthew: are you talking about? This is my dream job.
David: And besides if they are all these things, when do they have time to do any one of them? Right.
Matthew: No, that's fair. And I think that this, since we're talking about this from Google too, I think they have a, a vision that is swayed by the fact that they can afford to pay people a whole bunch of money and they can get kind of the top of the line folks who can do all these things.
David: And I thought you were going to say they have an aggrandized version of themselves.
Matthew: That may be true too. I mean, you've seen a lot of the talk about how it's actually how so many people at Google do hardly anything.
David: well, that there was actually that with that project Veritas interview with that engineer from Twitter, it was like, yeah, I only work like four hours a week. And I spend most of the rest of the time just screwing around.
Matthew: Yeah. There's been a lot of that from some of these some of these companies coming out. So I don't think that's probably the focus on the security team though.[00:17:00]
David: Well, I'd like to think that's not true.
Matthew: Yeah. All right. They tend to, instead of having tiers, instead of having analysts, engineers in a tier structure, they typically have them focus on a specific subject matter, like end point or network or automation or threat Intel. aNd I don't know if you want to talk upside
David: Yeah. One of the challenges about, you know, having someone put on these hats. Or converting others to that is the security folks today are, are invested in who they are, right? They're security people, you know, and you want to say, well, instead of just being a security person is going to do security stuff.
You're also going to do this programming thing. I think
Matthew: and project management. yeah.
David: I think it messes with their identity about who they see themselves as and they're going to be resistant to wanting to do that work. And then additionally, if you bring in. Software engineers and say, we're going to turn you into security people, then you're going to [00:18:00] have security folks saying, well, you can't just bring in some, you know, ragamuffin off the street and say they're a security person now, because that's, that's, that's who I am.
That's what I do. They don't have the experience I have. They can't possibly do my job. So I think that's where you're going to run into. Resistance. I don't think you're necessarily going to get too much resistance from the, the, the conversion side, you know, the software engineer moving over to security as you are going to have the security, the indigenous security folks resisting bringing in the, the the foreigners.
Matthew: That's interesting. Yeah. Yeah, we'll talk a little bit later, a bit more about that, but Google did definitely make the comment that they had better luck training software engineers to be security folks because they came from the world of, you know, site reliability, engineering and dev ops and all that kind of stuff.
And automation first, whereas the security folks have typically been a little more, they come usually from an IT background or straight from college. [00:19:00] All right. Oh,
David: Yeah. Cause I think from, from that perspective, I think the, the, the key thing there is then what you're talking about is you've got, you know programmers and site reliability engineers, that's what they do. And they're doing it for a certain purpose. You know, what you're doing when you bring them into security is just transfer that purpose from, you know writing, creating a widget that does X to doing a security thing instead, whereas security guy, they do security stuff. And it's, it's not so easy for them to do something that is, be done by anybody because it's not strictly security. It kind of reminds me of, there's an anecdote from World War II, where they were in the Manhattan Project. I can't remember, it was something during the enrichment process. I forget exactly, but somewhere during the enrichment process.
They needed to have these [00:20:00] dials be watched and the needle had to stay within a certain variance of, of the you know, there was a lower limit and there was an upper limit and the need to need to stay within that limit. And all the scientists were like, we need PhDs to do this. You know, this is too important.
They need to watch this and adjust it as needed to keep it in, in the thing. And what they did. Yeah. Well, what they did was they, they they brought just women in off the street to do the same thing. And they compared the two and they found that the women did a way better job at maintaining the balance between the upper and lower limit than the PhDs did.
So they ended up actually having women do all of that because they were much more diligent in that task of keeping it between things, not even needing to know. What, why it need to be in there, just, you know, keep it in here, adjust this to go up, adjust this to go down and make sure it stays in that limit.
Matthew: [00:21:00] I think I've heard of that. Yep. All right. In terms of physical location there is none typically there is no sock and they're geographically scattered and typically hiring the best people for the role, no matter where they may be.
David: You know, this also requires really good communication and collaboration tools to, to do that, to be successful at that.
Matthew: Where did the modern sock come from? There's two kind of driving forces behind it. The first one is it came from top tier software companies like Google, Facebook, and Netflix. These are companies that are engineer first. They hire everybody as a software engineer. For a while, they were only hiring software engineers, so they needed to do security.
Google explicitly credits this change to the Aurora incident. Operation Aurora was an incident at Google, supposedly from the People's Liberation Army in China breaking in during 2009. A lot of other companies, I think they said there were 34 companies targeted, including Yahoo, Symantec, Northrop Grumman, Morgan Stanley, et cetera, et cetera.
But you had this big [00:22:00] software engineering company where the majority of employees were software engineers. And they had to do security better than they'd done before. And they came at it from a software engineering background. And the second one here is the drive to automate yourself out of a job.
wHich is pretty sure what David's been saying since we first worked together. At this point, what was that? 10 years ago?
David: pRetty close,
pretty close.
Matthew: Yeah. That's something you've been saying forever. So, and then this comes from kind of the site reliability engineering and the engineering first ethos of these tech companies.
So I said there was two things because, and I said that because I had two bullet points here, but now that I've read them together, I realized they're actually really the same bullet point. And I should have said, this is all coming from the fact that these are engineering first tech companies. And if you're not sure what site reliability engineering was or is, it is a fairly new discipline focused on creating reliable and scalable software systems.
It focuses on reliability, even if it [00:23:00] is changing frequently both in terms of updates and utilization
David: Now, what I would say is some of the characteristics of the modern sock when you dig down into it is,
Matthew: made of wool.
David: well, that's the winter sock. The summer sock is more of a silk.
Matthew: Ooh, nice.
David: But a characteristic of the monitor stock is that they understand the organization that they're working in. You know, how does the organization make money? What are its key mechanisms, processes, business units, software, third party relationships, integrations, things of that nature.
They know what needs to be protected and any associated regulatory requirements around that. What, what are the risks that could hurt the organization from making money and they can identify those areas where the SOC can influence those risks, they understand how the cyber defenses work. You'd be surprised how many SOC analysts don't under, don't understand the, what the, what does, what all the [00:24:00] cyber defenses are and what their purpose is and how they actually protect the organization,
Matthew: Yeah. I I've been guilty of this myself in the past, simply because sometimes it seems like security is so divorced from the business. That you really don't need to know. What the business does, like you're just looking for the malicious behavior and for certain types of things you don't, but to be really, truly effective, I think you're right.
Like you can get like 60 to 80 percent there without knowing what the business does really, but you can't get all the way to, to a truly great sock
David: right? I mean, it goes back to the, the whole Sun Tzu know your enemy and know yourself, right? Which takes us to the second part about knowing the threats, you know, why would they are targeting the organization and what would they need to do be?
Matthew: hold on. Can we actually go back to knowing yourself? So sorry.
David: fair point.
Matthew: I'm, [00:25:00] this is, I'm having a hard time with us, honestly, because most, a lot of times most companies are so big, they don't know themselves at all either. Like there's, you know, shadow it and what if nobody has.
What if nobody's keeping track of what everybody has the left hand or the right hand doesn't know what the left hand's doing. Like this is like, I know that you kind of glossed over it and you're like, you should know your business and know yourself. But I feel like a lot of companies don't a lot of companies don't know themselves, not just the security folks, like the company itself.
David: Because once you get to a certain size, it's actually, it would be difficult for that to be true, I guess.
Matthew: No, but, but, and your point still stands though, because if you don't know the company then and you don't know where your, your key, you know, the crown jewels that everybody talks about are like, how can you effectively protect them? Instead, you're effectively trying to protect the entire battlefield. It's like walking into a a battle with another army and you're trying to spread your army across the entire front to protect the whole thing.
And [00:26:00] the other team, the other, the other army just has to focus their focus down to a point and punch their way through your line.
Your point is good. I just, it's, ah,
David: maybe that's something else that, that you have to consider in a modern SOC as well as that maybe there isn't a modern SOC, right? So if your company is so large that you can't know yourself, then maybe you reorganize or reorganize the security defenses for that to say, you're not going to, and maybe build a task force or something or, or.
Mini socks around this concept to say your function is to protect this line of business. Instead, you know, you're going to protect Citibank's ability to issue credit cards to people who earn less than 50, 000 or something like that. You know that might be too narrowly focused, but you know, that's the example I'll give and say, you know, that's your job is to protect that capability [00:27:00] within this organization.
Instead of saying, Well, we're going to have one group that protects the entire organization, all the aspects of it and how it, how how it does and what it does. And the the idea there is if you're able to distinguish between this line of business and this line of business, and then have those two security teams.
Learn that and understand it. Then those two teams could talk and find out where the rub lines are between them. And maybe improve the defenses where those two meet also. And then maybe if you do that enough, you've got a fairly good overlap across the entire organization and you have an adequately defense, but it's a defense in depth overlapping fields of fire kind of thing versus one overarching centralized thing.
Bucker.
Matthew: yeah, yeah. Maybe I could definitely see that.
David: So now that we're done talking about the understanding the org, we'll move on to the second
part of this on Sue. Dick of understanding [00:28:00] the threats. So, you know, why would they target your organization and what would they need to be? Do to be successful based on the reason that they're targeting, you know, so if they're going to target you to steal money from you, you know, what would they need to do to be successful to steal money from you, but they're targeting you because they want to steal customer information.
What would they need to do in order to be successful to steal that, you know, what database would they have to get access to, you know, that kind of thing. So what you want to do is you want to be able to then be able to put those two things together where. You know what you need to protect, and you know what the attackers are going after, and you marry those two up
Matthew: something we've heard from companies before talking about how the attackers seem to know their company better than they did and know where to go and what to pull. And they were like, Whoa, we didn't even know this was here.
David: right, because they invest no time in understanding themselves, right? and then, you know, when they, when they marry that up, they have to say, okay, what would the organization need to detect? When the above two [00:29:00] things overlap you know, what logs do you need and what tools do you need in order to do that detection?
So if all your customer data stuff is in a database, you know, what do you need to detect an attack against that database? So, and then once they have this stuff, then you build your detections based on that maybe and we'll get into that in a second. But then, you know, they direct their prevention, the detection configurations based on the above, you know, what they figured out from that analysis, and they mirror that with their, their breach and attack simulation tools to build in.
The determination about how successful those tools, what you have at detecting what you just determined was necessary. Let me skip that. And, and they, they know how to manage all of this together. Then another characteristic of, of the modern SOC was be that it has a threat hunt team that their output then feeds back into the processes that we talked about above for the detection. And the prevention,
Matthew: [00:30:00] Yeah, this is, I mean, I'm sure that I'm sure that there's companies that do this really well, but I've heard a lot of talk about kind of disin tank, like separated threat threat Intel and hunt teams and content development teams that are all in separate silos and all reports to different people and don't do a great job of feeding each other.
David: right? And this is, you know, where we're talking about kind of everybody does everything kind of thing here, where the knowledge we're talking about here, and this is really what we're talking about for this entire section here is it's, it's an understanding. It's a knowledge problem. It's not that necessarily that they need to do a lot of work.
In order to fundamentally understand these things. And I think once you've understood it, you could document that and bring and educate new analysts slash engineers much faster on this as well. And if you're able to take the, you know, what we're talking about here. And have that fed into an on prem AI where you could ask [00:31:00] questions and get the answers, then, you know, that's going to short circuit this learning curve as well.
And the maintenance of it also.
Matthew: Yeah. And I think a lot of this. A lot of this is all things that a classic sock does. I think the main difference between it is the feedback loops and the automation.
I mean, a big part of it's that knowledge part, like you were talking about, but in terms of the putting it together, creating the detections using breach and attack simulation to validate the detections setting up the feedback loops internally. I think the main difference between a modern sock and a classic sock is setting those up to happen automatically and not setting those up to happen via like person to person meetings.
David: And I don't know, I would disagree because I think the modern SOC, this is a more formal decision. That they've come to, to do this, whereas the classic sock is much more ad hoc saying, Oh, this thing is in the news now, or I heard about this thing. So now this is the way we're going to do it versus [00:32:00] actually looking at the organization, looking at the, the attack and, and looking at where the overlaps at.
I think the, the, the classic sock is much more may do some of these things, but in a much more ad hoc fashion.
Matthew: CC as more of a, a, a journey up the maturity scale. Then
doing it
David: Part of that. Yeah.
Matthew: Interesting. All right.
David: Another characteristic would be that they have the automation first mindset as well as the skill set, and they don't do they don't perform actions arbitrarily. If there's no value in performing an action, it's not done. Another characteristics. Other characteristics around the logging. They know what logs they have in the state of those logs, whether they're good, bad or incomplete which informs how they write their detections.
They know what logs they don't have. They economize their logs so they don't collect logs that are not needed or not used and they're able to prioritize getting the [00:33:00] logs into the sim or the data lake based on the organization and the threat knowledge that we talked about a minute ago. And they have a plan for scaling logs.
At some point, they're just going to be too many logs to manage and search cost effectively. And at, at, and at some point in the future no one is going to build detections anymore. You can, you can write this down. No one will build detections at some point in the not too distant future. And it will be all left up to AI because this is a too, it will be too big for people to do.
Matthew: I think, I think the logs has to be included in that, right? Because, all right, so let's, so right now let's say that you see, all right, I'll, I'll give you an example. I was reading up on the DFIR report. One of their recent posts, they talked about how the attacker got into the database of publicly exposed database, they brute force their way in or their credential stuff their way in, then they use the when XP command shell [00:34:00] that that's been around forever.
I remember when I was working for an army contractor and we had to watch for that with Oracle logs. But those logs, let's say that your company is not integrating those logs. So a threat Intel event or your threat Intel team decides that we need to monitor this XP underscore command shell functionality in a database.
I think that the AI would have to own that end to end, like it would have to reach out. It would have to configure the database to produce the correct logs. Cause that that's not a default log gain by the database. It would have to. Pull those logs in then it would have to make sure that they are correctly parsed and then it would have to write the detection for it as well.
I think you, I think your solution there has got to be end to end on that one.
David: I disagree. I think what you're going
to
have, I
think you're close. what I think you're going to have actually is you're going to have several different AI. Here we go down the AI
Matthew: Oh yeah. Now we're talking about
the
David: can have, you're going to have different specialized AIs for those purposes, right? Cause you have to know the first part, you know, what do you [00:35:00] want to go?
What do you want to get? And then you have to figure out how to get it. And then you have to monitor it once you've got it. So I think what you're going to have is you can have three specialized AIs instead of one that does all three of those things,
Matthew: no, I can
David: at least
initially.
Matthew: tHat actually be a really cool product. cAuse then, cause right now, when you try to bring in, like trying to bring in those logs is a giant undertaking and especially trying to, cause you know, the database people are always like, ah, your logs are impacting our performance.
And
it's a multi month evolution to get these logs in and then get them formatted and then write out the detection. You're talking like three to six months to do this. So, but if you could have an automated method of doing that,
David: but one time it took me six months to convince the database people to turn the logging on.
Matthew: my, yeah, you're, no, this is yeah, that was, I remember that exactly. It would like increase the query time by like 5 percent or something that was too much for them because they were optimizing the living hell out of it,
David: But anyway moving on to the next one. We could probably [00:36:00] set up a, do a different podcast on that alone.
Matthew: man.
David: And maybe we should sometime is just talk about, you know, logging and, and that is a separate separate podcast. Another characteristic of, of the SAC is. There, I think there are critical tools that have to be implemented and configured correctly for the modern side.
So the first one being the SIM or a SIM like tool. You know, I'm not sure if you need to call it SIM specifically, but a SIM tool that does risk based alerting. And part of that risk based learning, or what I would say under the umbrella of risk based learning, is it assumes you have automated asset ingestion and prioritization.
So it's connected to the CMDB. Uh, and so any new assets brought on is automatically understood and, and it, it's criticality and everything is understood by the SIEM because of what's in the CMDB about it. And the same thing for identity. So that's either tied to an identity management solution like [00:37:00] Corian or SailPoint or something like that.
Or the HR system. So when a new person comes on, you know, you know what they do, who they report to in their chain and all that. It's integrated with a case management system. Cause you don't do the monitoring in the SIM. It goes to the case management system. iT's integrated with the vulnerability assessment tool, because I think along with the risk based alerting and the CNDB data, that the SIM automatically take into account the vulnerabilities for each individual system based on what the VA tool.
Reported as last scanned and any detection which is built into the SIM is going to be detection as code. aNd that brings us to the case management system. Now, the case, now, I'm probably gonna have some people disagree with me, but for case management and SOAR, I actually see SOAR as a component of case management.
Matthew: Yeah, I think these days, I think these days it's being built in XOR is a combination ticketing platform and automation service now is a base ticketing system and there's [00:38:00] an automation capability to build in
David: but under a case management nurse or all high, all high true positives are automated away. Virtually, if not complete, invent event enrichment takes place automatically. You know what? What's known about the sources, the IPs, the domains, the emails, what's known about the targets, what's known about the people or the users involved in the incident or the event? All the the, the SOAR platform is integrated with all the SOC tools and as a baseline or minimum, this, the SOAR can contain any system in the entire network. Now going back to the case management, the they'll have playbooks that are recommended based on the alert from the SIM and those will be, you have dynamic playbook creation and modification as part of those playbooks for the case management system and the case management system is going to have communications integrated, you know, chat, phone, [00:39:00] email, so that the transcript from the phone is going to be in there for bridge calls.
Any chats between analysts or analysts and employees are automatically added to the case. Same thing for emails. aNd there's going to be an AI integration for automated reporting for anyone outside of the SOC. So the, if the upper if management wants that under know what's going on, they'll, there'll be an A.
I automatically sending them updates on the progress based on the data that's in the case management system. And as we mentioned before, you have breaching tax simulation that's going to inform the prevention and detection or mediation for your, your your sock, it's going to validate and test all your detections.
Matthew: Yeah. And I think this, I'm sorry.
David: No, go ahead.
Matthew: I think that this really, I think that breach attack simulation is kind of the automate first in a nutshell. Cause the, the comparisons would be purple teaming and [00:40:00] red teaming, which still have a place, but purple teaming and red teaming are. You know, very tough to do.
They're very manpower intensive. Their activities that you do, you have a red team over exercise over like a two week period, or you have a purple team exercise. Breach and attack simulation. It doesn't do a good a job. It doesn't go as deep, but it does it everywhere. And it does it every single day.
It could even do it every minute if you were crazy and you wanted to do that. And I think that's kind of the site reliability engineering part where. Like when you're validating, your website is up. You're not checking to see if your website works once a year. You are checking every minute. You need to know that it works every minute of the day.
David: Yeah, and I think the you know, the the and the simulations could be built or should be built along with the detections at the same time,
Matthew: Yeah, I agree. You shouldn't roll out a new alert without a new detection, or you should have a pipeline set up such that it comes out pretty quickly afterwards.
David: and, [00:41:00] and along the same lines of the site reliability engineer, that's how all the security tools need to be managed to for updates configuration in order to ensure that the security tools are simply functioning and running, not talking about custom configurations and alert creation and stuff like that, but just the management of the care and feeding of the security tools should be done in a site reliability method. And maybe because of that, maybe your security tools aren't even part of the SOC, maybe it's a dedicated team in the IT space that will do the, the care and feeding of those security tools. And you're simply consume them. The, the trick there is that it would have to be a very customer focused, dedicated team, because you can't have them putting off security.
Tool configuration needs for the SOC or changes because they're off doing something else for another part of the IT. So that would be the, the wrinkle there. [00:42:00] Another characteristic of the SOC would be that it's, it's only Monday through Friday, nine to five, but it's got a 24 by seven 365 on call and the on call has to be very restrictive.
You know, no one leaves the house without a laptop and connectivity. Cause if, cause at this point where we're talking about very well tuned alerts. High degree of automation. If if it gets to the point where someone has to be called, the shit has hit the fan and they need to be on the ball acting very quickly.
Matthew: I've had someone tell me recently that no one good works night shift, and I'm sure there's exceptions to that rule, but generally speaking, probably mostly true.
David: Well, it really depends on motivation.
Matthew: Yeah. I mean, I'm sure there's people that are night. Yeah. That are just night owls and prefer working the night shift. But there's probably a lot of people that
David: Yeah. I mean, there's some, cause there's some people that are really good that simply don't want to deal with the politics and they'd rather simply do the work at night when there's, when they don't have to deal with a lot of people.
Matthew: it's a generalization, like
David: I know, no, [00:43:00] no, I'm not, I'm not disputing that. I'm just saying there are exceptions to all rules.
Matthew: That's fair.
David: And finally, a characteristic of the SOC is they've moved away from the team structure of tier one, two, and three. And I say that from a functionality perspective, not from an actual. Pay banning perspective. You'll start, you're still going to, I think you're still going to have tier one, two, and three analysts for HR and pay purposes, but that's going to be more a reflection of the knowledge and skill, but they have not the function or work that they do.
Matthew: Yeah. Just like junior analysts and analysts and senior analysts. Yeah. That makes sense. Yeah. Cause you still can't. And frankly, I think this is going to be one of the hardest things about this is moving away from the tiers is trying to figure out how to integrate brand new analysts into this. Where if you're expected to, well, you know, in the old model, the brand new analysts would spend six to 12 months at tier one, where they would just triage alerts and mark them as false positive and then push them up.[00:44:00]
Like they're going to have to learn fast and companies are going to have, well, we're going to talk about this later. I'm sorry. I'm getting ahead of myself.
David: You had better be sorry.
Matthew: Darn right. All right. So what are the pros and cons of a modern sock? So the first pro is according to Google, and this is weird. This is, this is, Difficult.
According to Google, the modern sock is so much more effective than the old sock that it's hard to believe. And they linked to a podcast there and I listened to the link podcast, but it didn't really provide much in terms of specific numbers. And I I'm, I'm a numbers guy. I'm a concrete guy. We do have some anecdotal metrics that I have.
I spoke with an analyst who worked for me years ago, and then he moved to a more modern sock at a bank. It's Frankly, like David was saying, it's almost kind of a maturity journey. You don't just get there. It's not like one year, you're classic. And the next year you're sorry, you're here. Modern. He moved to a more modern sock.
And he said, for example, they received an average of eight alerts per day, and they were expected to spend an hour to investigate each one deeply. [00:45:00] They had gotten rid of the vast majority of their false positives that wasted their time. From RBA presentations at conf I would say that RBA like aggregation would be considered a characteristic of a modern sock and Splunk came out with RBA years ago.
There is another tool that we talked about a couple of weeks ago that goes on top of data lakes and provides RBA like rule aggregation. It's been added. I think I was watching the presentation for Palo Alto the other day, their XDR tool, and it appeared that they've added RBA like capabilities into that tool as well.
So it looks like everybody's kind of moving to this over atomic detections, which I am all for.
David: Finally, everybody's wise enough
Matthew: I know, right but from looking at RBA presentations there was an Accenture presentation where they said they reduced event count by more than 70 percent and reduced their false positive rate by 30%. So by moving to these aggregation rules instead of atomic rules that fire off an individual thing that would get you a long way towards reducing the numbers to the point [00:46:00] where the modern SOC would work.
David: because I mean, you can't possibly have someone on call where they're getting a, you know, an alert every hour.
Matthew: Yeah. So Anton mentions this at least once, actually, he, he talks about where people are chasing the characteristics of modern sock without actually really looking at the root driving force of being a modern sock, which is the engineering led automation first. They're, they're kind of poking around the edges at how do I make my classic sock more efficient?
And frankly, that's a worthy. Goal, like reducing your number of true false positives, reducing your number of events, increasing automation. Those are worthy goals. If you're not willing to make the full on jump.
David: ANother pro of the Microsoft is it scales the old methods require more and more people and does not scale
Matthew: Especially as we keep trying to increase the number of logs and the number of things we monitor. It used to be that you just had to monitor the security tools. Then they added the operating systems. Then they added your cloud stuff. Now we're adding applications. Companies [00:47:00] have hundreds of applications.
How are you supposed to monitor all of that?
David: right. And with your atomic alerts.
Matthew: Yeah. With high false positive rates.
David: So you just can't keep throwing more and more people at it. It's, it's, it's like, like I said, it doesn't scale
Matthew: on building a
David: giant empire.
Matthew: building your empire, then it's great. I
David: with a hundred security analysts. Another pro is it, it's, it adapts faster, you know, with automated playbooks, detection as code integrations, it all makes it easier and faster to change and update with, as the world, as it changes. Another pro is the modern socks, more resilient. It's not reliant on heroes or tribal knowledge. Yep. Sorry, Matt. Having never been a hero, it's easy for me to say that.
Matthew: I think the biggest con is probably modern sock is going to be more expensive. I think in terms of people, it may be [00:48:00] less expensive in terms of engineering and tools potentially. Cause you're, cause a lot of current, a lot of classic socks bring in the tools in lieu of analysts or in lieu of better detection rules.
Cause they kind of expect the tool to handle that for them. But definitely in terms of people, modern soccer, you're having to hire higher level or more capable analyst engineers.
David: But you're gonna require a few of them. I think it may tip the scale at more expensive, but not much more. I don't
Matthew: that's, I mean, that's
David: It's not gonna be orders of magnitude more expensive.
Matthew: I think the real problem is you're going to be paying, if you're bringing in folks who can code reasonably well, you're going to be competing against software engineer salaries. And admittedly, you know, the tech giants have wildly inflated salaries, but other parts of the country are not as, not as crazy.
David: Maybe. We're talking about though that the fact that this is not going to be geographically restricted. I mean, because if you can convince you know, a extremely talented software engineer to live in Des Moines[00:49:00] and get paid, you know, an exorbitant salary for Des Moines but not for Silicon Valley.
You know, you can still get the same capabilities of people without having to pay through the nose for them.
Matthew: That's fair. think I think another con is this is going to be a significant change, both mental and in terms of. The company and organization the transition will probably be painful.
David: Yeah. And I think there's going to be a huge upfront cost in time and effort. to do this trend to do the transition.
Matthew: Yeah, I think I agree. And that actually leads to kind of the next one. I've been, I've been thinking about this a lot, especially I read an anecdotal story on LinkedIn there was LinkedIn, this article, there's, there's a big, long LinkedIn conversation. And somebody from an MSSP discussed how one of their clients came to them and said they wanted to transition to this modern sock.
And it apparently fell flat on its face.[00:50:00] They commented three issues. Number one, new analysts weren't equipped to handle more advanced cases. I think that you can handle by, you talked about before, like just going tierless doesn't mean that every analyst handles everything. You're still going to have junior analysts.
You're still going to have senior analysts, and maybe you look at the type of alert and how difficult they are to triage. They also mentioned two issues around senior staff, senior staff don't want to do the triage and they'll refuse to do 24 seven monitoring, which. If you want to do coverage and you're completely tearless.
And, and I think what it sounds like to me, the problem here is that they took a normal tier one through three sock and they just sort of flipped the switch and said, now you're tearless, boom, everybody does everything. And I don't think you want to do this as a one and done change. I think this is really a gradual change over something like two years.
And the big reason is. There's three things you've got to do over those two years. You have to refine, refine your content. You can't [00:51:00] move to being a tierless sock if you have all the trash alerts and the false positives. You have to learn to code. You want to be in automation first. You've got to either hire or build those internal skills so that people will start coding and automating and trying to reduce that manual toil.
And three, there's a big change in mindset. I think a couple of those things he mentioned about how senior staff don't want to do triage and don't want to do monitoring, like that's a mindset change. Although again, if they just flip the switch and the senior staff are now suddenly expected to triage dozens of alerts over an eight hour shift, that's not a mindset shift, that's a, that's a problem with your content.
David: Yeah. And I think we'll get there in a minute when we actually get into that, you know, how do you get there?
Matthew: I think one of the cons as well, there, there was an episode of down the security rabbit hole where a couple of people talked about modern sock and several people said that most companies are simply not aware there's a different way to do things. They've been doing the classic sock for years. It's been [00:52:00] good enough.
They haven't been breached yet, they're just still doing it.
David: And they're not hiring people that are thinking about it either.
Matthew: No, they're, they, they keep hiring more government people.
David: Oh, the government sharp people.
Matthew: Yeah.
David: Yep. That's why they call it Mil Spec. But how do you get to a modern sock in the average company? I would say, you know, step number one is you have to plan. You know, Matt and I have gone through several things where we talked about, you know, what a modern SOC looks like, what we think the capabilities of it are, but you have to build your own definition of what your modern SOC is going to be.
aNd once you have that idea, you want to write yourself a charter that says, hey, this is what what. A modern sock is for us, and then you start building your, your sub plans underneath that. Right? So you have your staffing plan. You have your tool plan. You have your integration plan. You come up with your steps and milestones.
And as Matt said, be realistic. [00:53:00] This is not going to be done in days or months. This is a year's long project. And then finally you come up with your budget estimate. And once you have all these things consolidated together, you're going to build your proposal for your CISO or your leadership if the CISO is already on board with this and how your options, what your options are to get there.
Cause as I said, this is going to be a multi year thing and you're not just going to flip a switch and be there. So, you know, you're going to have an MSSP during the transition. Is part of the team going to do, build the modern SOC while the rest of the team does the status quo, something like that to say, Hey, here's different ways or different options to get to.
On the path to the modern sock.
Matthew: Yep. And then then you got to talk to your sister and figure out if it's even possible to make a change like this. Another comment they made down the scary rabbit hole is that most sisters don't actually know what their sock is doing. They're just all kind of parodying the common knowledge about socks that every sock is buried in alerts.
And they're just working like little furious hamsters. I know a lot of [00:54:00] this. A lot of this may be on those sock managers. They want to puff up their own group. Like, you know, the, we deal with millions of alerts every day. We are so busy, give us more money or some such
David: Hmm.
Matthew: Yeah. Anton also says that the modern sock depends on capabilities that most companies just don't have.
And that's that code first automation first. We talked a little bit before that, but the traditional way of bringing an analyst is you hire them from it, or you hire them out of college. They don't have those automation skills. They don't have those content development skills. don't have those thread Intel skills.
Like you're going to have to train them or hire them. thought maybe some places are doing kind of a dual build like the RBA I mentioned from Accenture, they actually built up their RBA stuff simultaneously and left their old content running at the same time. So that way they didn't have a gap. But you could do that with the sock to outsource this year, one and two functions to an MDR and MSSP, then build the modern sock internally on top of that.
You would start them off as, you know, only working the escalated MSS, MSSP alerts, [00:55:00] which hopefully will have most of the false positives removed, but maybe not it would be more expensive, but it might keep the CISO and the risk group happy that you're maintaining the old ways which makes me feel like we're dealing with tribal priests here.
David: Yep. This is the way we've always done it.
Matthew: Mm hmm. bUt a lot of companies seem resistant to outsourcing the SOC. Well, I've seen it both ways. There's a lot of companies that really, really want to outsource the SOC. And there's a lot of places that are like, Oh no, we don't want to outsource that. Yeah.
David: and there's, there's part of that, you know, the tribal priests are like, we can't be a world quality organization without a 24 by seven sock. You know, that's what makes us a real company.
Matthew: Yeah. Everybody wants, every CISO wants to be able to walk into that busy humming SOC room and show off all the monitors.
David: Yeah. He went to stand on the bridge and, you know, look out over his minions.
Matthew: So if you don't have senior leadership buy in, you probably can't implement modern SOC fully. You could probably still work in characteristics and evolve parts. And that's still an improvement.[00:56:00] Cutting down on the number of alerts that you're bringing in and making your analysts more efficient is a huge win.
Even if you can't get all the way there. So step three, you've got the buy in, you've got the budget, hopefully you've got the plan. Now it's time to start finding people Oh, Oh, we don't talk about that. I said before that we talked about this already, but we haven't talked about this already. The traditional method of finding and hiring analysts is bringing folks in from the help desk or traditional IT or directly from college.
But Google, as I mentioned before, found that software engineers and cloud engineers had a better background and mindset for doing this work. And traditional hires struggled to change their mindset. The problem here is that Google has deep pockets and pays a lot. So this probably isn't realistic for most company to bring in software engineers and train them in security.
So in a smaller, less tech focused companies, you're probably going to have to upscale. You're going to have to fight for the training budget to do the upscale, which is a structural issue at most companies. They don't want to pay for the training budget because they'll leave. But the, the common response to that is still the best response is imagine that we [00:57:00] don't train them and they never leave.
Yeah. One of the anecdotes and the pros and cons is that junior analysts don't do well in the cheerless model. And you've got two options here. You can hire the junior analysts give them a short list of items to work on. So they focus on the easier stuff to start with and then train and push them hard.
Or you can hire a mid and senior, senior level people that other companies have trained and basically poach off their training. But that second one's a little more expensive
David: And part of that also, I think is mindset. You know, if you are skilled at picking people, you know, when you, when you talk to them in the interviews, and a lot of that goes towards mindset versus what they currently know or their current capabilities. That might go a long way as well. It's just picking the right, right person with the right mindset or the right driver or, you know, where they want to go and what they want to do.
Also versus saying, Oh, well, do you know C sharp? You know, maybe that's not really important if the person has the drive to [00:58:00] want to do what you're what you're trying to do
Matthew: Yeah. I always I always filter for that myself in interviews. Next. So now that we've, we've, we've started, we started hiring the right people or we're starting to train them. nOw it's time to start adopting a modified schedule for your analysts and start pushing them to be improving things. We talked before an easy way to do this the way that Google does this is they have one week on and one week off, or at least they did.
When I talked to them years ago, I have no idea what they do now. oN their off weeks when they are not. Off does not mean hanging out and doing nothing. Off means they are not reviewing alerts, and instead they are working on improving the current state of affairs. Now, that can be writing a script.
That could also be improving content via tuning. That could be hunting and creating new content. That could be working directly in their tools to tune it and make them more effective. That could be working on thread [00:59:00] Intel to write new content. There's a whole bunch of things that needs, it could be, but the idea here is, as we talked about stuff that typically in other companies is completely separate, you know, improving the content, that's the threat detection engineering team.
They're over there. They do that. We don't touch it. You know, the thread Intel team does. We don't touch it. Start getting them to. Spread their wings and stretch their abilities and start working more into sort of growing themselves, start being more than just an analyst.
David: and you may need to be fairly no, micromanage might be a harsh term for this, but I need to have. High level oversight initially on what's taking place, you know, so watching if you're doing the detectionist code, you know, watching what they're checking in, checking out have tickets to track their work on certain items if they want to, if they want them to get something done in order to see that they've started working on it, what their progress is and everything [01:00:00] like that.
Matthew: Yeah. And this is where I failed. Cause I had, I managed to sock for a while and this is one of the things that I tried to get them to do. And I would talk in our regular one on ones to try and follow up with them. But we can, week after week, they would have nothing. No improvements to talk about. And I didn't press them enough.
I didn't use the performance management system. I don't know if maybe changing the job description is an option, but there needs to be some way, and this went to, this goes back to what David said about growth mindset. If they don't have that growth mindset, maybe this is not the right place for them now.
There may be another role in the company where they can move to, or maybe it's time that they moved out of the company. If they're not willing to change, like David said, you have to, it's a change. You know, your job is no longer just reviewing the alerts. Your job is now reviewing the alerts and improving things. So, and frankly, that may be a justifiable argument for improving their pay [01:01:00] too, but that's beyond the scope of this discussion.
David: Well, I think they'd have to prove. But the, you know, prove that and
Matthew: Yeah, but that would, that would theoretically be part of your plan too, is that when you have more capable analysts, you pay
them more.
So this will also help in reducing burnout people that just review alerts day in and day out for months at a time and also will improve career prospects both internally and externally.
Cause they're doing more again. But that's frequently, I see this a lot on, you know, career advice forums are like, I've been an analyst for six months or 12 months. What do I do now? So next, now that you've gotten them started on working on other stuff, start relentlessly pushing. To automate all tier one and two tasks.
And any repetitive, boring task. rEmember how in the pros and cons, we talked about how a sock attempted to move to this model and failed due to having senior resources on 24 seven. So I would, I would leave this. I would still be a tiered sock while you work on automating this. As you automate more and more tier one and two alerts, as you're constantly improving threat detection, [01:02:00] eventually you will reach a point.
Where you're looking at the number of alerts you get in per day and you're like, you know what? We don't need to have people looking at this 24 seven. And if a sufficient, like a high alert comes in, your true positive percentage is high enough that it's okay to justify a call because it means something.
David: Yeah. And you could use those metrics as milestones on your path to transition, right? You know, say once we get the point where we're, we have this many detections and we're getting this many alerts, then we're going to go to nine to five with on call or something like that.
Matthew: And anecdotally I spoke with a major vendor sock. I don't know if they want their name mentioned. I should probably go through here when I edit and remove a lot of vendor names that I've mentioned here. Uh, I spoke anecdotally with a vendor sock. They moved to nine to five, five day a week sock, even though they're a major vendor sock and they are being targeted by APTs because they had automated it and tuned it to the point where having somebody on overnight simply didn't make.
Someone's still on call, but again, an alert comes [01:03:00] in, it'll go through the automated playbooks. And then if it passes through all of them, then someone will get a call.
David: Yeah. I wonder if you could actually work out with the executives that as well as a milestone, maybe you could work in a bonus to say, once we've reached this point, you know, everybody's going to get a bonus once we've made the transition. So that might be some incentive for people to work towards this being successful and implemented successfully.
Matthew: could be. All right. And our final step, and this one actually needs to run kind of at the same time. Maybe I should have put it earlier above, but changing how your threat detection, hunt and threat Intel programs all function. We talked before. We've talked a lot about how Threat Intel needs to change.
They need to be integrated into the SOC, they're not separate organizations. They need the SOC as their customer. They need to be providing the SOC and Threat Detection Engineering with useful data. Work on a smooth flow between them. Threat Intel should be constantly feeding small chunks of useful, immediately applicable knowledge of adversaries to Threat Detection [01:04:00] folks.
Analysts and Threat Detection folks should be constantly in contact with Threat Intel, letting them know where the blind spots are and where useful threat hunts could occur. Analysts should be constantly providing feedback to the threat detection team. All the stuff that David talked about earlier. I think that you should be cross training your analysts with threat intel and threat detection.
As I mentioned before, like you should have analysts whose kind of subspecialty is threat detection or threat intel. And that's what they do in their off weeks. Maybe you can still keep your teams if you really want to, but they should all be kind of cross trained and doing the same thing. All analysts should be performing hunts, and as I mentioned before, the threat detection, a big part of this is getting your threat detection so good that your false positive rate and your numbers are both very low.
RBA is one way to do that. There are almost certainly other ways to do it as well that I'm just not aware of. bUt these three teams should not be islands. Threat detection, threat intel. And the sock, they should be incredibly familiar with each other and in constant contact. And then finally David mentioned this already, but I'm going [01:05:00] to mention it again because it's also important here.
The focus on automation first as well for testing. Think again, this goes back to the software engineering. When you roll out a new product, you roll out unit tests with the product. It's the same thing. You roll out a new piece of content, content as code content development is software engineering, content engineering, software engineering.
So you should roll out the automated tests to go with it and make sure that everything is functional going forward.
David: Yep. And that's not hard.
Matthew: You're right. It's not hard.
David: bEcause once you've built the content, then what your attack simulation is looking for is the content you just built. So you might even be able to, to fully automate the creation of the, the attack sim content based on what your detection content
Matthew: Yeah. Yeah. Because what you're doing here is you're not testing whether you're, you're not testing breach and attack. Stimulation is testing that the content it is not testing. If an attack would succeed,[01:06:00]
David: Right.
Matthew: is like automated red team or an actual red team or a purple team exercise. Those are going to try out the new and novel techniques to see if they're detected by other things.
The breach and attack simulation. And they're, they're actually leaning away from this. I know we talked about this a little bit, but when I went to RSA, they're, they're starting to lean more towards automated pen testing. Instead of validating your content, which makes me sad. Cause I think they're getting a little off where they're most valuable.
Yeah. Yeah.
David: mean, that just sounds sexy, which is probably why they want to move towards it, right? anD this is the thing you talked about before you talked to people getting into cyber security, like, I want to be a pen tester versus I want to be a blue teamer. No one's, no one, I mean, when was the last time you heard that one?
Matthew: I'm like, Oh, maybe I want to get into pen testing. And I'm like, no, why am I
David: no, no. Remind me of a SF guy I talked to one time said you know, asking why, why he didn't go to ranger school. He said, sometimes I think about ranger, going to ranger school and I said, I would just lay down until the feeling [01:07:00] went away.
Matthew: Oh boy. All right. I think that's about it. Any last words, David,
David: Yeah, I'm done.
Matthew: all right. Well, that looks like we have all we have to say for modern sock. Looking at the clock, there was only an hour and 20 minutes or so. This is going to be the longest episode.
David: I Told you it was going to be, I told you, I warned you before we even
Matthew: Totally worthwhile though. Follow us at Sarengeti psych on Twitter and subscribe on your favorite podcast app.
​
