Getting more from Your Penetration Test: Stop Checking Boxes | SecurityMetrics Podcast Ep 99

Is your penetration testing just a compliance formality? This episode of the SecurityMetrics Podcast redefines pen testing as a strategic partnership, empowering you to get the most out of your assessments.

Join Jen Stone and James Farnsworth as they discuss:

• The critical role of scoping: Learn how to align business needs with technical assessments for a truly impactful pen test.
• The difference between a vulnerability scan and a penetration test
• Unlocking report potential: Discover how to leverage pen testing reports for maximum security benefit.
• Tips for fostering a successful collaboration with your pen testing service.

Stop seeing penetration testing as a checkbox exercise and transform it into a powerful tool for boosting your organization's security posture.

Bonus Resources:

• PenTest FAQs:https://www.youtube.com/watch?v=EECUTDMn43U 
• James' Previous Episode: Hacking Your Career: How to Become a Penetration Tester | SecurityMetrics Podcast 95

Hosted by Jen Stone, Principal Security Analyst (MCIS, CISSP, CISA, QSA).

[Disclaimer] Before implementing any policies or procedures you hear about on this or any other episodes, make sure to talk to your legal department, IT department, and any other department assisting with your data security and compliance efforts.

Request a Quote for a PCI Audit ► https://www.securitymetrics.com/pci-audit

Request a Quote for a Penetration Test ► https://www.securitymetrics.com/penetration-testing

Get the Guide to PCI DSS compliance ► https://www.securitymetrics.com/lp/pci/pci-guide

Get FREE security and compliance training ► https://academy.securitymetrics.com/

Get in touch with SecurityMetrics' Sales Team ► https://www.securitymetrics.com/contact/lets-get-you-to-the-right-place

Transcript

Jen: Hello and welcome back to the SecurityMetrics podcast. My name is Jen Stone. I'm one of the principal security analysts here at SecurityMetrics, and I'm really excited about the topic today because it's about penetration testing. And the reason I'm excited about it; it's really important. And a lot of people get it wrong just from the get go.

Jen: But I'm also excited because I have James Farnsworth back with me. And we had you on recently to talk about what it's like being a pen tester and how it feels being in the industry and how we dig through all of that. And if you haven't seen that, you need to go back and watch it because it was excellent.

Jen: We have a lot of different topics in our back catalog. So if you're looking for something specific, there's a lot there. Just go and search it. I'm sure you'll find it. In the meantime, thanks for being with me. Tell people who haven't met you before. Tell people a little bit about yourself.

James: Good to meet you. If we haven't met, I'm James Farnsworth. I'm a senior penetration tester here at SecurityMetrics. One of the other testers always laughs at me because I'm always like, I'm a penetration tester. He's like, you forgot the senior. So... 

Jen:  Hey, yeah, own that. You earned that!

James: You got to stress the senior. They have me doing a lot of different stuff. Today, I think she'll get into the topic that we're talking about, but I get to be here because I get to also be on sales calls. That's part of “the senior”. 

So that's I think that's me. I've been doing this for about five years. All kinds of certifications, education and enjoyment. So we like technology here, and

Jen: That's great. 

James: That's me in a nutshell.

Jen: And I get to be on sales calls too. And not that we are salespeople at all, and not even that we enjoy the sales process. But part of a good sales process is to get it right. What are we actually offering? What's the product that people are getting from us? And that's a difficult thing to kind of ferret out from the beginning, because a lot of times we'll be talking to people who they know they need something, but they don't know enough to help us. Right? So starting with the fact that a lot of people don't understand what a penetration test is and how that might be different from running scans, maybe you could give a kind of an overview on What is a penetration test?

James: The simplest way that I like to state it is it's just verifying a bias, essentially like a technical bias. So when we look at penetration tests, everybody thinks they know what's going on from a security standpoint. Everybody thinks that something's configured a specific way. Everybody thinks there are certain permissions. And so really what a penetration tester goes through, they go through and verify those biases. Is what is actually happening reality?

Jen: Yeah, Right

James: Like what you think is happening is that reality and and and so what we're going through and doing is they they call that testing. Yeah. And there are different products or services that align with like how do we define this. So we can have a common conversation. And again coming back to your backlog of stuff are webinars for like the types of penetration testing we do here and different things like that. But common ones may be web applications, networks, externals, phishing, those kinds of things.

Jen: Right. Exactly. The one of the things that I'll get, especially with people who are new to… Maybe let's frame it in terms of PCI compliance. 

James: Okay.

Jen: Or any type of compliance, HIPAA compliant doesn't matter your compliance. A lot of times the reason people are doing security, “doing security” is because they have to be compliant against something. And my favorite ones are the ones who just want to do all of the  activities that will help them assure that.

But let's be honest, a lot of groups do that because of a compliance need. And so let's say we have somebody who needs to be PCI compliant because they take credit cards in some way and they'll say, great, I need a pen test with that assessment. Right. And so we'll go through and talk about that.

But just because it's a compliant checkbox, they don't actually know what they're looking for. And one of the biggest misconceptions I get is that it's just a scan. But that's different right. So explain to me the difference between a vulnerability scan and a penetration test.

James: I think that one of the best ways to describe that is penetration tests are manual. Automated scans are just what they sound like automated. Right. And so you don't really have that human going through and looking at that and verifying what it says. And so what happens is you can have these companies do automated scans, and we'll go through it and say, hey, this is an issue.

Hey, this is an issue. Hey, this is an issue. And when you get down to it, sometimes it is and sometimes it's not. The big differentiator is it's manual. But the other thing you get with that as well as less false positives. So if you look at scans, that conversation of, hey, like this is an issue, it may or may not be one.

So less false positives, but also one of the other big things that you get is you get to consider the impact in that business that you're working with. So an automated scan will just have a very narrow scope and say, okay, there's this issue and it won't tie it in with other issues that are there. It won't try and exploit it most the time.

And then also like it's not going to tell you how that really affects your business, because it's just going to look at it from a technical side and it's not going to go, oh, this was in that database. And that's all of your customer information, their credit cards in there, or oh, there was a password in there. And now you can pivot to this other machine.

It's just going to be very, very narrowly focused.

Jen: Right.

James: I think a theme in this that we're going to have is why are you doing a pen test that comes through, right. Yeah. Why are you doing them? But we when you look at a pen test, it really falls into hopefully a larger security framework or a larger security strategy. So this is a tool used to reduce the load on manual testing to, to help you find known configuration or vulnerability issues.

Jen: I love that explanation because then it also takes and shows that there is an interaction between scanning and penetration testing. That's very important. So as as an assessor, I will not very often, sometimes I'll get a customer that reaches out and says, look, we we saw this on our vulnerability scan. This is not correct. This is it's not even going to affect us.

What do we do? And my response is always, if the scan... As an assessor, if I get a scan that's not passing, I can't as an assessor go, yeah, you're probably just fine. We're going to ignore it. I have to have a lot more information than that. And so sometimes an organization has the internal skill to evaluate what they're seeing from the scan, demonstrate that, go in and do the extra research themselves internally and say look, this is not a problem.

And here's why. Write it up. Document it somehow get that, you know, modified in the the with the ASV scanner that they're working at all, all of the, you know, ducks in a row so that a QSA can say, oh, you're right, it's fine. Otherwise we can't do that. But sometimes if they don't internally have that skill, that's where they're going to want to turn to a pen tester or rely on the penetration test that they're having done to use that in interaction with the scans and say, look, this is a false positive rather than just, oh, we got we got a false positive and now we can't pass our PCI compliance. Right. There are other things that can be done. And penetration testing helps with that.

James: Absolutely. And while you were talking, it was also thinking, one of the ways I got into pen testing was I was on the support side of, of, SecurityMetrics has obviously like a support team. Right. I was on there and they have basically scan techs is what they call them. Yeah. Which is not a common term.

Like one time somebody asked on my resume, what is that?

Jen: What is a scan tech?

James: I, uh, look at vulnerabilities all day? It's actually one of my favorite jobs. And so when you're talking about that, like there are different ways around stuff where if you don't have the expertise. So I guess that's a plug if you use us like we have a great support team. So call in.

Jen: Yeah. Actually does that work? That's the really cool part about it. So that sometimes you'll work with a scanning organization that doesn't have scan techs and they don't work with you on those things. So you have to know for sure what you're getting when you when you get a service. Otherwise you might get a scan that you don't know what to do with that issue.

Right. But, I love that that was kind of a jumping off point for you, because when you see these vulnerabilities and have to dig into them, it starts giving you the knowledge to become a pen tester. That's how closely tied a vulnerability scanning and penetration testing is here. Here's another thing about pentesting that a lot of people don't have a lot of knowledge on.

And that is, as you were saying earlier, that we have a lot of webinars out there, a lot of things. But but there's something that we say, what what is the penetration test perspective? What perspective are we going to evaluate?

James: So one of the things that while we're talking that I want to highlight is while we're talking about scope, perspectives are part of that scope. And one of the things that we were trying to do in the conversation was, was tie in. Well, what is a pen test? What is a vulnerability scan? Because when we're talking about scope, one of the common things that happens is people will say, okay, but you're going to run scans on that, right?

And so it's like, yes, we will. But when when that's a common question or something that comes up over and over again, one of these hard things is like, we don't always understand what a pen test is when we're asking for one. So one of the most important things when we start talking about these perspectives, or what is scope or those different types of things, it's not always like the most simple answer.

And so part of the answer comes from understanding the difference between things that are confusing. When you're talking about automated scanning and pentesting are very interlaced. And so they can get confused. And also pen tests can be expensive. And so so it's like understanding that and understanding the differentiators between the two help in the scoping conversation. Yeah. I also like that you're highlighting perspective as well because really when we're like looking at scope, like what is scope really?

It's essentially defining what a penetration tester can test or where they can verify biases. Yeah. And it's basically like the main question we're trying to to define. And the main thing that we're trying to solve when we have these sales calls or these scoping calls, is to define like what is the need of the customer. Yeah. And so these questions of vulnerability scans come up.

These questions of what is pentesting come up in these questions of perspectives come up. And so all of this stuff, if it seems disjointed, it's because there is a lot that goes into quality scoping. Yes. Because if it would be concerning to me, if you get on the call with Pentesting firm and they're like, no, great, we can just do it.

Jen: And they don't know anything

James: And they're like, what? And and it is also I think one of these questions of perspectives and scanning is important because one thing I think when we're trying to talk about what is scoping, if I'm somebody watching this podcast, I'm interested in how can I know? Oh, like, there's a lot of pentesting firms out there.

How can I know that I'm going to go with a good pentesting firm or somebody who has at least the intention of doing well. Yeah. And that they care about your business the way that you do. Obviously, nobody's going to care about your business the way you do, but cares, right? Because it's your baby.

Jen: Listens, and and takes what you care about into account. Exactly. Or helps you understand why you should care about things in a slightly different way or from a different persp-. I love that I asked you a perspective question and you pulled it back to scope, because scope is something that if we don't start with scope, perspective doesn't matter, you know, because your scope really is what's the environment that you're looking at?

And you can look at that from different perspectives. But if you're looking at the wrong environment, how you look at it is not going to matter. And and scope is scope is so serious that, in the PCI, the new PCI 4.0, the re-emphasize on an annual scoping exercise. And people are like, what is what is this?

And especially if you're a service provider and you have to do it two times a year. And so just kind of looking at what is scope, one of the misperceptions I get is, well, we're just looking at our PCI environment okay. But the scope of your PCI environment might be bigger than you think it is or it might be smaller.

So I get those both of those things happen. So so typically when I look at how do you scope something, what I'll say is in what way can the systems and network components that you have in your environment, in what way can they affect the security of what you're trying to protect for PCI compliance? And that is cardholder data, right?

Account data. And if you are in the health care world, in what way can those things affect protected health information or in general, privacy and security? PII, even if you don't have information that needs to protect it in your environment to keep your business running, you have systems in your environment that need to be protected. So understanding what you need to protect, that's what starts the whole I get really excited about it starts the whole scoping conversation.

Right?

James: Absolutely. and when you're talking about that, one of the things that I found interesting because you asked me a question about perspective and it for all this, like I normally do, I guess.

Jen: You're so good at that.

James: Yeah. I'm, I'm I'm gonna wear that as a badge of honor. You. We get to talk about more stuff, but when we're talking about, the perspectives, it is a rare circumstance. I won't say it doesn't ever happen, but when we look at part of the reason scoping conversations are so important is because resources are limited. Whether that's money, whether that's time, whether that's internal buy-in any of this stuff is limited and requires effort.

And on the flip side of that, so those were all resources on the business side, but on the consulting side or on basically like a third party side, like our, our pentesting firm. Yeah. we also have limited resources. Yeah. Right. And, and we also have limited time. And also it's this question of how can we make the most effective use of that time.

So when we're talking about this scoping conversation, you had mentioned perspective. One of the reasons that's so important, like perspective is basically where is your starting point. Essentially. And it's like I know that there are pen testers that I work with that would take issue with, with that oversimplification, but it helps me to think of that. Right.

Jen: Like where it provides clarity.

James: Like where is my starting point. And I think that when we talk about that and you were talking about, okay, maybe this is what I'm interested in protecting, but I also have all of this other stuff. Right. And so when we're talking about perspective, it's essentially where do I want to start the test. And when we're talking about scope it's like what can that tester move to.

Jen: Yes. Exactly. 


James: And so you can almost view that it's like different zones. But one of the common ways you could also define perspective is in the services that whatever that firm offers, whatever that consultancy offers. So if you look at us, basically our simple perspectives are internal network layers, which could be that could mean you have web applications in there.

We still touch them, but basically like your internal systems then essentially like your external systems is another perspective where it's like, okay, I'm unauthenticated, but I want to see what I can do from your external perspective. And then applications is another one. Like those are very, very common starting points.

Jen: And that's, you know, those those common starting points is actually what the PCI Council has us do in their recommendations or excuse me, in their guidance, in their, in their rules. So yeah. For reals. it is you need to do an external, network layer pen test. You need to do an internal network layer pentesting and you need to do an application layer Pentest unless they don't have a perspective of one of those.

And sometimes you don't. You know, if you don't have an if you don't have a, an externally available web application that in any way affects cardholder data, you're not going to do that one. Right. But a lot of times, it's a misunderstanding. People will bring me as an assessor. We had our application pen tested. Here's the result.

Great. Where's your network layer? We don't need that. Yeah. Uh-huh. and so I like that you kind of broke it down in that way, but then the lines get fuzzy a little depending on their environment. Right. So you're looking at, for example, an AWS instance, right. What are the what does that even mean. Do you have like virtual machines set up?

And then it's kind of similar to what we just talk about. Are you using service layer only then it's not, right. But all of these require a good conversation and understanding of the environment.

James: I think when you were thinking about that as well, I think you highlight a really good point where things start getting muddy when we start having basically third parties so integrated into everything that we do, but we don't have control. And I think this is fresh on my mind just because it's been an internal discussion lately about what can we do in these circumstances and what permissions do we need.

And so I think one of the things that I like that you're talking about is there may be a business process and something technical to support that business process, but we can't always test it because our clients don't know that on that. Right? Right. And so when you were talking specifically about, well, like, is that a service layer or like what does that look like?

One of the other things that, that we that happens is usually like if they have cloud infrastructure, you can still look at that in different ways, right? Like server-less stuff. Yeah. Right. Like that's just going to be application. Like they have API stuff. So you can still test it while you're still not testing that infrastructure that's supporting the server-less.

You can still do different types of stuff to try and help that. And one other thing to keep in mind is; different cloud providers also have different strictures around what you can and can't do. Right. And so part of that's part of the other thing when we're talking about perspectives and scoping, and it's like we're all over the place, but it's because scoping is all over the place.

Jen: Yeah, it is all over the place.

 
James: But part of that conversation is going to have to be around defining perspective. And, and this other stuff usually starts from, like all the way back defining why you want to get a pen test and so I like that you've been bringing up like, okay, like in the PCI standards. In this standard we have clients and customers that have PCI obviously, SecurityMetrics does PCI 

Jen: We sure do!


James: So we have a lot of PCI customers.

Jen: We do it with the best of them.

James: Yeah. So and then there's SOC 2 ISO 2700, those kinds of things that we also see. And then we classify like another one as elective where people just have questions. And so when when we talk about scope and perspectives and all of these questions all start with understanding the business need for a penetration test. Right. And that's part of where the education comes in.

That's part of where all of this stuff that we've been talking about bundles in, because this, this conversation of scope starts with the question of why are you getting a pen test, right? Because usually what it is, it's almost like this primer that gets hit where it's either that compliance need or it's like, hey, we have a customer who wants us to get a pen test. We don't know what that means. And all of these standards have different reasons for you to do all of that.

Jen: So this why that you're talking about, does that feed into the objectives that you were talking about from a penetration tester? Tell me about that.


James: Absolutely! Thank you. These objectives of mine, I like that question, and I like that question a lot, because when we really look at what a scope, here's my oversimplified mission statement again. When we really look at a scoping conversation, it's us trying to figure out why we need a pen test and how we can fulfill on it. Yeah, right. That's really what it is. And in order to do that really well. Coming back to one of our other podcasts is the soft skills are important. The communication is.

Jen: Yes.

James: And so part of the reason I'm taking that detour when you're talking about objectives is because one of my things that I'm trying to get out of these calls is a clearly defined objective for why that customer wants that pen test, and that could be concerns that could be compliance driven. And you'll get lots of different answers to this question of why do you want a pen test?

And it's like, well, compliance. And it's like, okay, or it could be, well, somebody told us to or it could be we're really worried about this thing. Or it could be we just had a security incident, we solved our security incident, and we want to re verify, basically, that we know that those attack paths aren't there anymore.

It ranges all the way across the board. But when you ask does that dial into objectives? I like that because that's the starting point of defining an objective is what was the primer that got you on a phone call with me? Exactly. And after we defined like here was what that primer was, now we can start going, okay, well, since we're going to do this anyway. What are you concerned about?

Jen: Right.

James: What keeps you up at night? Like when we talk about this? Like, what is your worst case scenario? And I don't ask those questions to be like, heh heh heh, let's let's enjoy that terror.

Jen: Yeah, Terrify you into expanding this. No.

James: But it's in those where we start to be able to have these statements of clearly defined, like, this is what I'm looking to get out of it. So when you say what is an objective, it's essentially what is that client's expectation from that penetration test. Right. And and if you look at the real deliverable, right, it's my report.

And that report is going to be tailored towards what your expectation is. Right. And that's where the conversation starts. And so that's really what we're trying to get is what is your expectation. How can I manage those expectations so that I can actually deliver on those expectations and speak to those expect.

Jen: Absolutely.

James: So when you say what is this objective. It's really like what is that customer's expectation. And can we fulfill it. And what would that look like.

Jen: Yeah there's got to be a beginning. "Why". And if you understand those drivers everything comes better. It becomes better in the engagement. And I find that as well in the, in the very close related full on assessment portion. So the penetration test is one way to look at an environment. And assessment is another way to look at an environment.

They are all in the service of protecting something. And what is it that we're trying to protect that that that underlying why it helps drive. What is the scope it helps drive. What is your perspective? And and even from an assessor's point of view. Just as a side note, sometimes it helps you evaluate weight not to.

Sometimes it always helps you evaluate whether the security control that's put in place to protect something is adequate. I think some of my most interesting conversations in my career have been when I have found myself in in a deep conversation, we're not going to see argument, but with very...

James: Do you have a lot of those Jen?


Jen: Look, sometimes, is because I want to know, 


James: Gotta know!


Jen: where if I don't get it, I'm not going to go check a box or something, but but the deep ones tend to be with very knowledgeable technical folk who are trying to satisfy a compliance, need and are seeing. And they're blowing up over a question.

James: So that's really interesting because what you're saying is you're you're implying that it's going to probably take more than one person on the business side. To help with success.

Jen: Yes, exactly.

James: So can. So I know that you do audits, but it sounds like scoping and pentesting can be very similar. How how you scope it out in regards to who needs to be on the call. So can you talk to me a little bit about on these scoping calls, what internal shareholders you need in order to have successful scoping calls?

Jen: Absolutely. If you're doing some sort of network layer pen test, you need your network person on the call. And that's the last person that wants to be on that call. Typically, I don't know if you've seen this, but, we'll get, business, from the business side. The operations side will get, project managers that are on the call will get business owners, on the call.

We'll even get CISOs and CTOs on the call, but they might not have that deeper knowledge of what's going on from a technology standpoint, to be able to successfully scope how do things communicate? Because if things communicate, that's the potential for bad guys to get through those communication lines, right? So you want the people who know how those communication paths interact.

And so that's from a network perspective. And that all that all informs the the scope of an assessment. Because if you can communicate with it, you can affect it typically. I mean maybe we have an argument for ping whatever. but then, but then from an application layer, the other group that does not want to be on this call is developers, right?

We want your systems. You want your software engineers on that. Call a somebody very knowledgeable about how that application or set of applications works and interacts. So here you've got the people who, and I don't know if you have this experience as well, but as an assessor, the two groups of people that are the hardest to get to sit down and talk to me are the network folk and the developers, and those are the ones you want on that initial scoping call.

And they don't want to.

James: Yeah, absolutely. And I and I think one of the things that I like that you're talking about is this combination of technical resources, business resources and essentially leadership within there. Yeah. And I'm not going to say every time or all of the time you need all of those because there can be enough ownership. But a lot of this where you had mentioned, there's just like so much overlap between these roles and so much that each shareholder is trying to get out of this, each stakeholder, whatever you want to call them, has skin in the game and it's going to affect them.

And you're going to need buy in from all of these parties in order to have successful audits or successful pen tests. Because when when a hitch happens, yes, a big deal in these assessments and these pen tests is making sure that we can communicate, have fast responses right where it's like, hey, guess what? The internal agent we were using to access your systems, it went down and it's a physical agent.

So like we need the network guy 

Jen: Hands on. 

James: Yeah. And so I think one of the reasons I really liked what you were talking about is if you can get all of those people bought in, you're going to get much, much more out of the professional service that you are trying to work with, irrespective of the professional service.

Jen: Exactly.

James: So I like that.

Jen: That you know, the other value in these people and I, I'm not sure if the penetration testing team sees this maybe as much as, as an assessor, but a back to the argument thing when somebody from the, it's a software engineer really knows, or systems or networking folk really knows their systems and are arguing with me about something in a compliance thing.

Typically it's because there is a misunderstanding on how their technologies work, or either they don't understand what PCI is asking for because I haven't explained it well enough, or I don't understand their technology well enough to be able to say, oh, here's how you meet that probably already. Let's take a look at this. So these, these these, kind of flares are actually really good because they, they're indicators that something is not well understood.

James: Yeah. The thing that I think is funny is while you're talking, you know, like we talk with the business people a lot and I'm like, we get a lot of technical people, which is awesome. I want, you know what I mean? Like, I want them on those calls.

Jen: Absolutely.

James: I want them telling me about their systems. But one of the reasons I'm highlighting that is on these scoping calls, they tend to have, everybody likes pentesting because it's like the technical, yeah, we love it. And I'm like, yes, we should, we absolutely should. And I love it. One of the hard things that happens is because there is this I, I don't know where the difference is.

Maybe it's because it's not governance, risk compliance or right, like, I don't I don't know where the differentiator happened. But one of the roadblocks that happens is we start having these conversations and technical people will get on the line. And it comes back to this perspective question, and they will just start hammering. This is what we use.

This is what we do. This is how like our tech stack looks. And they're super proud of it. Yeah, they should be. Yes they absolutely should be.

Jen: Yeah.

James: The hard part that goes along with that is when we start the conversation from the technical side and neglect the business side, and it's really hard to understand. Okay. You do have this huge tech stack. How do I prioritize right. That tech stack it is. And so I when you're like I don't know if you have this. Yes we have it.

We also have it on the other end of like we we get really, really invested people from that technical side or from the security side where it is also like, I may not be in the weeds as much as the actual developers or the network engineers, but I am in charge of the security of this. And so we get that that love and deep passion of the actual technical systems they're using, which is amazing.

Jen: I love that.

James: And I think one of the reasons I'm highlighting that is we're going to have like, sometimes people roll their eyes when I start asking questions. I think that I'm always going to ask about the business side first.

Because it will qualify what you're telling me. Technically, right? Because at the end of the day, as may be broken as it seems, there is a business need for this pen test.

And that business side is going to define what is important technically. and then the reason I also ask that is from is, is to actually be a champion for the technical resources on those calls, because at the end of the day, when we're coming back to why are you doing this? It is very common for technical resources to say, hey, we need to change this.

Hey, we need to change this. Hey, we need to change this. Hey, there's a business side. Not like a business need. Hey, this is liability. This is like. And they get ignored.

Jen: Yeah! 

James: And so they'll come in and they'll go, okay. Part of the reason you do a pen test is to verify a bias. Right? And so when they're saying, hey, this is a problem, your bias is wrong. They go no it's not. And then you go, okay, well you're doing a pen test anyway. Like here's the business need. And we can tailor that to get internal wins.

Jen: Yes.

James: And so I think when you're talking about that, that's where my brain went where it was like, okay, from that side even if you're a technical person. Yeah. And you're very proud of that. And we want to have those conversations because it is it will save them time, them money. It will save us time. Yeah. And help us prioritize.

But the business side of even if you're a technical resource why are you doing this business need is going to reinforce your technical information, your technical knowledge, your technical concerns.

Jen: I think that's one of the most important things that we as a third party can do for for people that come to us is be that nexus between your technical and your business side and help them understand where each is coming from. And so that makes a much better outcome for the work that we do. Creating those communication paths.

James: I also like that because I think and I know I keep like hammering, like there are lots of stakeholders, so make sure they work together. But it also is like an ongoing thing where if you even look at like our side of things, we also have technical and non-technical stakeholders to deliver on a project. And so when all of that is coming together and we can communicate on both of those sides and have everybody on the same page, it's going to make not just that scoping call, but that pen test, that audit, whatever the professional services. So much better.

Jen: Okay. So I'm going to take a short reach or no, please do something completely different.

James: Please do.

Jen: All right. Pentest is done. I know it's done. Where is my report?

James: Oh, Okay.

Jen: Why does that take so long?


James: Okay. Well, I was going to use profanity. I'm not going to now.

Jen: But everybody asks me, so we need to talk about it.

James: It's like, what kind of quality of report do you want? And that's just like the unfiltered side, right? Like, and and the reason I say that that way is because I want to do a good job on those reports. Yeah. And and the person on the other side also would like a good report. Right. And, and part of this scoping conversation, I, like you said, when am I getting the report!

One of the questions if it's not asked that needs to be addressed. So this is for everybody on there. If it's not asked is what's the timeline you're looking for.

Jen: Yeah.

James: Because if we're looking at this expectation side of thing like what is the scoping calls. What are your expectations as part of the expectation. Part of the reason that gets asked so often is because the expectation of timelines aren't set from the customer side, right? Or from or from the professional service side. Right. And so this could be in regards to any firm.

Right. and so I guess I'm all full of tips today, but I know 

Jen: that's why we're here. 

James: Yeah. Another thing when you're to consider when you're going into these scoping calls, because I think one of my goals when we're talking about scoping is why am I interested in education for like what is a scoping call is because like, this is what I do for a living.

And so to me it may be very simple to be like, well, I know I'm going to do this scoping call. I know what to expect. I know what the common gotchas are, but like if I'm doing something in my house, I don't know what the common gotchas are because I'm not always in it. I don't deal with plumbing like I don't deal with electricity, but if somebody was there to say, hey, consider this, this is one of those big consider this moments where being clear in your expectations, this is a really, really big one.

Because if we're looking at the business side of why am I doing this? It could be compliance driven or it could be, like a customer driven. And there are deadlines, right? In regards to those. So where is my report? Yes. Yesterday would have been great to have the report, but it takes a while sometimes. One that can be one of the reasons.

Expectations on both sides weren't set. And when we have these conversations, that's why I like being on the scoping calls. That's because we can set expectations and understand expectations so we can fulfill on them. Which is just it makes everybody else's life easier. Yeah. But barring that, usually if that first point isn't like is addressed, where it's like expectations on both sides are agreed on.

Usually there's not any surprises, right. Which means you don't have to worry about that question. But sometimes and I will say sometimes, because people are human and it is a professional service. And we talked about that manual piece as opposed to automated Humans are humans 


Jen: Humans are humans.


James: And so if it's like, where is my report? Maybe something happened in the testing process where we thought something would happen in the setup because like, if we're doing an internal test or we're doing an application layer and you're supposed to give us credentials or something happens, and maybe that sets us behind because it took longer to provision the accounts, or it took longer to reset the beacon or any of this stuff on the timeline.

Usually the second place where you can see that causes an issue and where is my report is maybe there was an issue in the process, whether that's on the business side or whether that's on the professional service side. Yeah, it it could be in either of those. And so this again comes down to expectations and communication. And one of the reasons that those scoping calls are important is because if you've defined that expectation, if it gets to this second layer of hey, we have an issue because there's a breakdown in process right now, we can speak to the expectation and say, what can we do?

How can we do it? Because I think in there ownership is important on both sides, because if there's a problem, like if somebody is taking ownership on either side, that yeah, it's a problem that you're not getting your report on time, which doesn't happen for the tests that I do. Usually just saying.

Jen: It's not as common. Yeah. But I wanted to explore the ideas why and what people could do about it.

James: Is the biggest thing is communication.

Jen: Communication, communication. I've had a couple times where the customer has reached out to me and said, I told them I need my pen test done on the 16th of August. Did you tell them that's when you needed the test done or the report? Did you tell them that's when you needed to report in your head? Because those are two different.

Maybe they were like, yeah, we can do that on that date for you, thinking, oh, they have other things happening they want not that you're going to get a one day test, but you know what I mean. Yeah, but there was a time frame that you had to fit in because for some reason maybe they're doing a tech freeze and you want to, you know, put it in that time.

But if if the right information wasn't understood at that initial thing, they're not going to know, oh, what you meant was you wanted to report that day because of these other things that have to line up with time. If they had understood that, then that would have been the pintos would not have been done that day. It would have been done weeks before.

Right? So there there is a a process both for penetration testing and for assessments where you're doing the evaluation. But writing up the report takes quite a bit of time. And you're I mean, look, not to throw shade, but you're not writing 500 page reports.

James No, I'm not.

Jen: but you are still writing reports that take thought and it can take days to put it together and and to go back, especially if you're trying to balance it out with other customer's needs. Right. And so that was that was rude. I'm just salty right now because.

James It's okay I...

Jen: These new 4.0 are just exhausting. Yeah.

James: That's like that's like ten times the size of our reports.

Jen: Don't don't get me started I, I'm four days behind a report writing right now. And it's just because there's my estimates are off because the 4.0 has just. Yeah it's slamming us. but either way. Yeah. What I, what I really like though is you're not just writing. You're not just like, throwing technical things on a page and walking away.

There's actual words explaining and recommending and, and and supporting future needs for these penetration tests for the customer. It's not just here's what we found by. Yes. And then you also have a QA process where somebody goes through and says it, are these rational sentences right. And can we because we want to make sure that whatever we're delivering can be, taken and used, not just, oh, we have this other box that's been checked because that's of no value.

James: Absolutely. And one thing that I like that you were talking about there was like, we're going to put thought into it. And so when you start talking about that, I started thinking about our there's a part in the report that's called the narrative.

Jen: Yes.

James: Right. Where it where it's, it's not like the here's my elevator pitch. It's it's the here's my longer. And then we can get into the weeds with the issues later. But it's like the, the so what of the report I was. Yes. And the reason I'm bringing that up is because this conversation of scoping like if, if I'm putting a plug in for a pro tip of like sometimes I feel like people are like, we just want to get this out of the way.

A common gotcha is it's almost like an eye roll, not all the time. And I'm not trying to, like, drag anybody through the mud, but it's like it's sometimes hard to see the value people get. Very like, I'm doing this in my job. I need to do this thing and like, I just need to get through this. But one of the reasons the scoping call is so important, and one of the reasons we take that time to talk about the business need, the technical perspectives, the technical stack, all of that is for this "So what" part of the report. 

Because if we don't have that objective, like your expectations, like what you were looking for, the deliverable you're looking for, it's kind of like pin the tail on the donkeys type thing. And so that's one of the reasons when you were talking, I was like, that's a very good thing that you brought up in reports, because they are very heavily tied to setting your to set for success means a good scoping call in those so that we can give you a good deliverable.

And that's really for Pentesting the report is the deliverable, right.

Jen: And it just keeps coming back to that question why are we even doing this anyway? And if everybody can be on the same page from the of that from the beginning, the entire process is smoother and the results are going to be much more applicable to the environment that the and the customer that's receiving it.

James: Absolutely.

Jen: So, well, we could go on, I think on, on and on about penetration tests, but I think this is a really good place to kind of wrap up. Is there anything that that you really thought maybe we should talk about from penetration testing before we close.

James: In regards and within the scope of this conversation of scoping.

Jen: Of scoping and scoping.

James: Yeah, that communication is the biggest thing. And I think when we're talking about pentesting services, one of the ways that we get the most out of these pen tests is through communication. And so when you're like, what are you thinking about? I'm like, the big thing that I want in these scoping calls is communication. And partnership. Because if we have that we can deliver.

And so if if everybody forgets everything we talked about and you have concerns and all of that, just get on the call, even if you don't have all the stakeholders, if that's not optimal, just communicate right. And we can work from there. So that's the big thing is what do you want? And I think it comes back to what we were talking about last time too, where it's like the soft skills.

Jen: Yeah. Critical 

James: Critical. And so yeah, that's that's what I think of when you say, what's that last thing? Just communicate with us.

Jen: Absolutely. Well, we're going to have to have another conversation because we have not covered everything with penetration testing yet. It seems like this is going to be a series that. Thank you so much for joining me today. And if this is your first time listening to James, I know that by now you're saying, oh, I'm going to go find his his.

The other thing that we did, it is also on penetration testing, but it is more on the softer side of things. And then all of the topics that we cover, we typically cover everything from security, just straight security to compliance to risk governance, even a little bit. we, we talk to specific people about specific solutions. Sometimes we try and do that without being super salesy.

And, I think there's something for everyone in this space and really grateful for everyone who continues to to listen and, and, offer comments. Thanks for being here with us and we hope to see you again next time. Thanks for watching. To watch more episodes of SecurityMetrics Podcast, click on the box on the left. If you prefer to listen to this podcast, it's available on all your favorite podcast platforms.

See you on the slopes.