Product Ownership Capabilities Series - Evolve with Validation

Paul Kuijten and Fredrik Wendt join Dave West again to discuss the third piece of the Professional Product Ownership Capabilities model - Evolve with Validation. They discuss the importance of understanding and validating the "why" behind product features. They discuss the evolution from traditional requirements to user stories and outcomes, and the need for continuous experimentation to validate assumptions. The conversation highlights the shift towards more empirical and economic validation methods, the role of AI in enhancing this process, and the importance of team collaboration in achieving product success.

Transcript

Lindsay Velecina: 0:00

Music. Welcome to the scrum.org community podcast, a podcast from the home of Scrum. In this podcast, we feature professional scrum trainers and other scrum practitioners sharing their stories and experiences to help learn from the experience of others. We hope you enjoy this episode.

Dave West: 0:20

Hello and welcome to the scrum.org community podcast. I'm your host. Dave West, CEO here@scrum.org today's podcast is focused on the capabilities and skills professional product ownership, something very near to my heart, the professional product ownership. Capabilities and skills provide a view of what we believe@scrum.org are important skills and capabilities product ownership, they describe the things a product led organization needs to think about when evolving its capabilities. So if you're product led, you need to think about these capabilities. To talk about this very interesting subject, actually joined by two fabulous PSTs, professional scrum trainers, Paul chiton and Frederick went they were heavily involved in working with the community building this this model. So I'm really excited to have them on the podcast. Welcome to the podcast, Paul and Frederick.

Fredrik Wendt: 1:18

Thank you.

Dave West: 1:21

It's great. It's great. I'm doing very well. It's nice to see your smiling faces. Obviously, our listeners don't see that. But just for reference, listeners, they are both smiling because they're also excited to talk about this. This is actually, though, I mean, the fourth podcast. You know, it's like a Star Wars movie. Now, right? There's a fourth podcast on this subject. Hopefully this is better than the fourth Star Wars movie. But anyway, and in the first podcast, we discussed the overall model. The second we we zoomed in on engage with vision, which is the first sort of tree of the model. The third podcast, we discussed the lead to value. And the final podcast, we're going to talk about evolve, revalidation, the final leg of the model. If you want to remember the model, it's, it's pretty easy. It's vision, value and validation. It's the three V's, which, if some of you attended professional Scrum Product Owner class you you'll be aware that this is a sort of theme where we've expanded that and really added a lot more content and and detail about these 3v of vision, value and validation. And today we're going to be talking about validation, evolve with validation. Frederick, can you kick us off and sort of give us the sort of high level view of this capability, absolutely.

Fredrik Wendt: 2:46

So in relation to the other legs, this is, in a way, the smallest one, because it has the least kind of topics or areas to master, but it's a critical one, and so it's not the, you know, least important, just because it's the last one. So the overall idea is complexity and learning and not knowing what the market will think and what users like, what will delight them, etc, and what to do about that problem. Really. How can we grow our understanding of all of these topics and subjects in in a way that is not just ad hoc and not just random and not just, oh, we were lucky. So how can we use validation to expand what we know and learn as quickly as possible about the all of the unknowns around the product?

Dave West: 3:39

No, so, so, so, so Paul, you know, you do this quite a lot. You spend a lot of time with clients. That's one of your things. Tell me a little bit more. How do you validate ideas? What is all this about?

Paul Kuijten: 3:53

Well, I think the first thing that that's like, traditionally the home base and where it all started. Maybe you might even say 30 years ago, with Scrum, it's this whole validation that we with Scrum called empiricism, right? So it's delivering a product every every two weeks, and then also seeing what that product does in the marketplace. So over the years, we become better and better at delivering a product every two weeks, but hey, seeing what that thing does in the marketplace, still needs a lot of improvement. So, so, so, so my my efforts with clients are really based on that, because a lot of them, they have the scrumping down. They're delivering regularly, but they don't really have that feedback loop with the customer in place. So, so, so that's, that's one element of it that we work on a lot. The other element is, we used to think very much that building actual product is the means. To validate whether you're heading in the right direction. But, but there's, there's, there's many more ways nowadays, so, so, so you don't have to actually build product to collect data to to get some sense of whether you're heading in the right direction. And that's, and that's what we call discovery validation. Is what we just built, the one day skill scores on as well. But that's an area that's becoming more and more important for product ownership, so that you grasp that as well, and you can not just build too much stuff and invest too much heading in the wrong direction. So those are the two main things I would say, Okay,

Dave West: 5:41

so let let me see if I get this right. So yeah, it's funny, as you were talking both of you about the sort of evolution of this thinking in the context of Scrum. When scrum was created, delivering working software, which was really the focus of Scrum at that time, was really hard. It's sort of like an act of God to get stuff out the building, you know. I mean, it was, I it was, you know, you had to move CDs around. I mean, it was tapes. I mean, it was, it was hard. So the emphasis was very much on done and actually proving that you could get and refining that muscle. Now, what's interesting though, is, as we've improved our capabilities in that space, and and, and as Scrum is being used in more and more variety of fields, the actual delivery that, though, is still a challenge. Let's not get us wrong. Isn't perhaps, you know, as important as delivering the right thing, right and what we're talking about in this area, really is, how do you deliver the right thing, and how do you validate sure you're continuously empirically. I love that word, Paul, delivering the right thing over and over and over again. Frederick is that is, am I got that right? Yeah.

Fredrik Wendt: 7:00

And I, I just remembered that there's, I saw this recording of a talk that Ed Deming, did you know? I think it's back in the 80s, and he has this phrase. I'm not sure exactly how it was, but it's something like, there's nothing as useless as perfectly building the wrong thing. And I think that's kind of head on what we're aiming for here, right? We, like you said, it took a lot of effort back in the days to get products out and later on, learning whether it actually was the right thing or not. Sure you can do some kind of interviewing and user status, etc, but it's when it's in the hand of the real users that you get the real learning and

Dave West: 7:42

and also, let's just be honest, back in the day, hate to say it that what you kind of were given, what you were given, and you were willing to use it, because that's all you'd ever get given, and it was better than perhaps the environment that you, you know, previously had. But now we've got more sophisticated users doing more complicated work, you know, if it's an internal system or if it's an external system, you've got the, you know, there's the sheer competition of demands on it, on a use, on a human being, like, you know, whether it's banking or Whatever, in the you know, generations, Generation Z, Generation X, are more likely to move based on the experience that they have with the service that they provide. So, so you've got all of those things, and it's really changed. So, Paul, when you're working with clients and helping them do that, is it what? How do you do that? I mean, what? What are the sort of ideas that that you have to, you have to solve or get in there.

Paul Kuijten: 8:46

Yeah, so, so, I think you know, over the years, we went from requirements, then you had business requirements and technical requirements, and we sort of already shifted that in a lot of areas, to have it not be requirements, but user stories. That's also a user story, which is not necessarily a requirement, but much more like a vehicle to engage in dialog together to come up with the best solutions. So that's sort of a evolution of what teams were working on. But I think the next part of that evolution is okay, you know, we're talking user stories here, but we're still only talking solutions, and we're not talking about outcomes and impacts that we intend to arrive at with these solutions. So so a lot of it is, and I think it's my dream that nothing will ever be pulled into a team with any customer outcome or desired business impact associated. With it being front and center. Yeah, so that's really what's important, I think, to bring that, to bring that desired outcome for the customer, to bring that desired impact for the business at the at a more granular level, at the smaller level, instead of these big business cases related to projects that nobody ever looks like, looks at, and then also be able to measure, to measure whether you're actually achieving that stuff. So, so, so that's like the the major bit, and that's also why we think it's such an important part of product ownership, because people employing product ownership, are also owning what lands in those teams. So It's too, too big of a thing to leave it up to the UX designer in the team, because a lot of those ideas have been in UX traditionally, but it's way bigger than that. So, so that's sort of the kind of things I'm trying to get across and actually get some, get them, maybe, to phrase some experiments, instead of, is there a product backlog items, these kinds of things

Dave West: 11:15

you've said so much there, Frederick, you're sort of dying to jump in,

Fredrik Wendt: 11:22

which was to build on the next thing, but and what all of the things that Paul described, part of this evolving with validation is also figuring out, okay, so we ran a test and experiment, we did an AB test, or we something like that. And so what do we do next? And figuring that out how to feed the learning back into a decision making process. That is the essence of leadership and product leadership, making decisions, where do we want to go next? And so that whole how you set up yourself and your team and your organization to make decisions, that is essentially how you evolve your product, right? So being cognizant of that decision making process is also tightly connected to the data and the learning we get from these experiments and the validation we do. Can

Dave West: 12:15

I just sort of like lean into something, because I think this is something that's very often confusing, and I really would love you to to sort of put the record straight here. So I, you know, I'm working in a team. I'm given a I'm given a backlog. I know that I'd said that on purpose, by the way, and and I took those items put in my sprint. I'm working on an item, I'm delivering a feature. I'm not doing experiments. So how the point that I think you're raising it sort of like tickles that What? What? What do we mean by experiment here? Because I know that when I was working at commercial union, you know, in in the 90s, listening to, you know, Blur and Oasis, whilst I wrote code, I didn't think I experimented at all. I delivered features. So what would you say if somebody said, I don't experiment. I deliver stuff.

Fredrik Wendt: 13:15

Oh, yes, you do. You experiment in the most expensive way. You build it with high quality, which I assume you did, Dave, and you excellent clean clothes and all of that.

Dave West: 13:26

Let's not go that far, but yes, let's pretend Yes. But the

Fredrik Wendt: 13:29

overall idea you You did a very good job. You made sure it was going to scale, it had was secure and all of these things. And then you shipped. And hopefully someone had decided, whether it was you or someone else that this was going to create value, and you just hope that it's going to create value. So the question is, and you likely got some feedback, right? Either people loved it, or they used it, or they sent a bug request, or they just said, No, this is not the right product for us anymore. So that is an experiment. It's just very expensive, very high risk. And the way we want to address in this capability area here is to try to find ways of learning these things with lower risk, lower

Unknown: 14:14

cost.

Dave West: 14:17

So so when I'm so hang on, so I I've been given this feature. So you're telling me that the first thing that I should ask is, okay, what's the cheapest way that we can, we can, you can do some work to validate that this feature is actually the feature that they want, that it delivers value. Yeah, is that? Is that right?

Paul Kuijten: 14:39

So, so, but there's a, no, there's a, there's a step before that. So, so, so you're gonna need to ask why a couple of times to figure out what actually the intended value is with that feature. Now, because you get you're not gonna, you're gonna, you're not gonna even spend the least amount of money if you don't know what the desire. The outcome is so,

Dave West: 15:03

so hang on. Are you telling me I just need to get to the bottom of this? Because, you know, I'm pretending to be, you know, Dave West, in the, in the, in the 90s, I still had hair, by the way, which was a, which would be a shock until 96 and it will fall out. But anyway, so I am Dave. I got I'm Dave West, sitting in the 90s, I've been giving a feature. The first thing that I need to ask is, why? What is the intended value? Is that? Is that? Is it my job to determine that? Is it everybody's job to determine? No,

Paul Kuijten: 15:35

not. I think, I think that's the job of product ownership. It's not the job of you as you Dave West, as a developer, right?

Fredrik Wendt: 15:44

So, ah, but that's the team accountability in Scrum, right? If we want to lend this in to make sure that what we're doing is useful and valuable. So you got your feature request, Dave, right, that you had in your hand. So I would say, yes, accept it and go with it. But you know, before you start jumping into thinking of solutions, etc, just make sure, do I understand? Who is going to be happy when I deliver this? And if you don't know, someone usually knows, right? Or you could ask, What if this is actually going to take, you know, three months, or we're not going to be able to complete this technically this year. What happens? And when you say that to whoever you talk to your stakeholder, or someone that asks for this feature, then they're going to start explaining the value. If we don't do this within six months, we're going to get fined. Okay, so you want to avoid the fine. That's the end goal we're trying to achieve. Okay, now I can go with that, and I can make sure, and I can start taking accountability and responsibility of actually achieving this outcome that the stakeholder actually wanted.

Dave West: 16:49

So what you're saying is that at a product owner level, it's important that you refine the product backlog items so they have clear connection to the why. At the developer and team level, the scrum team level, it is our responsibility to ultimately take that why and really lean into it, to build experiments and functionality and capabilities that both validate that why and execute on that. Why in the most economic form is that the essence of what you're describing, yes,

Paul Kuijten: 17:26

beautiful, beautiful, yeah.

Fredrik Wendt: 17:28

And for some of those features and ideas, there's higher uncertainty or there's a higher risk on problem when you definitely do it wrong. And for those you want to use cheaper techniques than just building it, shipping it, and see what the market thinks. And that's where we have this capability of evolving with validation. How can you learn and validate stuff without not just

Paul Kuijten: 17:55

building it? But there's yeah so, and then there's a, there's a ton of stuff related to that underneath. So, it's about it's about matching the investment level on the confidence that you or the evidence that you have. It's about designing experiments or hypothesis that will give you a high likelihood of getting some meaningful data. It's about how you feeding data back into the decision making process and all of that stuff. So that's, that's really what this is about. Yeah,

Dave West: 18:32

so what I'm taking from it really, which is something that you know I probably should already know, considering I'm a product owner is that ultimately, this, this distinction about the why and how and what, and really it is a team sport. However, I'm leading the team with respect to the why, and then the the team engage everybody is responsible for, for the economics of the delivery, really, and, and that's something that historically, teams have not felt necessarily, that accountability, you know, you basically, I deliver amazing, reasonable quality, though I always aspire to a little bit better than I did. You get sidetracked on a Friday, you know? But, you know, I, I had, I everything was a nail that I was given, and I hammered it. I never once stood back and said, Okay, I understand the why. Let's look at the, you know. Let's look at the predictability of that outcome, and let's look at validating it before we invest in 100,000 lines of code, and Dave West's amazing piece of software.

Paul Kuijten: 19:52

And I don't know, yeah, Frederick, go ahead.

Fredrik Wendt: 19:56

So even Ed Deming, back in the days they had this. Problem. But what has happened in the last couple of decades is that the world has turned a lot more digital. A lot of the services we consume and use, et cetera, they're digital. And so the pace of how competition can ship and learn and get in front of customers has changed a lot. And so while you Dave, you weren't asked that question as often, is this going to be? You know, financially, economically wise, we in that this landscape of a higher level of competition, it's just a tougher climate these days, and we need to be more certain that what we're doing is going to be the right it the right thing.

Dave West: 20:44

Yeah, I think, I think that is that, yeah, in the 90s, you know, it was he wasn't a massively digital world that we were living in and the level of competition, it was, you know, that, what do they say in the land of the blind, the one eyed man is king, or you don't have to be the fastest runner. You just have to be faster than the slowest one. You know that that was definitely the world that we lived in as we've become, you know, as everybody's got faster at running, that bear is getting very close to biting us, you know. And I think that everybody has a responsibility and an ownership. And that's one thing I really like about Scrum that is a little different from how traditional, waterfall esque, you know, even, dare I say, Silicon Valley's approach to delivering software is, you know, we encourage more ownership by the teams, more engagement by the team. So that product ownership, though there is a product owner, ultimately determining the why and set in the direction. It's, it's a team sport, really. It has

Paul Kuijten: 21:53

to it is, it is and it's, it's gonna become even more wild in the in the quite near future. Because, like you said, we becoming better at running faster, but, but hey, we have this whole AI thing coming, and that will, that will that will make everybody able to do anything. Dave West could be a movie director. So, so, so, so the whole, the whole emphasis will be shifting towards IDs, towards, hey, what outcomes would be valuable? Hey, how are we gonna have an impact? That it will shift more and more towards that, because you can much more directly achieve those in, you know, and

Dave West: 22:42

there's even so we're obviously doing a number of experiments@scrum.org with the use of llms. And really, what is the future and, and what's what I'm very cognizant of, is that the you can't predict. So, you know, you build a model, you create a series of data packs. You then execute that model. It then behaves, is used that that you need interaction between the user and the model creates and and then you have to refine that model based on experience. What's interesting also, is it that the cost is lower to get things out and to experiment, but the but the likelihood of success is also lower because you you know it's it is very, very worrying as a as a product owner, it just means that I have to spend my whole life, testing ideas, asking why, refining the vision, going back to the vision, it's just exhausting, chaps, I'm not gonna lie, but, but I have a team around me that, uh, that can help me. And I think that's that's a really important message, and

Paul Kuijten: 23:58

you have aI helping you, do

Dave West: 24:01

yes to the moment. It doesn't seem to be able to help me that much. But I just, I was, I was trying to cheat and use AI to pick my English Premier League fantasy team and, and it's, it's, it's not, not done very well, actually, yeah, it's not quite as good as I would like it to be. I'm afraid, which, which is, which is probably good, and probably means it's going to be more fun in the long run anyway. So, anything? Alright, so we're coming at the end of our time, and, you know, we try to keep these a little short. Is there any final words for somebody that's looking at this capability in the capability model, you know, evolve a validation and saying, okay, you know, what's the first thing I should do? What's the thing I should focus on? What you know is there any sort of final words around that?

Paul Kuijten: 24:56

So I would say, experiment your. Experimentation.

Unknown: 25:02

So,

Paul Kuijten: 25:04

yeah, so, so the way that you work, you're gonna need to figure out how if that works or not, too so, so if you're gonna, if you're gonna try and make steps in using data more in your development process. Then, then, then experiment with that as well, and then also take your environment along in the fact that it's an experiment, because that's that's often you know that that takes away a lot of resistance from those around you when you're all of a sudden, say, hey, we don't know if we're sure we're going to be testing. And then if you frame that as an experiment, then it usually takes you a bit further.

Fredrik Wendt: 25:54

And of course, if this field is kind of new to you and you want to learn what's in and out of this area, then, of course, the discovery and validation class with us is a good start to give you the central ideas of where to go, and then you can learn from there.

Dave West: 26:10

Yes, well, thank you. That's a nice plug for a class. And actually that class changed our behavior in scrum.org It actually created the beta program that we now have on the website, which we've never had before. It also created, though we did, beta classes, we weren't explicit, so we our betas were quite expensive. So we looked at, how can we reduce the cost of that, meaning debt, data back. So by putting things on the website, obviously, we started to understand who was interested in this class, and then we had the opportunity to reach out to them and talk to them without them having to go on the class, which gave us the ability to improve and refine our understanding of the class. I obviously we have, we have to get better there. But that was a start, and it was driven by that class, which is, which is kind of cool. Well, gentlemen, as always, I learned to a couple of things today, which was, which is always a good thing, this idea of value and validation. I think you know something that Paul said about it's not just at the business case. Should we even do this, this, this, this set of features, this product goal. It's at the you know, it's at that level, obviously, but it's also at much smaller granular level, whether it's a feature, a capability, a part of the value stream, those ideas often are not as well understood as you would think so, validation in the most economic form possible. We learned about product owner setting the why, and teams really leaning into that why and trying to do the most economic thing to validate that assumption and move that process forward. We sort of highlighted the fact that validation is a team sport, and that you've got as product ownership is supported by empowered teams. Because I think that scales it in a way that that perhaps things like product ops, etc, don't which is which is interesting and and ultimately we, I think we learned today that value is what we all pursue and value, particularly in this modern digital age with the prospect of llms and AI, value is is a very fleeting, complicated beast, and you have to manage those surprises frequently To determine whether something's valuable and whether something isn't. So thank you for listening today, social community podcast. Thank you Paul and Frederick. Thank you spending the time, which I'm always very happy to see your faces. So thank you for that, and thank you for listening everybody. If you liked what you heard, please subscribe, share with friends, and of course, come back and listen to some more. I'm lucky enough to have a variety of guests talking about everything in the area of professional Scrum, product thinking. And of course, a little bit of agile sprinkled in. Thanks, everybody. Scrum,