Getting2Alpha
Jeff Patton: Beyond story-mapping: the evolution of agile product design
Intro: [00:00:00] From Silicon Valley, the heart of startup land, it's Getting2Alpha, the show about creating innovative, compelling experiences that people love. And now, here's your host, game designer, entrepreneur, and startup coach, Amy Jo Kim.
Amy: I first knew Jeff Patton as the father of story mapping, a visual way of mapping out and prioritizing your product backlog.
I now know Jeff as a colleague and friend, someone who deeply understands how teams come together to build products and where that process can go off the rails. Here's Jeff talking about how to find the right MVP to test your ideas.
Jeff: When I talk to people who are doing really well at product development, and I ask them for advice on what one thing they should do, the people that have really kind of come through, come out the other end and gotten really good at this stuff, the piece of advice they give me and the piece of advice I agree with is just spend more time with your customers, [00:01:00] spend more face time with them.
Now, if we're going to go through for sharp validated learning cycles, the trick to that is focus. If you've got a product idea, whether it's a whole new product or just a new capability or feature, the thing I want people to do is understand who it's for and to imagine that new product, new feature, new experience.
Imagine just enough that they can find their first big, scary assumption, first big question, and then figure out what the least they could possibly do to validate that scary assumption or answer that big question. It's to not go all in. It's not to have a great big idea and build a great big idea.
The first, most important thing to do is to come up with a solution. With the first thing that scares the crap out of you and get out and spend time with people and validate it, maybe just talking to people to confirm they actually have a problem. You think they do validating, maybe showing the simplest of prototypes, simplest of solutions [00:02:00] to validate that you're in the right ballpark.
Amy: If you're a product manager or you aspire to be one, Jeff's work is fundamental background knowledge. Listen in and learn how to navigate around the common obstacles of corporate life that can keep your product from succeeding.
Jeff: I'm Jeff Patton. Right now, I'm a consultant. I work with product organizations and my focus is on the What I call contemporary product thinking. It's just going from maybe more traditional, uh, build it and they will come style of product management, where I might do some market analysis and guests to something that's more culture of product than thinking that space more around experimentation.
And it's, it's based more around building strong customer empathy and, uh, experimenting a lot more. Now that's what I can do today, but, uh, look, I've been around for a long time. I've been in software development for 25 plus years. I'm known in the agile community as, uh, the agile [00:03:00] UX guy or an agile product guy.
Um, I was early in on this agile stuff because for the decade prior to getting involved with agile development in 2000, I had done a lot of different roles, but I was a product manager when I encountered agile development. So I've spent, uh, since 2000, 2001, trying to take what was good about agile development and put good product thinking back into it.
So, uh, was a working product manager up till around 2005 and a consultant for a big company called Thoughtworks after that. And now I work for myself.
Amy: Awesome. So product discovery is being tossed around a lot these days. And I think a lot of people are really interested in what product discovery is, how that relates to customer discovery and how all of that can be harnessed to build better products that people actually want.
How would, how do you use those terms and how do you understand them and, and then help your clients understand them?
Jeff: Discovery is a weird word. We've [00:04:00] been looking for vanilla words to talk about what we're doing for a while, uh what does discovery mean.
Look, I run into people all the time that think discovery means a lot of this upfront stuff that we do that may involve a lot of traditional user centered designs sort of practice. But look, I've learned a lot of traditional user centered design practice and it isn't quite the same thing as we mean when we say discovery.
Start to incorporate lean thinking things like lean startup and lean UX, uh, kind of thinking when we start to embrace that stuff, it's less about design and more about validation. I don't know if that makes sense, but it's funny. I used to use the word discovery. I and other people that I've worked with, we were talking more about those design cycles that.
Operate ahead of developing software, but years ago ran into a guy named Marty Kagan, who I follow behind. And Marty is a big name in the product management world, has written a great book on product management. And he used the word discovery, and I use it kind of the way he does now. And what Marty really meant by [00:05:00] discovery is this validated learning stuff is really validating that we're building the right thing before we build it.
And that's an awful lot of experiment. experimentation, all the stuff we do all the way up to releasing small versions of software to smaller groups of people and really validating this as the stuff they want and can use. So these days when I say discovery, I mean validated learning.
Amy: As a product developer who wants to do a good job, how can I run a smart, validated experiment?
How can I run a product experiment and get that validated learning?
Jeff: Well, there's two answers that I want to give. When I talk to people that are doing really well at product development, and I ask them for advice on what one thing they should do, the people that have really kind of come through, come out the other end and gotten really good at this stuff, the piece of advice they give me and the piece of advice I agree with is just spend more time with your customers, spend more face time with them.
Now, if we're going to go through for sharp validated learning cycles, the trick to that is [00:06:00] focus. If you've got a product idea, whether it's a whole new product or just a new capability or feature, the thing I want people to do is understand who it's for and to imagine that new product Product new feature, new experience.
Imagine just enough that they can find their first big, scary assumption, first big question, and then figure out what the least they could possibly do to validate that scary assumption or answer that big question. It's to not go all in. It's not to have a great big idea and build a great big idea. The first, most important thing to do is to come up with a solution.
With the first thing that scares the crap out of you and get out and spend time with people and validate it, maybe just talking to people to confirm they actually have a problem. You think they do validating, maybe showing the simplest of prototypes, simplest of solutions to validate that you're in the right ballpark.
It's to start the flywheel spinning, start cranking and validated learning isn't [00:07:00] deciding on what your big MVP should be and then starting to build it. It's not like Silicon Valley, that HBO show. It's a, it's about building lots of small things and doing lots of small things over and over again.
Amy: So tell us a story about a project you worked on either with a client or one of your own projects from your long career in project management where you actually ran some experiments and had a breakthrough with the validated learning.
So we can get a feeling for what one of those experiments might actually look like.
Jeff: Now this is a real story and it's fairly recent and it's one that I've, I've talked about recently and I had to get permission from the company to tell it. Recently, you know, just months ago working with a company called car max big auto retailer. Car max, one of the things that's super important to them is that they make car buying experience. Easy fast fun and one of the things that everybody hates about car buying is the finance guy or getting finance.
One of the characteristics of that experience that they've imagined, uh is [00:08:00] a way for you to apply for financing online and make that super fast and easy and then when you start shopping for cars, either online or on a mobile app or going into the store. You know, every single car that you're qualified to buy. You know what the price tag is on that car, not not just the price tag on the car, but the payments you'd pay the finance terms you get and things like that.
Look, that's their vision. That's what they wanted to do. Now, they went about this in a validated learning way, meaning, look, they talked to a lot of customers and validated, yeah, that, that is a pain point, and they built simple prototypes to show people what the car buying experience would look like if you applied for simple financing and then started shopping, and, well, it essentially changes the shopping experience, so it's a little like shopping at Amazon, where you can click buy it now, and You've already done the financing and it's all done.
No sitting in the finance guy's office for an hour after you've decided to buy a car. And sure [00:09:00] enough, everybody loves that. They also have to work with on the other side with lenders to make sure that they can actually change the way that they do financing with lenders. They've got to work through the whole store experience and everything is looking good, but our spider sense was tingling.
So something was interesting when they were looking at prototypes, when people looked at prototypes of that car buying experience, well, they saw the cars. But people engaged a little less with the prototypes than they thought they should. You know, if you watch somebody look for a car, they get, people are really passionate about the cars they're looking for.
They're about to spend thousands of dollars and they, they care a little bit about that car. And they were noticing that people weren't engaging with the prototypes the same way they would when they were looking for their own car. Now, for them, it was a fidelity change in the prototype to actually stitch together a jankity version of our prototype that used their real car data, the cars that were online, and real finance data.
So, the minute you filled out this simple application, you immediately started browsing cars and seeing, not just a prototype that showed [00:10:00] example cars or possible cars. You really saw the car that you were looking for. They built this prototype and there's a longer backstory on how it's part technology and part hand waving and things like that.
It's a, it's a story about, uh, developers wiring live website data to back end spreadsheets that contain finance data. And it's definitely something that wouldn't scale. But the weird thing that happened with that simple prototype and what was super cool was being there when they were testing it and being there when they put that in front of customers is everything changed the minute they put it in front of customers.
And the I like telling this story because it's a bad story that the minute they put it in front of customers, everything kind of changed. Customers started getting wrapped up in the finance data. They started noticing that the rates they were getting on cars did not make match the super low rates. They were seeing advertised at banks, which is the case when you're buying a used car or when you're buying a late model car.
Other weird things would happen where a car that was more expensive would have a lower [00:11:00] payment than a car that was less expensive, and that caused them to kind of get confused about why that was. And at the end of the day, adding finance data on top of the car buying experience kind of messed things up.
It didn't speed things up for people, didn't make things funner, it didn't make things easier. At the end of the day, it didn't make CarMax look particularly good, and the, where they ended up with on this was that adding this into the experience just took a tough decision and made it even tougher. And that was a surprising thing for them.
Now, in the course of doing that, they did find out that there are ways to incorporate this stuff. So when their salespeople are assisting people, they can take the finance guy out of the picture and put the financing into the selling so the salesperson can help with that. Uh, but connecting the financing experience to the buying experience was messy.
Now, what they learned from doing this kind of quick cycle, validated learning was that, uh, this is a bigger problem than they thought it was. Yeah. And they're not going to give up on solving it, but as of this year, [00:12:00] they, they found some easy, better solutions, and a lot of companies will keep a backlog.
They talk about moving this thing into the way backlog, uh, meaning, okay, we want to solve this problem, but it's a tougher one than we thought going into it.
Amy: That's a great story and it really illustrates how important it is to get things in front of customers and treat it as an experiment you're eager to see the outcome of.
Jeff: You know, for them, the cost of actually wiring up a live website for a national change that sells billions of dollars a year and has, it sells over in excess of a million cars a year, the cost of, for them to build out this functionality and find it out the hard way, that if it wasn't helping, would have been.
I don't know how many millions of dollars, but you know, I tell this story because it was recent because it was real and because look, most of the time we hear fabulous success stories and what people don't hear is that to get to that success takes lots and lots of iteration, lots of experimentation.
And [00:13:00] when you tell that last best success story, that's pretty cool, but with a lot of companies I work with. I see lots of failures or what's interesting, lots of, Oh my gosh, I'm glad we did that experiment and learned this wasn't so trivial and wasn't so easy because we would have spent a ton of money.
So I see that a lot more. I can probably think of some more successes, but what I think people start to realize is look, there's no shortage of ideas. It's just really hard to find that correct thing. That the thing that hits it out of the park.
Amy: Exactly. So what do you think stops companies from doing these kind of experiments?
What are the roadblocks that you see that you help them get around, but that I'm sure get frustrating to you to see them over and over again?
Jeff: Yeah, I've been thinking a lot lately about what those roadblocks are. the roadblocks with humans, then human nature in general, the, the, Biggest human nature roadblock is that when we come up with ideas, we tend to think our ideas are awesome.
We [00:14:00] have this future perfect imagination of our ideas. If you go into a startup accelerator and I've been inside of startup accelerators to help coach and I've been inside them when I teach public classes, I like teaching inside of these big co working spaces where there are lots of startups. And I joke that those should be horribly depressing places to work.
You should go in there and people should just be, uh, their faces should be hanging. They should all feel terrible because they're all working for businesses that are likely to be out of business next year. Everybody knows anecdotally that. The failure rate of startups is 90 percent or so for tech startups.
But those people are not sad. They are optimistic and it's not because they're stupid. It's not because they think that the failure rate of startups is super low. They know it's super high. But what's interesting is they're all confident that the failure rate of startups is high, and they're all confident that all these startups are going to fail, but they're pretty confident it's not going to be their startup that fails that attitude carries through in larger organizations, [00:15:00] especially organizations that have.
Had a success in the past they tend to believe that look we have a successful product. So therefore Everything we do from here on out is going to be successful They don't realize the failure rates for new features new capabilities new things that we put in our existing products Those failure rates end up being well, maybe not 90 percent, but the failure rates are certainly over 50 most of the stuff we we launch or build ends up in you know Jokingly call it the thud zone.
The it's not awesome. It's not awful. It's just kind of somewhere in the middle And if we knew it was going to land there, we might have not built it in the first place In large organizations, there is a human bias We we believe that all of our stuff is going to work and in very large organizations.
There's a lot of Crofton protection built around this organizational self deception thing. Oh, look, there's a joke that a friend of mine, David Hussman, uh, tells that we talk [00:16:00] about failing fast. And David says the difference between failure and failure and learning is the amount of money you spend to do it.
If you spend hundreds of thousands of dollars launching something, you can't call that learning with a straight face. You can't call that failure with a straight face. We have to pretend it was successful. We can't say we just spent hundreds of thousands of dollars, but at least we learned something. We can't call it failure, uh, either because that's, uh, that's political suicide in a larger company.
So we have to do a lot of hand waving to say, uh, well, it wasn't as successful as we thought it would be. It was still successful. So in a lot of organizations again, we believe our ideas are awesome. And then there are a lot of organizational self deception mechanisms we use to prove that everything we launch what's pretty much successful.
Amy: Wow. So do you ever see organizations that you really admire that are very good at running experiments?
Jeff: Yes. I see lots of them. And what's interesting is that, you know, the concept of teaching and [00:17:00] talking, I name all sorts of organizations that I've worked with that do cool things. And you might think that the organizations are perfect, but, but, but, but they're not, uh, every organization struggles with this.
I, I'm talking to you this week, you and I tried to talk last week to do this, but I was in Sydney, Australia. And our bandwidth sucked and we couldn't record this, but I was in Sydney working with Atlassian. Atlassian is an organization I love. Uh, they are super good at getting out and getting FaceTime with customers.
They are super good at experimentation, but Hey, they're not perfect. If you're a user of Atlassian products and some of the people listening to this, my guests, uh, are, uh, Atlassian has grown a huge amount. They've got huge market share, but they've got big, diverse customers and they struggle with the same kinds of things that Existing product companies struggle with when you've got big diverse groups of customers It's hard to satisfy everybody and it's hard to be Super tough about your strategy about who your product is for and who it's not for and but the way they work Look, they have it down.
[00:18:00] They work hard to make sure that evry product manager, every UX person, and especially product development, the developers, the people who write code, get FaceTime with customers and users. Uh, they work hard to be diligent about the way that they do experiments, uh, to really validate what they're doing before they do it.
Amy: That's awesome. So I look forward to hearing more stories about how to do the stuff. Well, from, you know, the front lines, cause I think those are super inspiring and very actionable for people. So one more question, Jeff. You're a very inspiring guy, certainly to me.
Jeff: You should meet my kids. They don't think I'm that inspiring.
Amy: Yeah, either do my kids, so that's their job, right? Um, yeah. So where are you getting your inspiration these days? What are you seeing, playing, reading, watching that's bringing you inspiration and new ideas?
Jeff: Oh, am I seeing, playing, watching? You know, I mentioned earlier in this conversation. Well, it's people.
It's the people that I work with and [00:19:00] talk with. I mentioned earlier in this conversation, Marty Kagan. I mentioned my friends at CarMax, my friends at Atlassian. With a lot of product companies, there are individuals that I work that are just really connected with what they're doing. As far as a community goes, or a community that I pay close attention to, is this emerging community.
Sort of contemporary product community. Um, I uh go to every year and happily i'm able to teach some workshops there But I go to every year a conference called mind the product Uh, but look i've paid to go to that conference before and i'll keep paying whether they Invite me to run a workshop or not. I love That community and product tank meetups, uh, that that's around it.
So I've participated in product tank meetups in a lot of different cities. I've participated in mind the product conferences and then look out of that community and the thought leaders that speak at those conferences And at product tank meetups, I keep running into more and more people that are thinking and working this way.
Amy: I think that's a [00:20:00] great suggestion and we'll make sure to post a link to that and folks can really check out all the...
Jeff: I should have said I should have said your stuff.
You and I have a mutual friend that introduced us, and more and more, I'm getting up to speed on your stuff, and more and more, paying attention to how game thinking applies in everyday products, uh, is also pretty freaking relevant. And, uh, actually, you're one of, when I look at talks and, uh, things that I found online, uh, that are inspirational, your stuff too.
Amy: Well, thank you. I love hearing that. Well, feelings very mutual. So such a great friendships made. All right. Thank you so much, Jeff, for joining us today and sharing your stories and your insights and your hard won wisdoms. I really appreciate it. I love talking to you. I'll look forward to seeing you at the mind, the product conference coming up.
Talk soon.
Outro: Thanks for listening to Getting2Alpha with Amy Jo Kim, the shows that help you innovate faster and smarter. [00:21:00] Be sure to check out our website, getting2alpha.com. That's getting2alpha.com for more great resources and podcast episodes.