We found ourselves at Lean Agile Brighton 2023 interviewing world-class Agilists getting their "Talks In Ten."
As we gear up for Lean Agile Brighton 2024, we're excited to share a special episode recorded at last year's conference featuring Julie Starling. In this TalkInTen, Julie dives into the actionable use of data to drive enhanced delivery and improve organisational flow.
Are you tired of having the same conversations, delivering “late” and being put under pressure for decisions you didn’t even make? In this session, I’ll walk through how we managed to flip the script in a highly regulated, risk-averse financial institution by introducing probabilistic forecasting and flow metrics, as well as ditching wasteful estimations.
I’ll cover what you need to get started (very little!) and we’ll explore things I wish we knew when we started our journey, including what is involved in making the mindset shift at an organisational level. I’ll also be sharing some of the success stories we’ve seen, which include teams improving their ways of working based on data, as well as critical decisions being made by stakeholders at the earliest possible opportunity rather than when we’ve missed a deadline.
Probabilistic forecasting has helped move our risk-averse stakeholders away from traditional and comfortable but ultimately higher-risk ways of working… however, we’re still on our journey and there is so much more to come, so I’ll also touch on where we are going, which includes using DORA metrics.
Episode Highlights:
Actionable Data: Learn how to use data to increase flow, generate conversation, and enable experimentation.
Probabilistic Forecasting: Discover how to implement probabilistic forecasting to flip the script in risk-averse environments.
Flow Metrics: Understand the importance of flow metrics in enhancing delivery and improving decision-making.
Tune in now for valuable insights and practical strategies for using data to drive enhanced delivery and organisational flow!
If you enjoy the show, please leave a review!
Use code PRODAGILITY24 for 50% off interactive slides at AhaSlides.
AhaSlide
Host Bio
Ben is a seasoned expert in product agility coaching, unleashing the potential of people and products. With over a decade of experience, his focus now is product-led growth & agility in organisations of all sizes.
We found ourselves at Lean Agile Brighton 2023 interviewing world-class Agilists getting their "Talks In Ten."
As we gear up for Lean Agile Brighton 2024, we're excited to share a special episode recorded at last year's conference featuring Julie Starling. In this TalkInTen, Julie dives into the actionable use of data to drive enhanced delivery and improve organisational flow.
Are you tired of having the same conversations, delivering “late” and being put under pressure for decisions you didn’t even make? In this session, I’ll walk through how we managed to flip the script in a highly regulated, risk-averse financial institution by introducing probabilistic forecasting and flow metrics, as well as ditching wasteful estimations.
I’ll cover what you need to get started (very little!) and we’ll explore things I wish we knew when we started our journey, including what is involved in making the mindset shift at an organisational level. I’ll also be sharing some of the success stories we’ve seen, which include teams improving their ways of working based on data, as well as critical decisions being made by stakeholders at the earliest possible opportunity rather than when we’ve missed a deadline.
Probabilistic forecasting has helped move our risk-averse stakeholders away from traditional and comfortable but ultimately higher-risk ways of working… however, we’re still on our journey and there is so much more to come, so I’ll also touch on where we are going, which includes using DORA metrics.
Episode Highlights:
Actionable Data: Learn how to use data to increase flow, generate conversation, and enable experimentation.
Probabilistic Forecasting: Discover how to implement probabilistic forecasting to flip the script in risk-averse environments.
Flow Metrics: Understand the importance of flow metrics in enhancing delivery and improving decision-making.
Tune in now for valuable insights and practical strategies for using data to drive enhanced delivery and organisational flow!
If you enjoy the show, please leave a review!
Use code PRODAGILITY24 for 50% off interactive slides at AhaSlides.
AhaSlide
Host Bio
Ben is a seasoned expert in product agility coaching, unleashing the potential of people and products. With over a decade of experience, his focus now is product-led growth & agility in organisations of all sizes.
And ominous, there we go. Make sure that's all okay. Yeah, all right. So I will do a quick intro and then we'll hand over to you. So here we are, not at Lean Agile Brighton because when I interviewed this brilliant person, unfortunately we were standing near a thoroughfare, I think would be the technical term, and it was rather noisy. And even though the episode is currently number two, Out of all the episodes we recorded at Lean Azure Brighton, the sound quality just wasn't something that I was happy with. And I don't think it did Julie Starling. What's the word I'm looking for? I don't think it put the right kind of spotlight because I think it was sometimes hard to hear what you were saying. So Julie was kind enough to come back for us to do a little redo on the talking tone, so Julie Starling, welcome back to the podcast. Thank you very much and it's a shame we're not in Pearson but it is nice to not be stood next to toilets this time. It is, it is and I do apologise for that but unfortunately the room that we had assigned to us was busy. However, here we are, we find ourselves on a lovely autumnal morning, it's Halloween as we record this and Julie, I would love to hear once more, and our listeners would love to hear, is your talk intense and what was it you were talking about at Lean Agile Brighton? Yeah, so it was a bit of a case study of a journey we've been on at Principality Building Society. So talking a little bit about how we use probabilistic forecasting and flow metrics and all about using data in a real actionable way to sort of influence the future. So it's not about heavy into numbers. It's not about just like monitoring and updating. It's all about using that information to take action and to have sort of. real managing expectations, being able to deal with the outcomes and influence the future and also avoiding that illusion of certainty that we all fall into the trap of. It sounds really dreamy. I mean, who doesn't want to avoid that illusion of certainty? Because I think it's something that's a trap that many kind of product and agile people fall into. And you used a word there probabilistic, which is a word that perhaps some people aren't familiar with. Could you explain that a little bit? Yeah, yeah, so I guess maybe it's good to start with deterministically. So when we think deterministically, we think in certainty, we think, you know, that there is a outcome, whereas when we think probabilistically, we understand that we're talking about likely what is likely to happen. And as we're learning more information along the way, we're sort of updating that outcome based on the information we have. If you think about weather forecasts, they're a great example of probabilistic forecasting because they tell you what the weather is probably going to be like in seven days and as they get more information they update that and that's essentially the same when we use probabilistic forecasting for software delivery too. We start at a point and as we learn more or as we hit blockers and dependencies we sort of update that view and we're never giving it it's definitely going to be this date it's like it's probably going to be around here or you're probably going to get about this much and we talk about the probability we use as well because that's a really important sort of factor to this so we might say we're 85 percent certain but we need to recognise in those statements that you're also saying there's a 15 percent chance this isn't going to play out the way we think it is and that's that whole nature of avoiding the illusion of certainty because we don't know. No, that is inherently hard to predict. Now you mentioned, yeah, that does take 85% because it was a figure that you used. Is that almost like a confidence rating on how confident you feel that you're gonna be able to hit that date or date range? So kind of, so we use what is called like a Monte Carlo simulation and what it does is runs like thousands and thousands up to like a million simulations. And what it says is out of all of those random simulations that are run over your data, 85% of the time it'll come like out on or before our data or it'll be 85%. It'll be this many or more items. So what we're saying is when we do these simulations, 85% of the time, the answers within this range, 15% of the time it's not. So it's kind of like confidence, but it's based on real data and it's based on a simulation. And it's given sort of the best sort of view as we can of what the outcome could be. And 85% is a number you would need to agree on organizationally as well. You might find that... the organisation you're working with is okay with a little bit more risk. So you could go 70% or you might want to go up to like 90% if it's a little bit more important. So in your talk, was it just on the calculations or was there some kind of cultural or leadershipy type elements in there as well? So I try to keep it as light as possible, because whenever you talk about maths and data, sometimes people's eyes glaze over, and that's not the point of this stuff. The point is, it's about the whole managing expectations piece. It's about using that data to do something with it. So there's a bit around where we talk about the education of it, because like you said, it sounds a bit dreamy, doesn't it? You get to manage expectations, you get to influence the future. but actually you have to have some really uncomfortable conversations to do that and you have to do it quite early and when you're telling people oh yeah you can get all this stuff they'll be like oh that's brilliant but then when your forecast starts moving around that's the bit people get uncomfortable with because they're used to being given certain plans even if we don't deliver them and we're always late or they're always wrong they're used to certain plans so when a date starts moving around people get uncomfortable with that and part of what we talk about is Well, actually, you haven't lost any certainty. You never had it anyway. What we're giving you is more information. These things were happening under the cover of certainty anyway. So it's about really facing into that and doing something about it whilst you're still on. It's a really interesting pattern, isn't it? I mean, it's something that I was faced with quite recently. And I say, well, I say quite recently, it was recently, excuse me, but it's also something that happens over time, particularly, I suppose, if you're training and you're explaining something, and then people say, well, like, that won't work, because what if this happens? And so then the question is always, well, does that happen at the moment? But yes, and is the way you're dealing with it at the moment? Good. No. Okay. So what if you tried something different? It doesn't, it may not necessarily change anything, but it'll give you a different perspective through which to view the thing. No, no, no. Are you going to say that? How much of it is really, are you truly trying to understand how much of it is just, you just want to push back on the change, even though you know that stuff isn't really working right now. Yeah yeah absolutely and like yeah happens so much like especially like in the estimating culture that we've I think we're coming out of but we've had for a very long time where people are like I want you to tell me how long this is gonna take so we know the least about it now as we will know because we're only just starting you're not gonna know how many meetings you're in you're not gonna know how much you're gonna context switch you're not gonna know if you're gonna uncover complexity or get a dependency you're not expecting all that type of stuff but we want to know you know, give us a guess of when it's gonna come out and we won't hold you to it, we promise. But then that becomes a date, you get held to it. These things happen along the way that you couldn't have known at the start. And probabilistic forecasting is about, well, when we get that data, we'll just update our view. It doesn't disrupt your teams because it's like a simulation. So it's done on real data. It's not opinion, it's not gases. People actually can carry on with the value added work rather than... taking them off into a room and planning for a day. Again, sounds dreamy. Yeah. What was the question I was going to ask? So you're moving away from an estimation culture or an estimation style approach. And so there is this idea that the more valuable something is to measure or to calculate actually, the more it will cost to capture the data in the first place. Is that necessarily true in your experience? No, so one of the things I talk about in the talk actually is how little you need to get started. So you only need 10 data points to get started and those data points are literally, so whatever you define your system as, so whatever you're forecasting from the start to the end point, you need 10 start and end points and that's enough to get started with probabilistic forecasting. It really is that simple. There are some caveats like you can't use one team's data to forecast for another team. That's not how it works. I guess that's pretty expected. Your future needs to look like the past. A lot of people then think that means, oh, I need to be doing the same type of work over and over and over again. It doesn't mean that. What it means is you couldn't, so say you shaped your stories in one way or your work items in one way, and then you did like a fundamental change, you shape them in a different way. You probably need to look at what data is right to use. You might need to build up 10 points or maybe you're like. completely changing your tech stack and all the data you've got is on a completely different tech stack. You probably need to build up a little bit of data. So as part of this, it's like looking at the data you're using to feed into what you forecast. Awesome. There's so much I would love to talk to you about and delve deeper on, but unfortunately, this is supposed to be a talk in 10. But it's really, really interesting. I think there's a huge amount there for Agilists and kind of product people alike. And I do wonder, and it's something I haven't got any data on as such, but how, and not a question to answer now, but a little cliffhanger is, how much is this used in product organizations? Because this is something which has been born from, I say born from, at least popularized by the Agile world. But I guess, and I feel that there's a huge degree of potential application for, in more kind of product focused organizations, because this is a very worthwhile and very mathematically sound approach. But Julie, our time is up, I'm really sorry. It's been so nice to see you again. Perhaps we'll get you on for a full episode at some point, because it'd be great to explore this in more depth. Now I assume that for people to find you the best places LinkedIn. Yeah, LinkedIn or I do have a really raw website that I've just started. It's only got about three blog posts on, but yeah, I'm building that up over time if anyone's interested in that. juliestarling.co.uk. JulieStine.co.uk. Everyone, I recommend everyone just goes, checks out those free blog posts. Julie, thank you very much for coming on. Everyone, thank you for listening. Thank you.
