The Product Experience

How to improve your relationship with data - Heather Williams (Director of Product, Elsevier)

• Mind the Product

Share episode

Discover the secrets of successful product management with Heather Williams, Director of Product at Elsevier, as she shares her transformative journey from UX design to leading product teams. Recorded live at the Mind the Product Roadshow in Amsterdam, Heather unpacks the power of user research and interaction design in shaping innovative product strategies. Learn how Elsevier's product teams integrate data-driven decision-making through cross-functional collaboration, including roles for data analysts, UX designers, and subject matter experts, all working towards enhancing product experiences.

Featured Links: Follow Heather on LinkedIn | Elsevier | 'Five things we learned at the Pendomonium + #mtpcon roadshow - Amsterdam 2024' feature by Louron Pratt

Our Hosts
Lily Smith enjoys working as a consultant product manager with early-stage and growing startups and as a mentor to other product managers. She’s currently Chief Product Officer at BBC Maestro, and has spent 13 years in the tech industry working with startups in the SaaS and mobile space. She’s worked on a diverse range of products – leading the product teams through discovery, prototyping, testing and delivery. Lily also founded ProductTank Bristol and runs ProductCamp in Bristol and Bath.

Randy Silver is a Leadership & Product Coach and Consultant. He gets teams unstuck, helping you to supercharge your results. Randy's held interim CPO and Leadership roles at scale-ups and SMEs, advised start-ups, and been Head of Product at HSBC and Sainsbury’s. He participated in Silicon Valley Product Group’s Coaching the Coaches forum, and speaks frequently at conferences and events. You can join one of communities he runs for CPOs (CPO Circles), Product Managers (Product In the {A}ether) and Product Coaches. He’s the author of What Do We Do Now? A Product Manager’s Guide to Strategy in the Time of COVID-19. A recovering music journalist and editor, Randy also launched Amazon’s music stores in the US & UK.

Lily Smith:

Welcome to the Product Experience Podcast. Earlier this year I chatted with Heather Williams at the Mind the Product Roadshow in Amsterdam. Heather is the Director of Product at Elsevier and we had a really good chat about how product teams can work better with data teams and, just in general, how product teams can really embrace data in their day-to-day work. Embrace data in their day-to-day work. The Product Experience Podcast is brought to you by Mind, the Product part of the Pendo family. Every week we talk to inspiring product people from around the globe.

Randy Silver:

Visit mindtheproductcom to catch up on past episodes and discover free resources to help you with your product practice. Learn about Mind, the Product's conferences and their great training opportunities.

Lily Smith:

Create a free account to get product inspiration delivered weekly to your inbox. Mind, the Product supports over 200 product type meetups from New York to Barcelona. There's probably one near you, heather. Welcome to the Product Experience Podcast in this amazing setting. It is quite unique.

Heather Williams:

It is quite unique.

Lily Smith:

Oh, actually I said I wouldn't refer to anything visual because lots of people will be listening to us, but for anyone who is listening, we are sat in a room full of clocks. So before we get into our topic for today, give me a quick intro to who you are and how you got into product and what you're up to these days.

Heather Williams:

So I am Heather Nice to meet you, lily. I got into product only a few years ago maybe eight or ten, I kind of lose count. I came on a meandering path through user experience, so I started as a design intern and decided this was really good. I'm going to stick with this and pursue it further, and I think I spent about 15 years doing user research. It's probably my strong area user research and interaction design. And then I can remember this opportunity came up when a product manager was going out on maternity leave for a while and I thought there was a chance for me to try that out. And I did for eight months or so and I said, oh, this is actually much harder than I thought it was going to be. So I went back for a while and skilled up and then I returned. So I've been doing digital product for perhaps eight or ten years and started managing teams about four years ago.

Lily Smith:

Cool, and tell me a little bit about what you're doing today.

Heather Williams:

Well, today I'm here at the Roadshow the Mind Product Roadshow in Amsterdam After we speak. I look forward to seeing the Roadshow kick off. Later this afternoon we'll be on a panel about the flux of product.

Lily Smith:

Nice, cool, and you're Director of Product at Elsevier. So tell me a little bit about your managing teams at Elsevier, because one of the things we were going to talk about today was how data and kind of insights fit into products yes, fits into products and how product teams and product people can work with this as a general kind of topic area, but also with people who specialize in data and in insights.

Heather Williams:

Yeah, At Elsevier, we've been quite focused on being data-driven in our decision-making for a very long time, even before I moved into product role, when I was a UXer supporting other product teams.

Heather Williams:

Collecting data and using that data to translate that into design has been something that I've been familiar with since the beginning of my career. Now we've matured in that quite a lot over the last years. Where I work now I manage like four teams and we're probably there's probably half of the group that I work in. So we collaborate quite closely with many other groups in this part of the company and we are collecting data from many different sources, like our expert members of our team, if you would so like. Within our squads it's not just the engineering team, but it's also the data analytics and the user experience members, I would say, are bringing in a lot of that qualitative data as well as data. Scientists that we work with that also have a unique perspective on how to collect different types of data that they can use for creating different parts of the algorithm, evolving an algorithm or the experience. So we're using all of that data that we're collecting from different members and different sources to put that into what we're doing and bring the product experience to life.

Lily Smith:

How do you kind of arrange your product team? So I'm assuming you have a kind of a core product team of a product manager, engineering and design how do you bring data people into those cross-functional teams? Are they kind of assigned to those teams on a team basis or is it a project?

Heather Williams:

basis. It can vary. We try to as much as possible embed, but we don't often have like there's not usually a data analyst or even sometimes a UX designer per squad. So sometimes they will cross train between a couple of squads and we will try to do this in a way where it's kind of compatible, so that their mindset and their mental model like with a persona is shared between perhaps two different sort of value streams and what they're doing for one value stream they can oftentimes apply it to the nuance with another squad. But you're pretty much right on the spot, though, about what our squads look like. It's engineering At the core.

Heather Williams:

Generally there's always a UX person and or a data analyst who are assigned to that squad, as well as the product manager With data science. That's not always the case. That's probably more of a shared service that comes in depending on what the project is and the prioritization of that, then quite unique. Well, perhaps not unique, but in our division of the company we work quite closely with another sister business unit two, in fact, operations and one of our publishing groups, and so from those groups we'll often have what we'll call an SME, a subject matter expert that will come in and be a joint member of the squad during the relevant phase of that work. Their input will vary depending on what stage of the product lifecycle that we're in.

Heather Williams:

And they'll contribute to the outcomes.

Lily Smith:

And one of the things that I found, even in the small businesses that I've worked in, is that quite often similar or the same data questions come up again and again. So do you have a strategy for making sure that you're not repeating yourself and that you're pulling out the insights that you found historically and kind of making use of those?

Heather Williams:

That's a really good question, and strategy is a big word here.

Heather Williams:

I wouldn't say we have anything so formal as a strategy and maybe that's something to go back and for us to think about a little bit more what we tend to do.

Heather Williams:

I'm curious if this is what happens with other people but what we tend to do with our data is we do try to document a good bit so that if a member is promoted out of team or if they were to go somewhere else, there's some memory that's left behind of where is the data, how to retrieve more data and what decisions were made previously with this data. When we are collecting data, let's say we've just done an experiment of some sort, like an A-B test or even perhaps a usability test or some user understanding that generally does get documented in some artifact that is retained. But we also try to showcase that, like in a typical sort of showcase but also a team discussion where we can interrogate like what does this mean and how did we? You know what was the question around this, so we can understand the context and figure out how we can apply it in other contexts or apply it to you know the assumptions that we're trying to test or some other aspect of the development.

Lily Smith:

Nice, and what kind of format does that showcase take? Is that across the whole business, or I guess you're probably doing quite a lot of them, so it would be quite a lot of work to play it back to everybody.

Heather Williams:

It depends on, like, how big a project it might have been. If it's like quite a big foundational project, the stakeholder audience can be quite big and we will schedule like a grand showcase and invite folks into that initial discussion, but with calendar availability. Practical way that this turns out is like they don't all show up but what they will usually do. What we do now, which is quite common for us, is to record those sessions and they do get picked up, they do get watched, and then there's the, the slack conversations in the q a that will develop so it it does find a way of permeating and making the rounds, and slack has become a quite a it's like and teams, uh, quite a common way for us to distribute and broadcast the information as well.

Lily Smith:

So how, when I think about like product managers, I know that they're all sort of like super keen to make sure they're working really well with everybody because they want to get the most out of their team. What advice would you give to a product manager who's working with a data analyst or a UX researcher and maybe the advice is different for the different kind of function or role to really get the most out of that person when they're working with them?

Heather Williams:

Yeah, this is something I've been discussing a lot within our groups.

Heather Williams:

In fact, for particular reasons in our team right now, we are going kind of like resetting on a racy.

Heather Williams:

You know who's responsible and accountable or consulted or informed for different parts of the project, and partly because we've been looking at the SMEs who are allocated to the teams and what the expectations are, so they can kind of know what to do in those sister parts of the organization and who best to bring in.

Heather Williams:

And what I would say is, when someone new is coming in or you know, if we're looking for an expert or someone to come help us out with something, have a conversation much like this right here to learn a little bit more about, like, what it is that they do, what are they expected to bring in, how they can work together, who's going to do what kind of just the general etiquette of like, what kind of meetings are we going to have, and kind of play it out from there. I think we generally let our teams, you know, be quite empowered on what are their ways of working. I think the most influence that we generally want to have is that they've got the right resources so that they can, you know, go and do the work that they want to do, and you know that they're having that exploration on what works for us as a team, because they won't also, you know, oftentimes not be in the same time zone. They could be in different locations, usually working very hybrid, I think, like a lot of groups these days.

Lily Smith:

So I find that when I'm doing so, I do a lot of data analysis and use the research myself, just because I like to kind of keep in touch with you, know the detail and I feel a need to be close to it and understand it as well. I think you understand and appreciate the work a lot more if you've had some experience of it, and appreciate the work a lot more if you've had some experience of it. So how much do you get your product managers to be close to the detail? Obviously, you just said that you like to do it yourself.

Heather Williams:

I can hear them in my head right now, like answering this question on my behalf. I do a lot of coaching on this. I don't expect them to be the data analysts we're quite fortunate to work with data analysts and user researchers to help us bring in that picture but I do feel like it's important for every product manager to know where the data is, how to interpret that data, Particularly if we're pivoting into a new problem space, kind of validating that that problem is real and worthy of our time. One of the things earlier this year that I'm reflecting on, this discussion in my head with my team about mapping out the user flow, putting the data on top of the user flow how well are all of these bins converting? Where are the opportunities laying out there? How big are those opportunities, so that we can understand how we might want to prioritize one over another?

Heather Williams:

I think they're generally very much on board with this and in my head I'm seeing them. Yes, we know we will do it and we're coming up now preparing our Q3 OKRs, so I do see the team diving into all of the data, into those user journeys, pointing back to the graphs and the user flows. Like this is the problem that we're going to tackle right now. So it's cool. The data into those user journeys, pointing back to the graphs and the user flows, like this, is the problem that we're going to tackle right now, so it's cool.

Randy Silver:

This episode is brought to you by Pendo, the only all-in-one product experience platform.

Lily Smith:

Do you find yourself bouncing around multiple tools to uncover what's happening inside your product?

Randy Silver:

In one simple platform. Pendo makes it easy to both answer critical questions about how users engage with your product and take action.

Lily Smith:

First, Pendo is built around product analytics, enabling you to deeply understand user behavior so you can make strategic optimizations.

Randy Silver:

Next, Pendo lets you deploy in-app guides that lead users through the actions that matter most.

Lily Smith:

Then Pendo integrates user feedback so you can capture and analyze how people feel and what people want.

Randy Silver:

And a new thing in Pendo session replays a very cool way to experience your users' actual experiences.

Lily Smith:

There's a good reason over 10,000 companies use it today.

Randy Silver:

Visit pendoio slash podcast to create your free Pendo account today and try it yourself.

Lily Smith:

Want to take your product-led know-how a step further? Check out Pendo and Mind, the Product's lineup of free certification courses led by product experts and designed to help you grow and advance in your career.

Randy Silver:

Learn more today at pendoio slash podcast.

Lily Smith:

And you said you do a lot of coaching on this. Is there? Obviously there's the kind of the right mental model for thinking about things and asking questions in the right way, but are there certain hard skills that you know you need a product manager to learn, even things like how to use different queries in Excel or something like that?

Heather Williams:

I probably would not do that. That would probably be beyond my skill level but there are the analysts that we do work with could certainly do that and fortunately, they do do that. Probably the one thing that I'm doing with my team and this is probably a reflection of my UX background is mapping out that journey, all of the minutiae of the events where things the happy path and the various unhappy paths and working through that and having the conversations about what's going on here and is this important for us right now or not, and usually it's letting them have that conversation back with me rather than me asking the questions. It's like, okay, tell me what we're looking at here and what's going on and where are we trying to get to in the end.

Lily Smith:

And is there anything else that really helps get a product manager into the right mindset of really digging into the data?

Heather Williams:

Yeah, I think it's also following your performance and reviewing that, trying not to be too neurotic about it, but we do have conversations, weekly at least, where we're talking about the performance, if it's going in the right track, if we want to try something else. So we do tend to have, through those regular data touchpoints, that sort of mindset to know if we are going the way that we want to go or if we need to pivot. We've done a lot of pivoting this year, which is fine. That means we're learning, we're paying attention to things and making the right calls.

Lily Smith:

Interesting. So what causes a pivot? Is that new information? I'm guessing.

Heather Williams:

Precisely, and a pivot's probably you know too big of a word. It's probably just an adjustment most of the time, new information, like we tried this and that did not work, or we've realized, you know, some event in the environment has changed and we need to go faster, so we need more traffic. It's really being nimble and being on our toes, which is both a lot of fun and sometimes stressful, depending on where you are. But yeah, I think my team does a really fantastic job at following the signals that are around them and adjusting to that. So if you watch this high five- I love it.

Lily Smith:

I think it is one of the things that's really important in a not just in a product manager but in a whole product team that understanding of when there's new data and new information, you have to respond to that as much as it's comfortable to stick with the plan that you created and all the love that you put into the plan and everything, but you have to let go and go where this new information is taking you.

Heather Williams:

Yeah, and that's stakeholder conversation, A lot of team conversation like, okay, what do we think this is? Who are we going to talk to about it? When are we going to go?

Lily Smith:

Yeah, so you mentioned as well obviously you've got a UX background and that you love mapping out different scenarios, so let's dig into that a little bit. What's the kind of first rule of thumb or, if you like, of creating a map Like when and how do you use that as a way to tackle a problem?

Heather Williams:

This is probably very unique to Heather and not going to work for many other people, or they may have better methods. I don't get perfect. In the beginning I really start with a data point and work out from there. Usually I'm starting with where are we trying to get to? And working backwards yeah, now that I'm thinking about this more, where am I trying to get to? What's the usual starting point? And then you find the usual starting point is not always the same. There's usually multiple ways through.

Heather Williams:

So, I kind of work through that one branch at a time and once I've got a general flow then I'll go back and look at the data. I'll see what data I have, if I have data at all, sometimes I don't. Recently this led to an interesting project with our team where we were looking at our user funnel and the branches of the user funnel and where we had data and not and we started from that there was also a usability test that had come later to like let's look into this and see what's going on in here, and we were able to see that this was a much more complicated part of the funnel than we had originally known before it started going into the data that said on top of that, and finding we didn't have tooling for it, like there were some gaps in our view of data. So we're looking at how to go and pull that in and put it into place. We're still working on that right now. In fact, it's been a really good process and a learning experience for us.

Lily Smith:

And do you tend to when you're mapping something out end to end. So I find usually people talk about doing this in like a workshop setting, but I find it much easier to do that sort of job on my own initially and then bring other people in to go. This is what I've got to like. Do you agree? Disagree? Where is this wrong or right?

Heather Williams:

I think both there's a place for both. Actually, it probably depends on what you're going to do with that funnel later. If it is the case where I'm using it for my own needs, like if I'm going to like manage something with this, then certainly I'll do it on my own and usually talk with my team about it, as they will well recognize. But I think if it's something where and I'm having this with one of my groups as well where we've kind of done a pivot around one of these things, I'm like, okay, I'm going hands off now. It's really for me, it's quite important that they pick this up and make this their own and they learn where the the ins and outs of that journey are, and I'll, you know, we'll socialize it in that sort of way so what's your best advice that you ever received?

Lily Smith:

oh, when you started working as a product person. Well, I say as a product person, I mean, a UX researcher is a product person, but what's the thing that someone kind of advised you and then it kind of changed your perspective on everything in a positive way? Really great question.

Heather Williams:

Oh, I've actually had so many good mentors, like one of them, jonathan, I'm thinking of right now who said before a big presentation, just fake that good mood until you make it. I was good doing that one right now. You know, coming in and talking about data, one of my earlier managers, gabby, had said to me we'd had a conversation around data when we were I think it was at a time where the company was really getting into the product model and learning on where's the data coming from and what are we doing that with our decision making. And that was probably one of the first big influences I had had with data analytics, like complementing the qualitative behavioral data that I had had up to that point. So that just the context of the conversation, that that's resonating right now as well.

Heather Williams:

Piece of advice I wish we, I wish I have to come back to you. At the end of the day I go oh, lily, I have it now because there's there's been, there's been so much. But I would say, if I was going to pass it on to others, know where you're good enough is. There is no perfection. Everything we do in product is constantly changing and sometimes you need to know when good enough is and move forward from there, and I'm sure I've heard that echoed in my ear or whispered numerous times in various forms, and that's a good truth to live with.

Lily Smith:

Nice. I love it, and it would be remiss of us not to mention the evolving world of AI and how AI is impacting product functions in general, particularly, I guess, on the data side of things. What do you see changing with how product managers and product teams work with data and how AI and new tools are going to support us?

Heather Williams:

Yeah, that's a great question. It's a question that we're also discussing where we work as well. So I think it depends again on who you are and what role you are looking at. Certainly, it's helping us go through vast amounts of data much faster. So I'm told I'm looking forward to that benefit a little bit more.

Heather Williams:

You know there's so many different facets to what we're looking at, particularly with Gen AI and different forms of LLM. We're trying to apply it to the right problem and in the right way, instead of having you know what are those metaphors, like you know, hitting a big, using a big mallet for a nail or something like this. But we're using it judiciously, so we're choosing where to use it, what the problem is and if this is the right technology to solve that problem with. So, as far as our experimentation goes, that's one perspective.

Heather Williams:

I know that we've got some developers who are experimenting with fast tracking some code so that they can see what kind of prototypes they can create, and they're doing code reviews probably similar things happening in design. I wouldn't be too surprised. I work quite closely to the research community and they're producing large sets of data usually and what we hear is that it's quite helpful and doing a lot of data analysis rather quickly, where we work at Elsevier, we're quite interested in the integrity of the content that we publish on the behalf of the research community and in our own work we're quite sensitive to the hallucinations that might come in to different models. So when I say we're quite judicious and thoughtful about where we apply the technology, this is one of the things that we have in our mind is is this the right tool for the problem right now?

Lily Smith:

It's interesting. You say that I wrote a LinkedIn post the other day and I know so many people who use. I mean. I think it even says on LinkedIn now like Generate it with AI. Yeah generate with AI. So at the end of the post I put in brackets not created with AI, and I always feel like that's going to be the thing in the future of like. By the way, I did not use AI.

Lily Smith:

This is all powered by my brain. There'll be a place for that, maybe not by my brain, but other people's brains. Well, heather, it's been so nice talking to you. Thank you so much for taking time out to come and chat to me today.

Heather Williams:

Thanks for letting me talk with you today.

Randy Silver:

It was a pleasure being here the product experience hosts are me, lily smith, host by night and chief product officer by day and me randy silver also host by night, and I spend my days working with product and leadership teams, helping their teams to do amazing work.

Lily Smith:

Luran Pratt is our producer and Luke Smith is our editor.

Randy Silver:

And our theme music is from product community legend Arnie Kittler's band Pow. Thanks to them for letting us use their track. Thank you.