Tell the User's Story via Effective Research Reports (feat. Susan Mercer)

Susan Mercer gives tips on producing effective research reports that address business goals

Sponsored by Watch City Research
Watch City Research is your trusted UX research partner

Disclaimer: This post contains affiliate links. If you make a purchase, I may receive a commission at no extra cost to you.

Transcript

Dan Berlin: 0:14

Hi everyone and welcome to another edition of the 97 UX Things podcast. Dan Berlin here, your host and book editor. This week, I'm joined by Susan Mercer who wrote the chapter Tell the User's Story via Effective Research Reports. Welcome, Susan.

Susan Mercer: 0:28

Thanks.

Dan Berlin: 0:30

Thanks for joining the podcast. Can you please tell us about yourself?

Susan Mercer: 0:34

Sure, I've been doing user experience work for over 25 years. But I focused on UX research for about the last eleven and now I'm Director of User Research at a travel company, and love working with my team to figure out what we can learn about travelers and how we can influence our product roadmap.

Dan Berlin: 0:57

Cool. You said that you focus on research, is there one area of research that you tend to focus on or an area that you like in particular about research?

Susan Mercer: 1:08

What I love about research is just being curious and asking questions and learning different people's perspectives. So, you never know what you're going to learn. And I that's what I love is that unpredictability of user research.

Dan Berlin: 1:25

Great. And can you tell us how you got there? So can you tell us your UX journey, how you discovered UX, and how you ended up where you are today?

Susan Mercer: 1:34

Yeah, so early in my career, I worked as a software developer and at the end of my first project, they asked me to help QA the project. So I only worked on a small part of it. So I was QAing everything, and I filed a lot of bugs that came back 'works as designed.' And I said, 'But why?' Because this doesn't make any sense. Well, that's how the designers designed it. So we built it according to spec, and it's worked as designed. So it's not a bug, go find others.

Dan Berlin: 2:04

Right.

Susan Mercer: 2:05

And that bothered me. Because why should we be building software that's confusing to people. This was still early days for the UX field. So I just kept working my way up the user research and development process, and finally discovered user research as its own field late in my career. But I feel this is where I've always meant to be because I see myself as somebody who can understand what the user needs and translate it to designers and developers. And designers and developers can build anything, but I consider it my job to help us build the right thing.

Dan Berlin: 2:48

Right, right. And what was that UX moment? What was that moment that opened your eyes?

Susan Mercer: 2:54

I think it was when I was in graduate school. All of a sudden, I realizeded that research is a specialty, because I just thought it was all built in as part of what you do as part of design. But as the field matured, it became a specialization. And that was always my favorite part anyway, because like I said, I get to talk to people and hear their perspectives. And at the end of the day, I'm one of those people that just wants to help other people. So this is my way of helping others is building tools that are easy for them to use to accomplish what they need to do, rather than computers being the trouble that everybody seems to think they are.

Dan Berlin: 3:36

Right. Right. Great. Well, thanks for all that. And well put in terms of what UX does for the world there. But let's turn our attention to your chapter, Tell the User's Story via Effective Research Reports. Can you please tell us about that?

Susan Mercer: 3:52

Yeah, so one of the things I've learned in my career is that having user research reports being consistent and credible is very helpful in my career as a UX researcher. So one of the reasons is that, at least in the businesses I've worked in today, the researcher doesn't have the authority to make product and design decisions necessarily. We're often influencing the teams that are making those decisions. And I see it as the researcher's job to really understand what would benefit the user, what the user needs, not what they say they want, but what they need. And then describe that and document that and use that report as a way to influence the decision makers to make the product more user-centric and to meet the needs of what users are trying to accomplish. So I think really the reason we're creating these research reports is to describe what we learned and what we recommend doing next. And also in the context of usability testing, document what was tested so that when we come back to it six months later, we know the context in which we were making those observations and decisions. And that was a lesson I've learned the hard way, being in house because a few years ago was given this particular team and was asked to shift gears and work on this product. And we had two years of research from another team that was no longer with the organization. And going back to their reports, and realizing I didn't have enough context to be able to know if their findings were still valid or not. So that's when I really built that into mine and my team's methods to make sure we document exactly what was tested.

Dan Berlin: 5:48

Yeah. Can you tell us a little bit more about that? What are those components of the context that's needed for the future generations of the organization?

Susan Mercer: 5:57

Yeah, so particularly when it comes to usability test reports, I often take screenshots, this is the screen that was tested. This is exactly what the user saw what the stimulus was for the user. Depending on the technology, your testing videos would also do that, just to capture the moment. And then in the report, I often like to put in a call out highlighting this particular area here is where they clicked. And this was where they didn't understand what they were going to get. So that we have pinpointed not only the screen, but the particular part of the screen that was confusing to them, or they misinterpreted, or whatever the situation was, so that six months later, I may be looking at the same screen, oh, we've already changed the design that I know that that particular insight is no longer valid, or maybe it is still there. So hey, we gotta fix this. We didn't fix it six months ago, why not? It's still causing a problem.

Dan Berlin: 7:01

And how about documenting that long term, in terms of documenting the knowledge that you've gained through the years of doing the research at an organization?

Susan Mercer: 7:12

Yeah, that gets into an interesting situation. I like to make sure that we keep all the reports in a central location. I don't think there's a good tool out there right now. There's a whole stream of work around research repositories, which is all about where do you put your research information? Where do you put your insights? How do people find them? I think the industry is moving along, but we haven't found a solution that I know fits for my team just yet. So we tend to go a little old fashioned with a Google Drive and timestamps and names of reports. But now we're looking at different ways that we can also tag them and search for insights. That might be in a report about how people look at a product that also might have insights into how people analyze reviews, things like that, it gets kind of convoluted. But we're definitely moving forward as a discipline on that. But I think just documenting in a consistent way, providing some sort of way for people to search and tag things and go back and find things that are relevant is really the key.

Dan Berlin: 8:26

Yeah, makes sense. So you mentioned the context and the visualizations involved for usability studies. But what else about the report itself?

Susan Mercer: 8:36

Yeah, so one of the things that I find about the report itself, you want to make sure to tell the story. And that's obviously in the title of the chapter. And the hallmark of a good research report, I think, is a way to be able to tell your insights in a way that the business stakeholders can relate to. Because at the end of the day, we're telling a story to the people who are influencing the product direction. And I see that there's four parts to that. And I like to structure of my reports to follow that story flow. The first one being, we had this idea, we had this hypothesis, or idea or set of questions. This is the context in which we're creating this report. The second is, here's how we chose to explore it. So here's the method we used, here's who we talked to, this is how we went about exploring it. And then the meat of the matter should be here's what we learned. And that's where whether you did a usability test or interviews or diary study, whatever method you use, capture some of the key things that you learned and show as much as you can, versus tell. So things like screenshots, things like videos, or anything that can show... even quotes illustrate the raw input from the perspective of the user and that is what resonates very well with stakeholders. So obviously put it in that context of here's what we learned, here's an example or two to back that up, and then at the end of the report, that's where I think the researcher can add the value of saying, well, here's what we think this means to our organization, whether that be in this in a set of recommendations, or whether it be in a set of How Might We questions? What are we going to do with this information next? And how can I guide you to figure out what we should do based on what we learned?

Dan Berlin: 10:41

Yeah. How do we tie that back to the the business needs and the business goals? We have these user stories and recommendations coming out of the insights, but you mentioned aligning it to the business, how do we accomplish that?

Susan Mercer: 10:56

I think that really comes up to play in the goal of the research report and the goal of the research study overall. I always start my research by understanding what is it we're trying to accomplish? Let's document that as a cross functional team. I work with my product stakeholders and data analysts and engineers to write down here's what we want to learn. Now, here's some of the questions we may ask to get there. And I have them help me create that list of questions. So that I get a good understanding of what we want to do, I often sometimes turn it around and ask them, What are you going to do with the information we gather? How is this going to help you in your business decision making, and that can give me insight into really what they're after as opposed to what they say they're after. So applying a little bit of research to the research goals. And then once you get that, that can then help inform your research questions that you ask and your methodology. And then I always come back to the research report and start with, Hey, here's the background, here's the set of questions we had, and then here's how we went and got the answers.

Dan Berlin: 12:16

Yep. Great. You had mentioned the visualization, especially for usability and you mentioned quotes and videos. But when we're doing just straight user interviews, maybe in the more formative portion of a project, how do we avoid the wordy report? How do we pull visualizations out of that?

Susan Mercer: 12:41

Yeah, that's a good question. And when I have the ability to work with a designer, they can usually come up with some good things. But myself, what I'll often look at is even if we, you know, maybe even did a survey, and we did some open ended questions there, I might code them, and then create a graph to show Hey, 37% of people mentioned this, 27% mentioned this, only 10% mentioned this. So there's a visual that I created out of open-ended text. You can do word clouds, too. Sometimes those are easy. Sometimes, if it's a sequential thing that you're investigating, creating some sort of timeline and showing where the pain points are almost a mini journey map can help give you some visualization to then say, okay, slide one is going to go into this detail here, slide two is going to go into this section of that diagrammed here. So there's various ways you can be creative. Another thing I often do in interviews is actually maybe have them sketch something out or draw something. And then you'll automatically have some visuals that you can use to help explain, here's how people are thinking about particular issues.

Dan Berlin: 14:00

Yeah. How are you doing that in our virtual world these days? Sketching?

Susan Mercer: 14:08

Yeah, the best method I found to do that virtually is to ask them to have a paper and a dark pen in front of them. And then I have them draw, and then they show it. Since everybody has video cameras these days. You just show it there or take a picture and email it to me. Either way, I've got some sort of visualization of something. You know, yes, there are a lot of online tools and whiteboards. But I generally find that if you're trying to do research with the general user population, they may not have the technical competency with those tools yet. So just regular sketching is much easier to do and can be very easily captured.

Dan Berlin: 14:48

Yep. I like that. You mentioned a dark pen. There's a story there.

Susan Mercer: 14:54

How many times have we all gotten sketches or looked at them later and they're in pencil and you can't even read exactly?

Dan Berlin: 15:00

Right, right. How about those recommendations? So you mentioned that earlier, in terms of bringing it back to the business, but taking these insights and making it as actionable as possible for the next step for the design team, how do we do that?

Susan Mercer: 15:17

I think there's a couple of things with recommendations. One, in some of the people that I've talked to in the field, asked me should researchers make recommendations. I think it really depends on your business and it depends how well you understand your business. Sometimes if I'm new to a particular business, for example, I'll take my findings, and then have a collaborative session with a cross functional team. Here's what we learned, what do we think we need to do? How can we make these things better? That's where I love the use of those, How Might We questions to get them to ideate? Oh, here's some various things we can do to fix the problems you saw you identified. Sometimes, in other organizations that I've worked, that have more UX maturity, the product team is more open to a researcher making recommendations, as long as we understand the business context. So that's where making some recommendations can be good. Always a good thing to talk through your recommendations with the main stakeholder before presenting to a group, for example, just to make sure you have buy in. But really, a researcher also needs to understand the business because good user experience marries the two together. It's no use telling the business to build something that doesn't match the company strategy. An example that I like, and that, you know, kind of absurd is, it would be like telling Instacart to start selling cars. You know, a car is something you can't drive to the dealership and pick up and deliver in two hours and charge to your credit card. That doesn't make sense. So knowing your business context, what your constraints are, can really help you. But there are also times where you're going to find differences. And that's where talking to your stakeholders offline can say, hey, I'm seeing this as an issue. I don't know how we're going to address it. But this is a big thing that keeps coming up. We need to crack this nut. How do we do that?

Dan Berlin: 17:25

On that point about ideating solutions and recommendations, you mentioned facilitating it through using How Might We statements, are there other ways that you've elicited that from others?

Susan Mercer: 17:39

Yeah, we've done basic brainstorming sessions too, around here are some of the findings, what do you think? We've also used journey maps, we're in the process of doing that right now, creating journey maps, where we're highlighting the customer pain points, and then working with the team on that journey to look at, oh, here's the pain point. Okay, what do we think we can do to solve this, but then you move on two steps down and that solution that solved that first point, is going to cause another problem at point B. So being able to look at that across that journey can be sometimes quite valuable, too.

Dan Berlin: 18:16

Yep. Great. So Susan, it's been a wonderful conversation. Is there anything else about your chapter and writing reports and telling the user story that you were hoping to convey here today?

Susan Mercer: 18:27

I think one of the things that I would also say is that is what I mentioned early on about credibility for UX researchers is key. And particularly when I moved to a new organization, the first thing I'd like to do is conduct one or two usability studies to build my credibility. And what I think you can do with the research report, to build your credibility is a couple of things. One of them is you should focus only on observable things and make it clear what you observed. So what you saw participants doing what they said aloud, don't make assumptions. And don't say somebody thought about this, or somebody thinks this is important. Well, they said it was important, make that crystal clear. And then when you do get to the part of the report, where you're making your inferences and your interpretations of the data, make it clear that you're doing that. So here's my interpretation based on what we heard. I think problem a is bigger problem than problem B. And here's perhaps where we should focus first.

Dan Berlin: 19:37

And potentially tying that back to one of the business problems stated in the beginning.

Susan Mercer: 19:42

Exactly. And tie that to their business objectives. And this is going to help us do XYZ better.

Dan Berlin: 19:49

Yep. I love your point of only observable findings, because we are researchers, and we can't make inferences and that's what we do.

Susan Mercer: 19:58

Exactly. I think that's something that folks new to the field don't always learn as quickly as I feel like they should. We need to do better training new researchers about that, I think.

Dan Berlin: 20:09

Right, right. So Susan, thank you for all of that very useful stuff for the audience in terms of writing those actionable research reports that resonate with your coworkers. Thanks for all that. In our final segment here, we like getting a piece of career advice, whether it's for someone breaking into the field, or who has lots of experience, do you have a piece of advice for folks?

Susan Mercer: 20:32

I think the biggest one I have is Be curious. Always ask why. You know, there's that joke about UX researchers and therapists and toddlers, you know, what do we all have in common? We ask why a lot. But really understand it, our stakeholders are coming to us with questions often, or we have our own questions. There's often a lot of assumptions built into these questions. As researchers, we should be curious and ask why and really try to break those down to the fundamentals. And that's going to help us figure out what people really need to answer as opposed to what they think they want to answer. And then, you've taken your first step on your right path to get the business the information they need.

Dan Berlin: 21:24

How about overcoming your own biases when doing that, you know, as humans, we instantly have opinions when we're when we're doing things? How do we stay as neutral as possible when when doing that?

Susan Mercer: 21:37

I think it's good to always check yourself. But I've actually found particularly recently, that even I tend to make biases and make assumptions and the best people to keep me straight and to keep me from making those are my team members, my other researchers that I work with. So we share a lot of things together and they always helped me question, Oh, yeah, I didn't think about that. You're right. I don't want to ask it that way. This is what I think should be doing instead.

Dan Berlin: 22:11

Great. So Susan, thank you so much for joining the podcast today. You gave our listeners lots of great information to help their UX research reports in their careers. So thanks for joining us here today.

Susan Mercer: 22:21

Thank you for the opportunity.

Dan Berlin: 22:23

My guest today has been Susan Mercer who wrote the chapter Tell the User's Story via Effective Research Reports. Thanks for listening, everyone. You've been listening to the 97 UX Things podcast companion to the book 97 Things Every UX practitioner Should Know published by O'Reilly and available at your local bookshop. All book royalties go to UX nonprofits as well any funds raised by this podcast. The theme music is Moisturize the Situation by Consider the Source and I'm your host and book editor Dan Berlin. Please remember to find the needs in your community and fill them with your best work. Thanks for listening.