Ask Participants to Tell You What You Don't Know to Ask (feat. Amanda Rotondo)

Amanda Rotondo, PhD discusses her book chapter "Ask Participants to Tell You What You Don't Know to Ask" and provides some insight about building team empathy in a remote world.

Sponsored by Watch City Research
Watch City Research is your trusted UX research partner

Disclaimer: This post contains affiliate links. If you make a purchase, I may receive a commission at no extra cost to you.

Transcript

Dan Berlin: 0:14

Hi, everyone, and welcome to another episode of the 97 UX Things podcast. Dan Berlin here, your host and book editor. I'm joined this week by Amanda Rotondo, who wrote the chapter "Ask Participants to Tell You What You Don't Know to Ask." Welcome, Amanda.

Amanda Rotondo: 0:28

Hi, there.

Dan Berlin: 0:30

Thanks for joining the podcast. Can you please tell us about yourself?

Amanda Rotondo: 0:34

Sure. I am a user experience researcher in the Boston area. I've been doing this for over 20 years. And yeah, I'm excited to be here.

Dan Berlin: 0:43

Excellent. Yeah. Tell us about your UX journey, how you discovered UX and how you went up where you are today.

Amanda Rotondo: 0:49

I kind of discovered UX as UX discovered itself back in the 90s. I was in undergrad and I came across the concept of hypertext fiction, which was brand new at the time. And it led me to realize what could be when online experiences were user driven. And how that was so different from... it was essentially brochureware and sort of lit the path for me that, wow, there's so much more we can do with this technology. So when I graduated, I got a job, information architecture, of course, and did that for a while and wanted to learn more about the way that what I was doing and the media I was putting out there affected people. So I ended up getting a master's in media effects from Penn State, and then went back to to UX research independently, and was doing UX research and design. But really, during grad school, I fell in love with research. And I did that for a while. And I kind of started teaching a bit to see if I wanted to maybe go the professor route. And I loved it. So I ended up going back for a PhD in human computer interaction. And when I finished that, I did love teaching, but I also really miss just being in the field. So I continued with consulting.

Dan Berlin: 2:07

Great, are there any highlights from working on your on your PhD that are worth sharing here?

Amanda Rotondo: 2:13

For my, for my dissertation, I studied the role of empathy in the success of design teams working at a distance. And this was in started in 2009. So this was way before the pandemic before, you know, non co located working was big, really, at all. And it was also before people were really talking about empathy. Now, it's such a buzzword. And this was at a time where I had to explain to people what that meant. So it was really new and really fun to be working. One of the things that my research... my research didn't come up with this, but it did substantiate it further, is... So when you have a group of people working remotely, so like now we'll just say by Zoom, it can be very hard to establish, you know, team cohesion. And the one thing that was discovered at this point that has the biggest effect on team's success working together and enjoyability of each other being on the team is having had met in person one time. So yeah, and it's so funny, because it's so simple, right? Like, you know, people are trying to think of all these very convoluted things of ways that, that they can build teams from afar and high tech, this, et cetera. And really, you just have to meet in person one time, sort of prove the other is real. And then, and then the relationship changes dramatically. I thought that was so powerful.

Dan Berlin: 3:39

Yep. Makes sense, you know, putting a real face to the name of the people that you're interacting with. And even just that one time, I guess it makes a big deal.

Amanda Rotondo: 3:48

Exactly.

Dan Berlin: 3:49

Yeah. Great. Thanks for that. Your chapter, "Ask Participants to Tell You What You Don't Know to Ask." Can you tell us about that, please?

Amanda Rotondo: 3:57

Sure. So this all boils down to asking one question, and that question is, is there anything you'd like to tell me that I haven't already asked? And it's kind of a strange question to ask. But I found it to be extremely powerful. And I sort of discovered it by accident. I was working on this project. I was in house at the time. And it was for a company that was working to conglomerate various data across multiple online retailers. So we had an attribution system where we had a big team in India that was going through all of these individual pages of items for sale and giving, I think it was over 100 attributes about each one so that they could then be filtered and sorted. So it's a very intensive, very expensive operation. And what was happening is the data from the system was coming back an absolute disaster. It was a mess there. There was a ton of incorrect data that was so weirdly incorrect that it didn't quite make sense that it would be, you know, a misunderstanding. So this fell on me to try to figure out. So we tested the system from just a, you know, is it is it functional perspective, and it was working fine. We did cross cultural tests, because we recognize that we developed the system for a different culture. So we did that, it was fine. We scheduled times to watch employees while they were doing these attribution exercises, and they were doing a perfectly good job, no problem. So, we're just baffled, like, what could possibly be so dramatically wrong? So in the end, we decided to, you know, well, it wasn't really in the end, it was sort of, as we went along, we were interviewing people and talking to various people and trying to figure out where to go next. And one of the interviews I did, toward the end was with somebody who was working the attribution system, just, you know, one of the people on the floor, and I was so frustrated at this point, because we had tried so much, and I couldn't figure this out. And I just sort of blurt it out, I said, you know, is there anything I'm not asking you, you think I need to know. And it was such a weird thing, it just sort of came flying out of my mouth. And the participant just sort of froze for a second. And, you know, kind of looked at me over Zoom. Like, "Well, you just went off script." And then she said,"Well, maybe about the volume bonus." And I was just what are you talking about? And so it turned out the floor managers were incentivized based on how many products had been attributed during their during their shift. So they were then prompted to further incentivize their teams to produce higher volume. So they had sort of this very informal deal, where it's like, hey, whoever, you know, whoever attributes the most product on this shift, I'll give you 50 bucks. And it was this very casual, very, you know, sort of informal thing. And what it did, was it totally upended the system and made it so the the people inputting the data were far more driven to just fly through and get as many as they could versus any concern for quality. So once we discovered that, we realized, well, the problem isn't the system, the problem isn't, you know, the mechanics, the problem is the motivation. And we were able to get in and tweak that system, or tweak their the informal system that had developed and solve the problem.

Dan Berlin: 7:50

That's super interesting. And it speaks to a number of things. First off, how there are happy accidents when we are doing research. And when you have a question that just comes out like that, and that's a wonderful question. But it speaks to the fact that we don't necessarily know all of the things to ask when we go into an interview. And it allows the participant to guide us to the most important things.

Amanda Rotondo: 8:17

Right? And you know, one of the things I always always think about with, you know, what makes somebody a good UX researcher, what sort of at the pith of that, and I think one of the most crucial characteristics is humbleness. You have to recognize that, yeah, you might be the one who knows how to ask questions. And you might be the one who knows how to analyze data, but you don't know anything about the product that you're working on. And you're talking to people who know everything about it. So you know, being able to just just say, "Look, I don't even know what to ask you," sometimes can can result in some some really great observations.

Dan Berlin: 8:51

Yep. And we can learn things that go beyond the things that we're thinking about. In your case, you were concentrating on the user interface itself. But it uncovered this business process problem, which, you know, user experience and business process are often inter intertwined. But it's interesting how that one question allowed you to uncover something that was... not beyond the scope of what you were investigating, but an important tangent.

Amanda Rotondo: 9:23

Absolutely. And I think a lot of the time we're set up to go into these projects a little bit skewed, because you know, more often than not, when we go in to do usability research, user experience research, were guided and sort of introduced to the project by the stakeholders who are most often, you know, higher up in the company. Maybe they don't even use the system. I mean, I can't tell you how many times I've had project owners who have never seen a screen. And so they're the ones really kind of framing the project up for us and telling us who to talk to and, and being the gatekeepers And it's it's extremely common that the problem is something that they don't know to direct us to.

Dan Berlin: 10:08

Yep. No, that's a great point. It allows us to... not eliminate, but check our biases in terms of how we're approaching the research and rely on the participant to guide us.

Amanda Rotondo: 10:19

Exactly. And even when the participants are stakeholders, right. So when you're doing stakeholder research at the beginning, a lot of times, the stakeholders are, you know, they're, they're existing within a power structure, they have a boss, the boss has a an agenda, they have people who work for them that they need to protect in certain ways. There's so many politics and they might not be empowered to speak out on certain issues or to, you know, really kind of say, "Hey, my boss is great, but you know, she's not really guiding you in the right way." So by giving this opportunity in a, you know, one on one interview, in a very sort of comfortable setting, it often lets that power dynamic kind of shimmy to the back a little bit, and lets people speak a little more openly.

Dan Berlin: 11:09

Yeah. So I was really happy when you wanted to write this chapter. Because that question is my favorite part of the research. Right? Is there anything I didn't ask you today that you think is important for me to know? And it's so important, or so useful outside of research too. I mean, we're always researching in our world, right. But when we're doing, as you said, when we're talking with stakeholders, or we are chatting with colleagues, we don't know all the questions to ask, and I think people appreciate when we look to their perspective for what we should be thinking about.

Amanda Rotondo: 11:44

Exactly. And there's two times when I'm out in the wild, you know, in my own personal life, where I realized, like, "Oh, God, I'm a researcher." The first is when I ask a question, and like, you know, unbiased, I'll say to my kids, like, "Do you want to go out for ice cream or would you prefer not to go out for ice cream?" And then the second is when I bust this question out at the end of, you know, just regular interactions and I do it, because like you said, there's so much value in it, you know, just in so many different situations.

Dan Berlin: 12:13

Yep. So I'm a UX consultant, always trying to sell business. And honestly, I always ask that question when scoping a project. It's because scoping a project is its own research project. And so look to the potential client to help guide me on the things I may not be thinking about.

Amanda Rotondo: 12:33

Sure. I think that's a great question to ask at that point. And also, I think it clues your potential client into the fact that you are aware that you don't know everything. And you may stand out within their their process as somebody who's who's truly curious. And like I said, humble, and, you know, really going to try to find the answers and not just presented a shiny research report.

Dan Berlin: 12:57

Yep. And I love what you said about keeping humble because that's what we have to do as researchers because we don't know everything by a longshot. And humbleness is what allows us to be curious and being curious allows us to uncover deep insights.

Amanda Rotondo: 13:12

Exactly.

Dan Berlin: 13:13

So it was there anything else from your chapter you were hoping to convey here today?

Amanda Rotondo: 13:16

Are you asking me if there's anything that you didn't ask me that I want people to know?

Dan Berlin: 13:21

Exactly. Well done, well done.

Amanda Rotondo: 13:25

You know, when this question kind of came flying out of my mouth, I was surprised at myself because it felt very uncontrolled. Right? As a researcher, I think you're supposed to sort of... there's this idea that you're supposed to be very formal, very, you know, bullet pointy, kind of, and this was not an instance of me doing that. That was this was not me playing that role. This was me being myself. And I think that to me, was a very powerful observation to that. You have to be yourself when you're doing your research. First of all, because, you know, you kind of have to have faith in yourself that you, you're doing a good job, you know, you know what you're doing, you've got the right instinct for this. And then second of all, just the extent to which participants recognize when you're being yourself and being genuine. And that makes a huge difference in how much they're willing to disclose to you.

Dan Berlin: 14:19

Very well put. You have to be yourself. People and participants will sniff out acting pretty quickly and clam up. If you're genuine, it's definitely gonna go a long way to keep them comfortable.

Amanda Rotondo: 14:31

Exactly. And every time I've tried, you know, early in my career when I tried to be the formal fancy researcher, it's been a disaster,

Dan Berlin: 14:38

Agreed 100% And it's a balance between you know, regular, everyday you versus what's in who you are in the lab. But you have to let that that personality come through, that genuineness come through.

Amanda Rotondo: 14:52

Exactly.

Dan Berlin: 14:52

Great. Well, Amanda, this has been a wonderful conversation. Thank you so much for joining me here today.

Amanda Rotondo: 14:56

Absolutely. This has been fun. Thank you again.

Dan Berlin: 14:58

My guest today has been Amanda Rotondo, who wrote the chapter "Ask Participants to Tell You What You Don't Know to Ask." You've been listening to the 97 UX things podcast. Thanks for listening, everyone. You've been listening to the 97 UX Things podcast companion to the book 97 Things Every UX Practitioner Should Know published by O'Reilly and available at your local bookshop. All book royalties go to UX nonprofits as well any funds raised by this podcast. The theme music is Moisturize the Situation by Consider the Source and I'm your host and book editor Dan Berlin. Please remember to find the needs in your community and fill them with your best work. Thanks for listening