Diverse Participant Recruiting is Essential to Authentic User Research (feat. Megan Campos)

Megan Campos provides practical tips for ensuring a diverse recruit during user research.

Sponsored by Watch City Research
Watch City Research is your trusted UX research partner

Disclaimer: This post contains affiliate links. If you make a purchase, I may receive a commission at no extra cost to you.

Transcript

Dan Berlin: 0:14

Hi everyone and welcome to another edition of the 97 UX Things podcast. Dan Berlin here, your host and book editor. I'm joined this week by Megan Campos, who wrote the chapter Diverse Participant Recruiting is Critical to Authentic User Research. Welcome, Megan.

Megan Campos: 0:29

Thanks, Dan.

Dan Berlin: 0:30

Thanks for coming on the podcast. Can you tell us a little bit about yourself?

Megan Campos: 0:34

Yeah, I'm excited to be here. Excited to be included amongst this group of really talented people, authors. So I'm a UX researcher. I am the director of Experience Research at Mad*Pow in Boston, where we formerly worked together. I have a degree I have my master's degree from Bentley University in Human Factors in Information Design and I care quite a lot about inclusive research practices.

Dan Berlin: 1:01

Great. And can you tell us about your UX journey? How did you discover UX? And how did you wind up where you are today?

Megan Campos: 1:09

Yeah, so like everyone else, I think I have a winding journey to get here. I was a sociology major as an undergrad, which led to a lot of interest in the nuances of identity, how people relate to one another, what society looks like, and why it looks that way. From there, I worked in academia for quite a while. I was in development; so fundraising, which is a whole thing. Found my way to website strategy, which involved information architecture, brushed tangentially with UX and gave me an intro to Bentley as a potential course offering, pursued that for a couple years thinking about whether or not UX was something I wanted to enter into, given my background in sociology and interest in research, and then ultimately took the course fell in love with it. Found Mad*Pow after graduation, and the rest is history.

Dan Berlin: 2:07

Great. Can you tell us about that transition from academia to UX?

Megan Campos: 2:14

Yeah, I don't think it was a direct transition. I knew that where I was working, what I was doing was not what I wanted to do. Fundraising is a lot like cold calling and sales, it's not fun. You're reaching out to people and asking them for money. And while I believe in education, it just wasn't a calling for me. So I was really interested in what this firm Jackrabbit was doing in Milton. They were making websites, doing design work, I've always had a little bit of a creative flair, so that's how I ended up trending in that direction. But I didn't find the work of just website design without any meat behind it to be really fulfilling either creatively or personally, professionally. UX and what it offered as far as depth and the ability to really dig into the human experience were super appealing to me. And I think that really explains the crux of the transition. The actual steps I took were a lot lengthier.

Dan Berlin: 3:08

Right. Can you tell us a little bit about that? Actually, you mentioned going to Bentley, but any other steps, they're worth mentioning to folks?

Megan Campos: 3:16

Yeah, I think the the most critical thing I did while I was at Bentley was do a lot of informational interviews. I am not someone who is naturally inclined towards networking. But I also didn't have that many connections to UX professionally outside of the context of my Bentley cohort or other alums. So I just started tapping into what network I did have asking... It was sort of a domino effect. I asked the first person I knew who was in UX to introduce me to someone else, had a lot of conversations about workplace, about the UX job description, about whether research or design were more my calling, and then ask that person to introduce me to someone else, and someone else, and someone else. And that was really how I learned about what it means to work in UX, how I understood that I wanted to be more on the research side than the design side, and how I built up a professional network.

Dan Berlin: 4:07

That's wonderful. And that was pre-pandemic. These days, that's a great way for folks to get out and network.

Megan Campos: 4:14

We were allowed to handshake then, which was nice.

Dan Berlin: 4:15

Right, right. It was. Great. Thank you for that. Your chapter, Diverse Participant Recruiting is Critical to Authentic User Research. Can you tell us about that, please?

Megan Campos: 4:25

Sure. So, I mentioned that I have an interest in inclusive research that, I think, is rooted in my sociology background. It's also rooted in my own journey through identity. I'm half Latina and half Irish, which, for me, growing up was confusing, interesting, continues to be something that I engage with regularly. But I think that made me more observant of who we were talking to when we did user research. So when I was in grad school, I noticed that we were fielding mostly white, mostly middle or upper middle class, mostly fully physically able people, highly educated, not a whole lot of diversity. And I would ask why we weren't making more of an effort to pull in a more diverse field and the answer was always that it didn't matter. We were running a usability test or we were looking at something that was in a vacuum, the real world didn't come into play so much. But the truth as I see it is that who we are plays a lot into how we behave, what we experience, what we believe, what our attitudes are. So I started thinking about how we could yield a more diverse recruit. I also wanted to understand whether this was a problem I was seeing or whether it was something that everyone in the field was seeing. In the process of developing a thesis around this, I ran a survey and as it turns out, a lot of people are seeing that same thing and they're a little bit mixed on whether or not to care about it. So my chapter really focuses on the need to have a diverse recruit and also the tactics that one might take to get there. I talked a little bit about some of the things that I do and that our practice at Mad*Pow does now on a regular basis to make sure that we don't have that all white, all heterosexual, all upper middle class, all highly educated field to work with and it's really yielded some interesting findings and insights in the studies that we run. So my chapter takes on those tactics.

Dan Berlin: 6:28

Wonderful. Yeah. And working with Megan through those years has been such a good influence on me and my practices as well in terms of the recruiting and making sure there's an inclusive audience there. So I want to thank you for that.

Megan Campos: 6:43

No, thanks for deploying the tactics.

Dan Berlin: 6:45

Yeah. Cool. So let's dig in there. Can you tell us a little bit about those tactics?

Megan Campos: 6:50

Yes, and fortunately I have my book in front of me, so I won't forget everything. So the first is to really do some thinking around who it is you want to recruit. I've done a lot of thinking recently around not wanting to boil the ocean. Because if you try to include absolutely every one of every identity of every level of intersectionality in every recruit, you're going to get, at best, one of everything. And as we know, there's a minimum you need to hit for user groups. So thinking really deeply about who you're not speaking to who could be a critical voice in your user audience and making sure that you're being thoughtful about how they get included in the recruit. But that first step is that identification. Think about what diversity and inclusivity mean in the context of your product or experience and who you need to include who hasn't been traditionally included.

Dan Berlin: 7:40

Question for you there. We often hear that the audience for this product is everyone. How do we get by that? How do we get around that statement?

Megan Campos: 7:51

I think slowly and steadily. Just because you're not including everyone with every study doesn't mean you can't eventually talk to everyone. I think there's also an argument to be made, I've heard other folks make this argument, that you could start by over recruiting or over representing those people who haven't been historically represented, because we've spent so, so long talking to the same group of people, it may behoove us to go the other way. Start recruiting more black folks, start recruiting more folks who identify as LGBTQIA+, because then we can start to build a database of information that will allow us to make more inferences, that will allow us to have historical knowledge about their wants and needs as they relate to our product or experience. And you can add on more characteristics as you go. But start by identifying the most salient, neediest groups that you haven't yet talked to you and work from there. So I think the short answer version of that is eventually get to everyone, but start with a select few.

Dan Berlin: 8:51

Yep, that's a great point, the knowledge base that we have is built from such a homogenous group. So let's going the other way.

Megan Campos: 9:00

Yeah. And we may circle around to this eventually, but I think the argument for inclusivity is that when you build for the needs of a niche or marginalized community, you end up making the product or experience better for the whole. I can think of instances when we've done inclusive recruiting where things that have been brought up as a really significant pain point for one group that hadn't previously been spoken to was a less significant, but still present, pain point for the more homogenous group of people.

Dan Berlin: 9:30

Yep. Yep, absolutely. Same with accessibility. It's the same story.

Megan Campos: 9:34

Yeah, exactly. It's worth thinking of them in the same breath, because I think the approaches we take are going to be similar across accessibility and inclusivity more generally, but we've only really focused on accessibility for a long period of time, when it's worth taking identity into the conversation as well.

Dan Berlin: 9:53

Right. When we design for everybody, we make it better for everybody.

Megan Campos: 9:58

Exactly. The other thing I found when we were going through recruit screeners in grad school, and then in the beginning of my time at Mad*Pow, is we were asking for a mix of people. We'd say we want a mix of racial identity, or a mix of ethnicity, or a mix of gender and usually it did not result in a mix. We'd get maybe one Latino person, or maybe one person who identified as gay, if any. I have a memory of this one recruit we ran where we asked for, I think, three to five people who were not white, and the recruiter came back with 100% white people. So the lesson learned there is to get really, really specific with the numbers that we ask for. Usually it's not a range, it's not three to five, it's no fewer than or, exactly three or exactly five. And when it comes to the breakdown of those numbers, sometimes you can look at your existing user population and make inferences there. If that doesn't exist, I've used census data with the acknowledgement that it is not 100% accurate of some people, especially immigrant communities, people of color, sometimes do not identify in the census quite as much so also speaks to the need for over representation of some groups. But I found that the more specific I am with my quotas and my margins in a recruit screener, the more successful I am at yielding a diverse participant list.

Dan Berlin: 11:21

Yep, yep. So it's so important to be specific in screeners overall. Tell me about that time it came back 100% in one group. What's what's going on there? Why is this happening?

Megan Campos: 11:34

Recruiters, I think, will tell you that there are certain populations that are hard to find. I would argue that we've always looked in the same places. A lot of spaces are unidimensional in some ways and if recruiters are going to the same locations, or the same phone banks, or the same communities to make their asks, they're going to get the same group of people every single time. I think there's a huge need with recruiters to go into new communities, look into new spaces, look at creative ways of recruiting and building trust with organizations like churches and hospitals. Because otherwise, yeah, it's gonna be hard to yield people who don't look the same as everyone else we've talked to because they're going to the same pools over and over and over again. So I think, really, the lesson learned is we need to ask a little bit more of our recruiters and really expect them to meet us where we're at with what we're asking for. And be really exacting with those requirements because we're paying them, which maybe sounds a little cold blooded, but at the end of the day we need to treat it like a job. We're asking them to do a job. And if they can't do the job, we should move on to someone else.

Dan Berlin: 12:41

Absolutely. And if they're not willing to put in the effort to look in different places, if they're only willing to rely on what they know, then they're not going to be a great partner.

Megan Campos: 12:51

Yeah. I think too, my perspective on this comes from just using recruiters. I've spoken to this at conferences, too... there's a lot to be said about the value of friends and family recruits and using personal networks to yield studies because, as with recruiters, our personal networks tend to look a lot like us. And so doing any kind of guerrilla study user research, or using a friends and family recruit actually probably means that you need to go outside of those comfortable networks.

Dan Berlin: 13:20

Great. Thanks for that. What other tactics do you have there?

Megan Campos: 13:23

Yeah, well, we've gone over the the making the argument piece, that arguing for the need for inclusive research is sometimes a little bit of an uphill battle. In our case, we're an agency, so clients will come to us with one expectation for what user research might mean. And we often have to make the case for inclusive because it's not something that's on their radar, or it's something that is undervalued. We've been lucky recently, I think inclusivity has actually been more of a selling point. But we've had to make the case, that argument that building for the niche or the marginalized really means improving the product for everyone, has been an effective one. Again, the cold blooded argument there is improving the product sells the product, makes the product more engaging. But the human argument is it's going to be a better experience.

Dan Berlin: 14:09

Right. If you tie it back to the business, that's definitely the way to get the business's attention. Right? And there is a clear business case for inclusiveness because it will make it better for everyone and get more more customers.

Megan Campos: 14:23

Yeah, I mean, definitely right now, while inclusivity is a bit of a buzzword, where we're pushing our practice development forward to make sure that we're being progressive in how we're attacking this, not just now while it's a hot topic, but going forward so that it's just embedded in our practice. And it's something that we're keeping at the forefront and growing and evolving as as our abilities become more refined and as the need becomes clearer. I guess the one thing to drive home with the recruiter piece a little bit more is staying with the numbers that you request. So with that one example where the recruiter came back to us with homogenous group of participants, we had to go back to her and say, nope, that's not what we asked for, we need to see at least this, this and this, and then came back again with not this, this and this. And we had to go back again and say no, we really mean it, this is what we're looking for, we won't budge on this. And then she delivered, we got what we asked for. I've been starting to be proactive about this with recruiters, meaning that those recruiters that we use over and over and over again, I'm reaching out to at the conclusion of projects to say, just so you know, this practice of turning around a homogenous participant list is not going to be acceptable anymore. Our expectation is to see participant lists that match our audiences, which means that they need to be diverse and sometimes over representing those criteria that we consider to be diversifying. And that in itself has gotten a mixed response, but I think it sets the stage that going forward this is going to be our expectation. And when I have to make a request to send a bid in for recruitment, the expectation is redundantly made there so they know what I'm going to be asking for. And if they're waffling or they don't think they can deliver or they're asking for more money for diversity, that to me tells me about the quality of the recruiter.

Dan Berlin: 16:11

Yeah. Now, you and I have the luxury of choosing recruiters generally, because we work as consultants or in an agency. But often enough, large companies have contracts with large recruiting agencies and some folks can't go out of that. Any tips for working with existing recruiters who traditionally may not have this, that folks are currently working with?

Megan Campos: 16:42

Yeah, I think it's outside of my realm of experience and expertise, so these are assumptions that I'm making here, but I think that part of it is being a partner to the recruiter in the same way that we expect a recruiter to be a partner to us, a good trusted partner. Which means working together and coming to a mutual understanding. So if you have a recruiter you have to work with, bring them to the table before a study is this happening to say, this is the direction we're going in, we need you to go with us. What are your current practices and do we need to make any adjustments? Can we help you make any adjustments to those practices so that you can reach the people we need to speak to? And again, I don't know how malleable those relationships are but it seems like if you're on a long term contract with someone there's, again, based on how we work with our clients, there should be flexibility and willingness to change according to the new standards or whatever the standards are that are evolving.

Dan Berlin: 17:36

Yeah, definitely. And folks may need to work with procurement, for example, to make that case.

Megan Campos: 17:43

Yeah.

Dan Berlin: 17:44

So was there anything else about your chapter that you are hoping to convey here today?

Megan Campos: 17:48

I think that there's a broader theme, right? So recruitment is one step in the research process. Research is a whole lot of processes all the way through to actually facilitating interviews, doing synthesis, analysis, and readouts. The whole process needs an overhaul. I think this is a really important first step but the things I'm thinking about that are on the horizon include moderator positionality, thinking about the the privilege and position of the moderator while they're facilitating the study, thinking about any biases and assumptions that are explicit or implicit, during the analysis process and thinking about how that bleeds into the readout and any future studies going forward. There's a tendency, I think, to treat inclusivity as a one and done or a silver bullet, and it is not. It is a thing that has to keep growing and evolving. And I certainly do not know the most about this, I know very little about this. I know one small corner of this. But I think if someone were to take away anything from this chapter is that there are small tactics we can take to make a meaningful difference that will add up over time. And I really hope that this push towards inclusivity is just the beginning of a larger trend that continues.

Dan Berlin: 18:58

Absolutely. There are so many moving parts of the research ops process of communicating with participants or using a recruiter or as you said, moderating, and there are so many opportunities for us to be mindful of ways to model our behavior and our actions to make sure that they are inclusive to everyone.

Megan Campos: 19:20

Yeah, agreed. Research is foundational. I know at Mad*Pow it's part of almost all of the work that we do. And so whatever decisions we make or practices that we change have a significant follow-on effect. I can see that happening even as we just start to think about it. So yeah, I think there's a lot of ability for researchers to make a change at every level.

Dan Berlin: 19:41

Yeah. So thank you for all of that. In our final segment here. We'd like to get a career tip for folks, either for folks breaking into the field or have been in the field for some time. Do you have a career tip for folks?

Megan Campos: 19:54

Can I go to one early stage what late stage?

Dan Berlin: 19:57

Sure.

Megan Campos: 19:58

Early stage I would totally recommend doing those informational interviews. Like I said, I find networking painful, it is not a thing that comes naturally to me. But I am so grateful to the people who took an hour to have coffee with me and talk to me about their careers and what choices they made and what they would have done differently and who they know, what they were, where they've worked, what job descriptions mean, what different titles mean. All of that was so valuable to me as I approached graduation and entered the job market. So new UX folks, talk to people and ask for other people to talk to. That's like, that's my biggest piece of career advice.

Dan Berlin: 20:37

Yeah and UX people are the nicest people in the world. And they were so willing to talk to folks, so people shouldn't be scared about just approaching strangers even, to ask.

Megan Campos: 20:49

Yeah, the first person as I was exploring whether or not this was a career for me, I bumped into someone at a conference who asked how I was doing and we started a two hour long conversation just nerding out about the possibilities of UX. And this is a person I've never met before. So yeah, friendliest, biggest group of nerds.

Dan Berlin: 21:06

Perfect. Yeah. That's UX for ya. How about that second piece of advice?

Megan Campos: 21:10

Just keep driving this inclusivity thing. I think it's not career advice so much as it is our whole practice needs a little bit of a shift and a little bit of a shake up. And so everyone has a voice, and everyone has some sphere of influence, and if we can use that to make our practice more inclusive of a broader group of people and give a voice to people who have not previously had a voice in the conversation that's commendable and that's what we should be trying to do.

Dan Berlin: 21:34

Amen. And design often has a unique place in a business's product timeline to be that department that does make those steps to make the process more inclusive.

Megan Campos: 21:47

Yeah. And at several different points on the timeline, right? We're, in theory involved from start to finish.

Dan Berlin: 21:54

Yep, exactly. Yeah. Great. So our guest today has been Megan Campos, who wrote the chapter, diverse participant recruiting is critical to authentic user research. Thanks for joining us today Megan.

Megan Campos: 22:04

Thank you so much for having me.

Dan Berlin: 22:06

You've been listening to the 97 UX things podcast. Thanks for listening. You've been listening to the 97 UX things podcast companion to the book 97 Things Every UX practitioner should know published by O'Reilly and available at your local bookshop. All book royalties go to UX nonprofits as well any funds raised by this podcast. The theme music is moisturized and situation by consider the source and I'm your host and book editor Dan Berlin. Please remember to find the needs in your community and fill them with your best work. Thanks for listening.