Remember the Four Questions of Critique (feat. Adam Connor)

Adam Connor discusses UX maturity in organizations and best practices for critique.

Sponsored by Watch City Research
Watch City Research is your trusted UX research partner

Disclaimer: This post contains affiliate links. If you make a purchase, I may receive a commission at no extra cost to you.

Transcript

Dan Berlin: 0:14

Hi everyone, and welcome to another edition of the 97 UX Things podcast. Dan Berlin here, your host and book editor. I'm joined this week by Adam Connor, who wrote the chapter 'Remember the Four Questions of Critique'. Welcome, Adam.

Adam Connor: 0:28

Hey, thanks for having me.

Dan Berlin: 0:29

Thanks for joining the podcast today. Can you tell us a little bit about yourself?

Adam Connor: 0:34

Sure. I am currently the VP of Product Design for Rocket Mortgage and the Rocket family of companies. I've just joined there, I've only been there about a month. Prior to that, I was Head of Design and Research for a company called Elixir. Before that, I was with Mad*Pow for a bit, and in a company called MassMutual.

Dan Berlin: 1:01

Great. Can you tell us your career trajectory? How did you discover UX? How did you wind up where you are today?

Adam Connor: 1:13

Like many people my age, I kind of stumbled on UX. I actually went to school to study film and animation originally, then for a number of reasons didn't see that all the way through and fell into getting a degree in Computer Science. One summer I really needed a job, and because I could draw from my illustration skills from studying animation and knew a bit about computers from my computer science classes, MassMutual hired me to basically make banners for their intranet, as an intern. I ended up staying there for nine years total but the reason I stayed was because not long after I joined, there was a guy there named Michael Rollins. He was working various digital projects and introduced a lot of the leadership there to this topic called Usability at UX and it caught on. I had the great honor to work with him and learn from him, and be a part of how UX matured at MassMutual for the nine years I was there. That's what really got me hooked. From there, I went to Mad*Pow, I was with Mad*Pow for 11 years. The last five or six years there, I really realized that, hey, I love this UX stuff. I love human-centered design. What's even more exciting to me is figuring out how to make it work in an organization, how to actually get people bought into it and collaborating throughout the process. That's where I did a lot of work- basically using all of Mad*Pow's clients as research participants and just studied what worked in certain organizations, what didn't work in other organizations. We developed the design transformation practice based on on those learnings and the methods and tools we developed there. That's what got me into Design Leadership. Elixir hired me to build a design and research program there. Rocket brought me in to help with the growth and scaling of their program. That's how I got in- those are the kinds of challenges I love- building the culture and organization design to really let companies get the value that design can really bring to them.

Dan Berlin: 3:51

That's actually been coming up in conversations a lot lately that I've been having with folks- the UX maturity model and where companies are on that maturity model, whether they're just starting out in UX and realizing that usability is a thing. as you said. Or whether it's deeply ingrained in their culture- what are some of the biggest challenges that you've come across in instilling UX in an organization?

Adam Connor: 4:16

Funnily enough, the maturity model itself is actually I think one of the biggest challenges. At least most of the maturity models I've seen, because I honestly think they're far too simplistic in terms of offering guidance. I think that's why they're adopted a lot because they tend to make it seem- not simplistic in terms of easy to get level to level, but at least understanding different levels. One of the things we found in our work at Mad*Pow is that maturity, like when you're talking to say, a design team- a team of one or twenty or a hundred- about what they want to do to get to whatever that next level is- strategies and techniques are focused on three different areas. We broke maturity down into three different areas. One is 'scale', which is about the volume of work that you want to do, the number of places you want design methods to be used. Strategies there tend to focus on things like tools and frameworks and things to make things repeatable, or take out a lot of the rework that needs to happen so you're not starting from scratch all the time. Next is 'strengthening', which is really about the quality of the work and the impact it has and that often gets to things like measurement and really framing of problems and understanding the problems that you're trying to solve for and the connection between research and design and iterative cycles, and things like that. And then finally, is'depth'. How is the organization using design? What types of questions is it trying to answer? At the shallowest level you have design as interface design. Design decides how our products and services look. Then you go to Product and Service Design- which is Design is how our products and services work. Then you move into Experience Strategy, which is, what types of products and services should we offer to these problems we want to solve for our customers. Finally, the deepest level, which not every organization gets to, but it's a possibility is- business strategy where design methods are actually helping organizations find problems to solve and new ways to offer value to customers. Those types of approaches are the things you need to work on, those really have to do a lot with the types of methods you're using and the relationships you're forming with the other areas of the organization. When we broke maturity down those three ways, it really helped us focus in with our clients on what they really wanted to work to achieve, and how to get there faster.

Dan Berlin: 7:03

Very cool. That's super interesting and definitely something unique- because we tend to talk about methodologies or Design best practices, not much talk about the UX maturity model, and different ways that we can be looking at it. What you've described is a much less simplistic way of looking at it, but it sounds a lot more meaningful.

Adam Connor: 7:23

Yeah, the impetus behind it was, you know, we'd be talking to people- I did this with Alexa, I've done this with Rocket and others. We talked about wanting to get to that next level of maturity, but that's where the conversations get stuck. We can sense that there's all these challenges but really, what are the meaningful actions we can take? What are we really trying to do? Is it a question of scales? Strength? Depth? Some combination, thereof? - and then kind of picking the right levers to pull to get there.

Dan Berlin: 8:01

Well, thank you for all of that. Let's turn to your chapter now. The chapter is'Remember the Four Questions of Critique'. Can you tell us about that, please?

Adam Connor: 8:13

Critique is a topic that I'm hugely interested in, because I fundamentally believe that great design comes from a group of minds, not any one mind. In order to collaborate with people, and really move something towards execution- getting out in the world, we need to be able to talk about the things we're creating and analyze them, and really communicate with one another to decide, are we moving towards our objectives? Is this solution really going to work towards the thing that we're trying to achieve? That's what critique is. Critique is a form of analysis, in which you compare something- a solution, a decision, a plan that you have for something against the objectives you have for it, and decide whether you think based on everything you know, if you think it's going to work towards those objectives, and why or why not? Those are really the four questions. What objectives are we working towards? What thing are we analyzing against those objectives, whether it's a whole solution or some aspect of the solution? 'Solution'- I use that to be very generally just whatever it is we're analyzing. It could be an interface we're designing, a research plan that we're putting together to answer some sort of question, a business process- do we think that this thing we're analyzing works towards these objectives? Let's take into account- what do we know about our audience? What do you know about the context in which this thing is going to be used, etc? Yes or no, and why or why not, right? Those are four questions and I say it's really important to remember that because many of us have had the experience of getting into these kinds of conversations where we're sharing a solution or a potential solution, and the conversation just goes off the rails. People are picking apart all sorts of different things. They start sharing ideas and brainstorming, they're starting to talk about things they've experienced in other spaces, and the conversation just goes in a hundred different directions, when really what we want to do is get to this level of analysis, not to judge it as like, Yes, Pass, or Fail, but, where are the opportunities we have for strengthening this solution? Also, what are the strengths of the solution that we might borrow from to make other aspects even stronger? Yep.

Dan Berlin: 10:46

The objectives there, you mentioned coming back to the objectives, and always looking back to the objectives when you're looking at the thing that you're evaluating? How do we get to the right objectives? I can imagine that having the wrong objectives on the table can lead us in the wrong direction.

Adam Connor: 11:06

There's really two challenges that we see there in the day to day is having the wrong objectives, but also not having any objectives, or maybe even a third one of not having the same objectives. There's a level of work that needs to go into coming out of the research that we've done, hopefully, we've done research, and just mindshare that we've created amongst the team of- what's the business problem we're solving? What's the audience problem that we're solving? What is the audience need, in order to feel like that problem has been solved for them? What is the business need? We need to really detail those things out. You can frame them in any number of ways, like How Might We-s use and Jobs To Be Done, or whatever. There's plenty of great tools out there for structuring that. But, we need to make sure we develop that as a shared understanding ahead of time. A lot of times, Aaron Irizarry, who I've done a lot of this work with, when we would work with organizations on this, we used to talk about observing critiques as like a canary in the coal mine. It's a great way to see just how together a team is because you can start to watch where people are critiquing a solution and I use air quotes there. But, they're all critiquing it against some separate set of objectives that they've got in their own head that isn't necessarily shared with the other team members.

Dan Berlin: 12:36

It sounds like the objectives are what separates opinion from critique.

Adam Connor: 12:43

Well, there's always some element of opinion in critique. Your goal is to be as objective as possible against those objectives. But, there will always be some subjectivity based on what you know and understand of the audience in the context. That's another thing to keep in mind with critique- it is not a substitute for research, it is a great form of analysis, it's a way to bring people together to really use the knowledge that you have as a group so that you don't have to test every little thing necessarily, but it is NOT a replacement for actual testing and actual data to validate a hypothesis.

Dan Berlin: 13:30

How about actually doing that critique? Doing critique is not something that is natural for folks where we're used to giving our opinions and telling folks what we think about things, but how can we be better critiquers?

Adam Connor: 13:48

I think the easiest step is to create time and space for it. Longer term, or overarching in terms of what you want to build in your organization- is more of a culture of critique where people don't necessarily need the space and time it doesn't always have to be a formal meeting. But, a great way to start is with those formal meetings. Thinking about, do we set up one hour every week for people to share work in progress? Do we create, something I've been a fan of is- if you use Slack or Teams or something like that, an asynchronous channel where you can post something- not necessarily to ask for critique in that channel, because asynchronous conversations can be a little tricky with critique, but to find somebody to get feedback from. Say, hey, I'd love to sit down with somebody for 15 minutes. Creating those spaces is step one. Then, when you're in those spaces, making sure that you set the context- here's the problem we're trying to solve, here's the aspect of the solution that we want to focus on right now, here's the type of feedback I'm looking for. Depending on who you're critiquing with, you might even want to talk about what critique is. My friend Aaron is a big proponent of going over the rules of critique with some folks ahead of time, really making sure they understand what's expected of them. Setting that context and really focusing the conversation goes a long way. If you're asking for critique, understanding what your role is as the recipient there, your role is not to defend the work in any way, it's only to provide information and insight as to why he may have made certain choices and provide any kind of additional details people need in order to give you that feedback. Ideally, if you are the person requesting it, participate in the critique as well. Critique the solution along with everybody else, that actually can help you get out of that defensive headspace a little bit and also demonstrate to others the type of insights that you're looking for.

Dan Berlin: 16:20

That's a great point. We forget that critique can be or has to be a two-way street. We have to know how to give it and we have to know how to receive it constructively and to take what people are saying in mind. I think it's important to also remember that critique isn't just for designs. There's opportunities for critique in many different areas of UX, whether it's content, or IA, or your research guides or reports or even your research moderation styles. There's opportunity for critique throughout all of UX.

Adam Connor: 16:56

I would say it goes beyond UX. We used to get made fun of a lot about this. But, Aaron and I used to have a slide in our presentation decks that would talk about critique is not a design skill, it's a life skill. Anything that you want to improve that, there's an opportunity for critique there. Lately, I've been talking a lot to teams about, what kinds of things can you critique? It seems to be coming up a lot in terms of conversations around feedback and giving people feedback. The things I talked about that you can critique are, you can critique things that people create- artifacts, constructs. You can critique decisions- whether decisions work towards objectives. But, you should never critique people. That's one thing I encourage people to stay away from. Once you get into that space, you're scratching at something that maybe isn't worth getting at.

Dan Berlin: 17:55

What's the difference between, it's gonna be subtle I presume, the difference between someone's decision and critiquing someone?

Adam Connor: 18:07

Tthe way I think about it, our decisions are notions at a point in time based on context that is present at that point in time. I think the mistake we sometimes make is that a decision is a reflection, or an extension of a human being. When you're critiquing a decision, you want to talk about it more as the decision itself, and the context that was used, the information that was used to make it- not the person's values, not the person's beliefs, not the person's attitudes. Once you start treading into that water, you're now making a comment about the individual. Chances are, you're going to do something that hurt that relationship quite a bit. Trying to, as much as you can, separate the individual from the thing you're critiquing is really important in my mind,

Dan Berlin: 19:11

One place where that is particularly hard, is moderation critique, because it's you. It's sitting with the participant and finding ways to get the data out as best as possible. That's you on the fly and your behavior, and being open to critique on those behaviors and how you can improve on that it was really hard, but it's so important.

Adam Connor: 19:37

I think that really also goes to the intent behind having a critique conversation. It's really important that entering into that conversation, both parties- the people giving and the people receiving have the right intent. If the person receiving the critique isn't really in the headspace of wanting that critique, of wanting to understand, like- Oh, how did my actions at this point in time affect, or maybe hinder my objectives? If they're not in that headspace, that conversation isn't gonna go well anyway. Both parties really have to be there. The people that are giving the feedback really have to be there with the intent of wanting to help that person help improve those situations in the future, not about- this is how I would moderate therefore, you should do it too.

Dan Berlin: 20:31

The skill is a team building skill, the way that you communicate while critiquing necessarily builds the team, because we're doing it in a constructive way.

Adam Connor: 20:46

You've got to think about critique in the context of all the other conversations when you're thinking about it in team building, too. If all of your conversations as a team are critiques, then that becomes the whole nature of your relationship. That's not necessarily a strong relationship, either. It's not that that means- just do all your criticizing and other conversations, you hopefully aren't doing any of the criticizing, but make sure that you know, there's kind of a shared understanding of when we're in this type of conversation- this is the way we're communicating. This is the type of stuff we're sharing. When we're in our end of the week team get-together to just kind of chat and close out the week, that's a different kind of conversation. We can talk about what's going on at home, or, what we're looking forward to for the weekend. It is definitely an important aspect of team building, but it can't be everything. Now every conversation is a critique.

Dan Berlin: 21:50

I want to go back to something that you said earlier about using other people's ideas that come out of critiques. This is something that you taught me many years ago, with Design Studio- I remember you giving the instruction of when you're doing the charettes- you're doing sketching, and you're iterating on those sketches, and critiquing each other sketches. We always encourage people to steal each other's ideas in the name of evolving and the design. That's a great way to build teams as well, to understand each other's viewpoints and to make the team better as a whole.

Adam Connor: 22:28

One of the best ways that I know to use critique, I mean, you can always do the standalone critiques like what we were talking about, but really pairing it with brainstorming with the group as well. This is the idea behind charettes in Design Studio. During a critique- critiques aren't about brainstorming. You want to actually work while you're doing the critique to try to make sure you avoid exploring other options, you kind of limit that as much as possible. But then, as you're having that critique, people are going to be getting ideas. People's brains just naturally go to problem solving those times. Following up with that critique with- Okay, here's where we are now. Let's look at a couple of these spots where we can make things better. Now, let's all brainstorm some different ways in which we could improve that. Using that distributed cognition, the brainpower of multiple people to really diverge and find other ideas- that can be really powerful at times.

Dan Berlin: 23:39

Was there anything else about your chapter and critique that you were hoping to convey to folks here today?

Adam Connor: 23:44

I really think that critique is one of those aspects of our discipline of any kind of design-oriented, creative discipline, that it's easy to take for granted. It's easy to say- 'oh yeah, I know how to critique'. Or, 'we do that all the time'. It's easy to overlook how vital it is, many of us have had negative experiences with it, whether it's through like art school critique or a company meeting that went off the rails. But. it really is a vital aspect of how we work, for this analysis that we do- whether we're doing it on our own in our own heads, or doing it verbally with other people. It really is critical and paying attention to how we do it together in formal settings, in informal settings, off the cuff- it's really important to the maturity of our practices and how we work together and with our partners. While it might not look like the next big- whatever buzzword you want to throw out there- it is one of those core things and it's really not to be overlooked.

Dan Berlin: 25:10

Absolutely. It's an acquired skill- not an innate skill. It's something that needs to be honed and practiced. And it's very important to UXers to do that.

Adam Connor: 25:22

Yeah, absolutely.

Dan Berlin: 25:24

So in our final section here, we love to get a tip- a career tip. Do you have a tip for people either breaking into UX or who have been practicing it for a while?

Adam Connor: 25:37

I would say my biggest tip- I usually give this to any kind of students- I've done a lot of teaching over the years. Don't get focused on UX too much. UX is great. But, there is so much to learn from other creative practices. Go study how musicians write music, go study how chefs prepare menus, go study how filmmakers make films. There is so much in those other disciplines that you will find that looks uncannily similar to what we do. But, at the same time, offers all sorts of new insights and understandings and perspectives that can bring new life, new energy, new ideas to what we do in UX. So, do what you can to get to those other events, those other conferences, those other areas and learn from those and bring those back.

Dan Berlin: 26:41

That's a great point, I love that. There's so much creativity out there, and we only have our UX view. But, if we learn how other people are doing that creation, and even that critiquing during that creation, makes our practice even better.

Adam Connor: 26:56

Yep, absolutely.

Dan Berlin: 26:58

Well Adam, this has been a wonderful conversation. Thank you so much for joining me today.

Adam Connor: 27:02

Thanks for having me. I really enjoyed this.

Dan Berlin: 27:05

My guest today has been Adam Connor, who wrote the chapter 'Remember the Four Questions of Critique'. You've been listening to the 97 UX Things podcast. Thanks for listening. You've been listening to the 97 UX Things podcast, companion to the book '97 Things Every UX Practitioner Should Know' published by O'Reilly and available at your local bookshop. All book royalties go to UX nonprofits as well any funds raised by this podcast. The theme music is 'Moisturize the situation' by Consider the Source and I'm your host and book editor, Dan Berlin. Please remember to find the needs in your community and fill them with your best work. Thanks for listening.