As technologists, one of the hardest things to remember is that we’re not our users and our assumptions can easily extend into our work. If we aren’t careful, we can easily end up designing products based on our assumptions and biases rather than insights from the actual audience. If we want to build better products, we need to include our target audience in the creation process and listen to their feedback every step of the way. In this talk, I’ll share what my team and I learned from a project where we worked directly with teens experiencing various forms of bullying.
Key takeaways from the session:
You’ll learn what types of biases to watch out for, how to challenge your own assumptions, how to engage hard-to-reach audiences, and how the user-driven insights we gained uncovered the struggles unique to their community and informed our design decisions and product iterations. Whether or not you work in UX design, this talk will help you create solutions that will ultimately be valuable for all humans.
About the speakers
Ariba is an innovation, design and UX strategist who builds inclusive and ethical products, empowers teams and facilitates dialogues where curiosity and experimentation can flourish. As a woman of color thriving with hearing loss, advocating for marginalized voices and communities means a great deal to her. She believes in designing for inclusion, prioritizing accessibility, building with an ethical code and being mindful of whose experiences we are excluding.
She is the Director of Innovation at the Ad Council, where she is focused on scaling design thinking and agile practices, keeping an eye on emerging tech trends, creating digital products for social good and nurturing a culture of innovation.
In her spare time, she teaches UX Design to NYC high school students through the NYOT Program. She is a member of Women in Innovation, IXDA, Within, Global CFP day and Shine Bootcamp. She enjoys lifting, knitting and exploring the outdoors.
Hi, everyone. Thank you so much for tuning into my talk today. I wish we were under different circumstances and could have been together in person. I am however, looking forward to connecting with you and chatting during my ama on slack at 5pm. Eastern. I really appreciate all the work that the organizers have done to make this conference virtual, allowing us to all connect and learn from each other while social distancing. My name is Ariba Jahan, my pronouns are she, her and hers. I’m the director of Innovation at the Ad Council. And I’m here to talk to you about how we can build better products by engaging hard to reach users. So I’m going to give you a minute to read through these headlines that are capturing what’s been happening in our industry over the last few years.
So, so this is what we get when we don’t challenge our assumptions. And today, I’m here to share with you four ways you can challenge your assumptions so that you can afford headlines like those, and instead build better and more impactful products for your audience. And the four methods I’ll share are helping us think about product development differently at the Ad Council. And I’m hoping they’ll be helpful to you in your own work. And just to give you some context on the Ad Council, we are a nonprofit organization in America that uses the power of communication to tackle some of the country’s toughest issues, from preventing wildfires to getting people out there to vote diversity And inclusion, safe gun storage and promoting girls in STEM. We work with industry leaders to develop campaigns that inspire action and create change. Our campaigns include everything from public service advertising that you may see on TV, or a bus salsa, or while streaming shows on your favorite streaming platforms, to digital experiences and social media activations. As the director of innovation at the Ad Council, my role is to facilitate innovative thinking within our teams. nurture a culture where curiosity and experimentation can drive and equip our teams to use a more human centered approach to our work. I get to do that in many ways, including training our staff on splash box, which is our approach to innovation. It consists of methodologies that you’re all familiar with, from frameworks like Lean UX Lean Startup agile And design thinking, with a strong emphasis on mindsets and team conditions to cultivate. Today we’ve trained 80% of the organization, which really gives us all a shared language approach and toolset for innovation. Part of making the learnings from the training tangible we created a project where our new innovators can apply the frameworks they learned to create a digital product for one of our campaign issues. And today, I’m excited to walk you through one of the digital prototype projects, where our assumptions definitely got in the way and show you how we use the four methods to challenge and uncover those assumptions. The campaign issue we worked on was teen bullying prevention. And the goal of this project was to leverage technology to answer this question, how might we influence teens to reduce pyramid treatment and empower one another by building and perpetuating a more empathetic supportive and inclusive culture. Our target audience for teens between the ages 13 to 17 years old, who also fell into the category of good eggs. We defined good ads as teams that believe they’re good people. You know, they value treating others with respect and kindness and they want to be the best versions of themselves. As you can imagine, we were pretty excited to contribute towards creating a more inclusive world for our next generation. For me, personally, this project really resonated because I’ve experienced bullying firsthand. My family and I emigrated to New York when I was seven years old. And I quickly learned that my skin color made me different enough to be made fun of and the clothes we could actually afford, wasn’t that Cool. Then once I was diagnosed with profound hearing loss wearing hearing aids did not make me popular in school. And instead my classmates would steal my hearing aids as a joke. And let me tell you, that was not funny. I mean, what hurt even more was when my middle school guidance counselor advised me to not pursue my dream. I wanted to go to New York City specialized Science High School to pursue my dream in becoming an engineer. She didn’t think I was capable and advised me to not take the entry exam. I still wanting to take the test, but I didn’t want anyone to know and I didn’t want to get in trouble. So the night before the exam, I forced my mom’s signature on the permission slip to take the test. It was the scariest thing I had ever done. I’m happy to say I passed. I got into Brooklyn Tech High School. would set me on a path to study biomechanical engineering at Syracuse University, and, you know, to being here with you. But see the experience with my classmates. And my guidance counselor taught me that the things that make me unique are the same things that lead others to treat me differently, to make assumptions about my capability, and to decide what opportunities I should have access to. So at a really young age, I learned how lonely confusing defeating and mentally draining it is to be bullied and to be treated differently. And as an adult, it makes me realize two things one, but none of that goes away with age, you know, I still experience being bullied and being treated differently. It just all looks different, and two how much a childhood experience can impact someone’s entire life because none of them That one away, they all stayed with me. So knowing how impactful this project could be, we dove in and did a tremendous amount of research, from reading through reports and articles on Gen Z’s to interviewing teachers, issue experts and teens to really familiarize ourselves with the audience and the issue. Our research gave us insightful findings, which helped us generate over 100 ideas and 20 concept designs. We then showed our strongest ideas to teens to get their initial feedback on which ideas to refine how to refine them and which ones to just leave behind. After many rounds of research and early user feedback, we felt pretty confident in our direction. We knew that our solution needed to address these three goals. One, we need to help teens successfully Recognize the impact of their action. Our campaign research told us that good ads feel a sense of responsibility to create the change they want to see in the world. So we wanted to leverage that, and help to successfully see the impact of their actions so that they can choose the kind of behaviors to we should double down on social media.
Knowing that 91% of Gen Z were on at least one social media platform, we knew we had to prioritize and emphasize the role social media would play in our solution. Three, leverage the rise of CGI influences at the time, for fictional characters, created by computer generated imagery had accumulated over 1 million followers on Instagram. We wanted to use this new social media trends can engage teens. Whether that was through storytelling or Modeling kindness. And in fact, if you look up little Michaela, the CGI influencer on the left, you see that she has 2.2 million followers on Instagram now and over 500,000 followers on Tech Talk. Again, this is a fictional character, and her most recent Tech Talk video was played over 11 million times. So keeping those three goals in mind, we refined our 20 concepts into five promising ideas that we then shared with teens for more early feedback. And the main thing we learned from their feedback on these five ideas was that we were wrong on all fronts. What we’ve focused on as goals or actually assumptions starting with the word bullying, The only people that use the word bullying are adults who are trying to end bullying. See, when we asked teens to recall moments of being bullied, they didn’t have much to share, and instead claim that they don’t experience bullying. But when we changed our language and asked whether anyone has ever heard them, or if they’ve ever heard someone else, suddenly they started sharing stories that involve rumors, drama, shaming and being excluded. Their language around mistreatment was different. We also learned that they’re actually pretty self aware. When we ask them to recall a moment when they hurt someone. They use words like guilt, bad, upset and regret to describe how they felt afterwards. So they knew the impact of their action and they didn’t need us to show show that to them. We also learned that they care a lot more about privacy and authenticity than we realized to the point where they have two different Instagram accounts. One is called Finster, which stands for a fake Instagram account where they post unfiltered content, and only allow a few people to follow them. And then there’s a second one called rinsta, the real Instagram account, which they keep hyper curated with perfect filters and crafted captions. And this is where they want more followers to, to help. And I think this behavior may have influenced the new Instagram feature called friends, where users are able to select which content they want to share with all their followers, and which content they want to share with a select group of people only. And when it came to CGI influencers, I mean, only you few teams even knew what a CGI influencer was. And those that followed one did it only out of curiosity for the characters fashion. And if you look at their handwritten feedback on the screen, you’ll notice what they wanted instead is someone relatable. Someone who may have had a tough childhood, or has gone through issues. And turns out they wanted someone funny. Really, they just wanted Kevin Hart. So as you can see, our assumptions were definitely checked. Before we jump into the four tactics that helped us uncover these assumptions, want to talk about assumptions for a moment. So an assumption is a belief we accept to be true without any evidence. And the reason it’s so easy for assumptions and biases to creep up even after a tremendous amount of research, is because our brain is constantly making assumptions in the interest of efficiency to The thing is, as long as humans have roamed the earth, our brains have created several mental shortcuts to help our species survive and process the world around us. So we have mental shortcuts that help us make sense of large amounts of information, shortcuts that help us determine what’s important to remember or recall, shortcuts that help us create explanations or meanings when there’s not enough context, and shortcuts that help us act quickly. You know, this is how we remember what’s two plus two. Remember how, how to ride a bike after we’ve learned it years and years ago, help us detect what frustration enjoy might sound like in someone’s voice, and know when to run from perceived danger. But these same shortcuts lead us to make snap judgments. Mr. And facts, jump to conclusions and make decisions based on our biases. These mental shortcuts are also Known as cognitive biases, and with 180 mental shortcuts active in our brain, it’s hard to spot when our own biases and assumptions are getting in the way. I’ll call out three that caught us off guard. We have a mental shortcut that leads us to be easily influenced by what is stored in our memory or what’s familiar. That’s why we keep using the word bullying to describe this interaction instead of checking whether this language resonates and is relevant with our audience. In fact, our early discovery research showed that bullying messaging and content doesn’t resonate with kids and teens. But somehow we were still using bullying in our language in our interviews, and that’s because that terminology is familiar to all of us. But once we saw a pattern in the responses we were getting We actively started changing our language which caused us to get different answers.
Another mental shortcut causes us to make judgments and impressions by relying only on the information made available to us. So when we see numbers as high as 91% of teens are on social media, and the fact that they spend nine hours of their day on social media, we easily place a lot of value on this number on numbers like this, and start making assumptions like they must be sharing every moment of their life on social media, we don’t readily question what information might be missing, like, does that entire group of 91% do all the same things to the same exact extent? And so what they’re on it? Why does that matter? And who’s not captured in this 91%? The third mental shortcut, I’ll call it out is the one that makes us drawn to details that confirm our own existing beliefs better known as confirmation bias. We were so excited we were so excited about leveraging the CGI mega trend that we paid closer attention to headlines that claim CGI influencers are the future of influencer marketing. Leaving us surprised when teens were not sharing our level of enthusiasm about CGI influencers. So that’s how easy it is for our mental shortcuts to come into our work and get in the way of making products that are right for our audience. And look, our mental shortcuts are neither good nor bad. They function on a spectrum. On one end, they can help us cope and survive in the world. And then the other end where they can cause us to miss critical information or lead with our assumptions. We just don’t get to turn them off.
So for the rest of my talk, I’ll focus on how we use these four methods in the team bullying prevention prototype project. And I hope it gives you inspiration on how to bring some of these methods back into your work. The first one being map assumptions. It’s an activity where we actively uncover critical uncertainties, unknown and blind spots about our audience challenge and solutions for exploring. See, the thing is, the thing about assumptions is that they don’t always look or feel like assumptions. They feel like facts, as you saw in my examples, so it’s important to have a diverse team in the room so that everyone can help catch each other’s biases and assumptions. So for our project, we started by asking ourselves questions around teams and their experience with Miss treatments. How does it show up and how do they resolve it? How do they How do they cope with it when the master assumption We quickly realized that despite the research we did we already did, we still didn’t know that much. And to get a sense of what knowledge gaps to prioritize Next, we placed them on a two by two grid like this, where the x axis ranges from known to unknown. And the y axis ranges from very important to not that important. This helped us tease out the critical unknowns we needed to dig into next, including the biggest question of all is bullying the challenge teams need solved. We then conducted user interviews to learn more about these critical unknowns to get signals on which direction to go. Some of our interviews were more open and conversational and others were with a sketch or a prototype. And having different teammates to interviews and going through multiple rounds of interviews really helped us make sense and gather insights. Around teams mindsets, language, behaviors, rituals, relationships, and the many different contexts they have to balance and exist in. And those insights helped us refine our hundred ideas into 20 informed concepts. But even at this point, we didn’t really know which idea would work. So instead of just choosing one idea and hoping it works, we decided to test five of our strongest ideas.
We we ran several experiments to explore whether or not our ideas could create the desired outcome and to collect new information. While mapping assumptions was a great place to start, it’s testing our hypotheses that allows us to uncover the invisible assumptions that we haven’t even noticed yet. So to test our ideas, we did three things. One, we converted our ideas into hypothesis statements to Get specific on what outcomes we believe our ideas will have, which helps us hone in on what we hope to learn from testing them, too. We identified our risk assumptions, which were the beliefs we held, about our ideas that if disproven would break our idea all together. And then lastly, we identified how we plan to test these ideas and assumptions and how we would gather signals. We converted our ideas into hypothesis statements by using a Madlib framework like the one you see on the screen where you fill in the parentheses with what’s specific to the project. It helps us focus in on the challenge to solve the solution and what the impact might be. I’ll demonstrate this using our CGI influencer idea as an example. So for the chat, for the hypothesis around the challenge We believe that teens have a problem, noticing the impact of their actions which leads them to mistreat each other. hypothesis around the solution is that we believe that a fictional CGI correct character whose life captures the traumatic realities of the teen experience, including teachable moments where the character models kindness can help. And the last one being, we believe that teens will follow and relate to the character, causing them to reflect and adopt the model to positive behaviors in real life. Then try identify our riskiest assumption. We started by asking the question, well, what must be true in order for this idea to add value to our users for it to be accessed and used for this to even work? When we look through our answers to identify the ones that if disproven, we could break our idea altogether? What stood out to me the riskiest assumption for this idea was our belief that teens will resonate with a CGI character. See, if teens don’t connect without a fictional character, then this whole idea falls apart. But that’s the great thing about testing with our users this early in the project, we’ll get to know that before spending any time and effort trying to learn how to create a CGI influencer. When we get this for each of our ideas, we ended up with a list of elements and interactions that we were assuming teens would love and engage with, but had no evidence yet. So they needed to be tested before we could move forward. Like, does anonymity matter to teens? Do they value influencers? Would they respond honestly to question prompts. The third part of experiment design is identifying how you’re going to test to get signals or what direction to go in. At this point, we wanted to test our ideas and assumptions without building anything. So we use open ended survey questions, sketches and rough prototypes to get feedback from teams during interviews and user workshops. Testing multiple ideas with teens was very insightful for us. But there is a limitation on how much we can learn from testing this way because users are only reacting to what we are showing them and what and what questions we’re asking them. In. In essence, every method has benefits and limitations, so it’s good to use a few different methods. When validating ideas. For us, we did the testing with prototypes as well as co creations. A co creation is when we engage, work with and empower our users to generate their ideas of their own. We co created with three high school classes to learn how the students perceive bullying, what types of digital interactions they enjoy, to test our own hypotheses and risky assumptions which I just talked about how to create and see what solutions they would come up with on their own.
This is a snapshot of the solutions students came up with when we asked them to design a digital tool that could help others be kinder and more inclusive. We purposely didn’t use the words teen or bullying to prevent influencing them in any direction. We also gathered user feedback on these new ideas. But co creations gave us insights we couldn’t have learned otherwise. You know, we learned about the way they experienced Miss treatments, how they cope or don’t cope with feeling hurt, and in what contexts these moments are showing up in. We also learned about how they use language in technology, their choices, their choice of vocabulary, the interactions they prefer, and how they see technology fitting into their lives. And these moments. The CO creations also uncovered ideas that we couldn’t have brainstormed on her own. For example, our students came up with this brilliant idea which they called in my bag, which is a phrase teens use to indicate if they were feeling sad or emotional. It’s an app that would allow users to airdrop messages to one another or to alert their friends if they were not feeling emotionally available at the moment or just wanted to be left alone. Earlier I mentioned that our user interviews taught us that teens were pretty self aware. But But what they showed us here is that is their own ability and desire to prevent that mistreatment from happening by self regulating Communicating in a mode that feels accessible for them. Students feedback on this idea was very enthusiastic and validating, you know, they used words like it’s fire. Or they said you can tell people around you how you’re actually feeling. Their comments showed that they wanted to be vulnerable and express their emotions, which is contrary to their existing social pressure of acting tough. The CO creation sessions, you know, they validated some hypotheses and they invalidated others. What what we did was we took everything we learned from the interviews, tasks and our co creations, and then turn them into three new co created prototypes. It’s been quite a journey from researching, identifying assumptions and interviewing for insights to coming up with over 100 ideas. testing our hypotheses and co creating solutions with with teams. There were many moments where our ideas were invalidated. And our assumptions were really checked. You know, that’s what learning looks like. And although we made progress, it didn’t come easily. We went out of our way to find a diverse mix of teens to interview and work with a local high school to plan co creation sessions. We needed parents to sign permission slips, coordinate around exam and homework schedules, and be okay if a teen totally flaked on a phone interview. But it was worth the relentless effort. Because the audience we were looking to serve was not in the room. And their perspectives were really crucial. You know, and that could happen even on projects not involving teenagers. Imagine we were working on a grocery delivery app, and our team is part of a specific sociality. On a community allowing us to have options and access that may be different from those in other socio economic communities and even other cities, causing us to not have the perspective and experience from a different socio economic community within the room, leaving us uninformed and just with our own biases. So part of being relentless is also constantly checking to make sure we are still on a path that aligns with meeting our users needs, our products desired outcome and ethical standards, you know, we must ask ourselves questions like these. Are we still solving the challenge that our users need solved? You know what perspectives are not in the room? who haven’t we tested with or heard from But should you know can our audience can our target audience use access, understand, get value from this?
What are the unintended concepts concepts of this existing in our world. What Can someone with mal intention do with us? In our project, even though we ran three co creation sessions that were super informative, that’s just three co creation sessions with New York City High school students. You know, we’re missing the perspectives of all other states. So our next step would be to learn from students from other regions. To empower our teams to be this relentless, we must create a culture where curiosity and innovation and experimentation can drive. And that means we should nurture a team that has less of a monoculture of thoughts and ideas. And more seeking perspectives on like our own and enough psychological safety to flag when we notice that we are in fact in a monoculture where we’re spending less time defending our ideas, but instead spending more time testing multiple hypotheses for diverse representation of the audience, where we’re not constantly focused on finding one perfect idea or solution, but instead can create space for everyone to embrace some ambiguity and stay curious, allowing us the freedom to challenge our assumptions and make sense of our audiences contacts, language, challenges and motivations before building something. And instead of creating reward systems that heightens everyone’s desire for perfection, and language that heightens our fear of failure. Let’s create a culture that’s just open to learning and being wrong. So think about what processes metrics rituals and promotion development milestones you have in place that might be working against us, and how can you reimagine them to cultivate a relentless team So rounding off, I want you to remember that our brain has mental shortcuts that are constantly on and helping us to cope and process the world around us. But because they function across a spectrum, these same shortcuts can also lead us to make assumptions, snap judgments, Miss certain information, and jump to conclusions. And while we all like to believe that we’re not that biased, or we’re not as biased as others, that thought is actually a bias in itself. And this bias blind spot can cause us to let our assumptions go unchecked, affecting our ability to make the right products for our audience. So remember to map your assumptions and unknowns, about the challenge in an audience you’re solving for to uncover beliefs that may get in the way knowledge gaps that still needs to be explored. Test multiple ideas to see if they create the outcomes you’re hoping for. And you know, the simple act of reframing an idea into a hypothesis is a great reminder that regardless of how exciting our ideas may sound, we don’t have any evidence backing it up. yet. testing this way gave us a ton of creative freedom. We knew that we could dream a little bigger, because our tests will signal what to refine and what to leave behind. co create with your audience to empower them to create solutions that you wouldn’t have come up with on your own. And by creating space for our users to join us in the process. We get out of our own way and end up contextualizing and sense making to a greater degree than ever. And be relentless in making sure that the audience you’re serving has been heard. checking to make sure that you’re solving the right challenge and aligned with your ethical standards. Lastly, nurture a team that six perspectives on like their own, focuses on testing stays curious and is just open to learning and being wrong. So instead of fighting the hundred and 80 mental shortcuts that evolution has ingrained in us, let’s use these four methods to uncover and challenge our assumptions so we can make better and more impactful products for our audience. Thank you so much for listening to my talk today. If anyone is looking to pick up a new read, here are some books that I’ve enjoyed that touches topics I’ve covered today. And if you’ve enjoyed my talk, you’ll definitely enjoy Ashley brewers and Jenny Sherry’s talk, so you should definitely check those out.
And like I mentioned, I really wish we could all be together in person, but I am looking forward to connecting with you. So whether that’s during my AMA on slack or just feel free to reach out over slack, twitter or Email would love to hear your experiences around challenging assumptions, testing to evolve ideas and co creating with your audience. I hope that you and your loved ones are staying safe and healthy and chat soon. Bye
Transcribed by https://otter.ai