2014 Main Conference Talk
Most companies consider strong product management to be the “glue” that holds together products as they are being conceived of and built, and most companies treat product management as either a marketing or an engineering activity. But modern startups like Airbnb and large corporations like JetBlue or Starbucks have proven that industry disruption is possible not by focusing on adding features or just improving sales, but instead by focusing on providing deep, meaningful engagement to the people that use the products or services. This engagement is achieved by designing products that seem as though they have a personality, or even a soul. These products feel less like manufactured artifacts and more like good friends.
Design doesn’t refer only to aesthetics or usability, although these are things consumers are most likely to notice or appreciate. Design is both a noun and a verb. It can mean the visual or tactical quality of a product, as well as the process by which products are conceived. Design is a more comprehensive way of thinking about people and human behavior than engineering or marketing. It is a product development process that uses empathy with a community of potential consumers in order to identify problems to solve. Design leverages a certain way of thinking in order to infer solutions to those problems that will have meaningful emotional appeal, and a strong market fit.
About the speaker(s)
He was previously the Vice President of Design at Blackboard, the largest educational software company in the world. He joined Blackboard with the acquisition of MyEdu, a startup focused on helping students succeed in college and get jobs.
Jon has also held the positions of Executive Director of Design Strategy at Thinktiv, a venture accelerator in Austin, Texas, and both Principal Designer and Associate Creative Director at frog design, a global innovation firm. He has been a Professor of Interaction and Industrial Design at the Savannah College of Art and Design, where he was instrumental in building both the Interaction and Industrial Design undergraduate and graduate programs. Jon has also held the role of Director for the Interaction Design Association (IxDA), and Editor-in-Chief of interactions magazine, published by the ACM. He is regularly asked to participate in high-profile conferences and judged design events, including the 2013 Cooper-Hewitt National Design Awards. He has taught at the University of Texas at Austin, the Center for Design Studies of Monterrey, in Mexico, and Malmö University, in Sweden.
Jon Kolko: Good morning. How y’all doing? Thanks for joining me this early on a Sunday morning. I appreciate it.
My name’s Jon Kolko. I’m the founder and director of a school called Austin Center for Design, and up until about three months ago, I was the head of products, innovation, and design at a start-up called “MyEdu,” which was acquired by Blackboard. Now, I’m an executive at Blackboard. Who knew?
I want to talk to you about product and product strategy today. This is a question that gnaws at me, because there are so few of them. Where do great products come from?
I went out on a trail to try to figure this out. I talked to a number of my friends and colleagues who run products that I think most of us would consider to be great. I think one common idea is that vision is where great products come from.
I spoke with Joe Gebbia, who runs products at Airbnb, and he said that, we generally have a North Star that we’re all headed toward, that somebody is setting a vision for where the company should go, and everyone’s going along with that.”
That’s one perspective that I think we hear a lot about when we hear about people like Steve Jobs. Not sure that that’s the only perspective, and in fact, process seems to be directly in contradiction to the idea of vision. I spoke with Alex Rainert, who runs product at Foursquare.
He talked about struggling through this process and trying to understand how to bring an idea to a product, to a start-up, to a company, and turn it into an organization. There’s something more methodical about growing something out of nothing.
I suppose a third perspective is that it’s just hard work. I generally find myself in this camp. I spoke with a friend of mine, Mark Phillip, who’s the CEO of a company called Are You Watching This, and he described how he’s been pushing this rock uphill for close to seven years.
In fact, he described how his credit score was zero, because he was bootstrapping the company. I didn’t know you could get a credit score of zero. Apparently, neither did he. He said he went four years without making a single dollar, and he’s happy to now, obviously, be profitable and be driving revenue.
There’s multiple views of how product comes to be, and I think they’re all pointing at this idea of product management as a discipline, and as an understanding of how to get something great into the world.
These are the folks that I spoke with, and I bet you recognize some of them, and I bet you recognize some of their products, because these are the products that we’ve integrated into our daily lives. They’re things that we use regularly and that I think most of us would consider to be good examples of products to strive for, things we might be proud to make ourselves.
Across all of them, I found this continuity of discussion, that there’s some amount of having a vision. Somebody has a good idea, and then, there’s some amount of driving that good idea through a sometimes-skeptical organization, driving consensus.
Finally, perhaps most importantly, is this idea of actually getting something to the market, shipping product, and the complexities of actually getting the good idea through all of the gates necessary to bring it out into the world, to unleash it on the world.
At the center of that is this notion of product, and I think that’s really fuzzy. It’s like, what is this thing if it’s no longer physical? Most of us are aware of how our worlds are crashing into this idea of product and service.
What does that mean? For me, I think there’s two qualities. I want to talk theoretically about product, but then, I want to talk very, very specifically about how product lived in my world, you’ll get a little bit of theory and a little bit of practice.
On the first side of this is this notion of a product-market fit. You may have heard about this if you’re tracking the idea of start-ups in San Francisco. I don’t actually understand what product-market fit is, and so, I try to figure it out by talking to those folks.
These are some of the things that I learned, and these are some of the things that I’ve experienced over the last 15 months of running product at a start-up.
The first is that there needs to be this idea of a broad technological or sociopolitical fit. Let me explain what I mean by that by means of watching a market leader drive this. Why on earth would Google be rolling fiber-optic cable to my house in Austin, at such a massive expenditure?
The reason is actually really simple, because they need a bigger pipe in order to achieve the type of revenue and growth that they want to achieve. They’re looking around at the infrastructure that exists, at the copper wiring that goes to most of our houses, and going, “This isn’t going to cut it, and so we’re actually going to build the infrastructure to get our products to market.”
Most of us don’t have that luxury, and so this becomes a constraint rather than an enabler. It means that we as designers need to be aware of what that infrastructure is and how we can leverage it, and understanding when it’s going to be appropriate.
All of these great ideas you might have that aren’t really ready yet because that technological infrastructure isn’t there to support them—I think most of us who are in our mid-30s remember back to going through our undergraduate design school and coming up with great ideas and wondering why we never saw them in the market.
The reason is because the infrastructure wasn’t there to support them. Pie-in-the-sky, blue-sky stuff has to get pragmatic at some point. The second piece of this story is social precedent. I don’t see anybody wearing Google Glass. Anybody wearing one of those things? Anybody got a computer on their face?
There’s not a big social precedent for that, and so it feels weird, but it actually feels a little less weird than it would’ve about five years ago, because we’re starting to see a social precedent in the realm of Bluetooth headsets. We started to see people walking around with computers on their face talking to themselves, and that was pretty strange, and now it’s less strange.
I’m not entirely confident that Glass is there yet, but it’s creating baby steps toward all of us coming to terms with that level of strangeness, and again, to drive product-market fit, you have to be at the right time in that adoption cycle.
If you came up with a great thing that was going to go on somebody’s face and help them achieve all sorts of goals, but they would look like a cyborg—we’re not ready for it. I hope we’re not ready for it in 10 years, but I have a feeling we will be.
Again, product-market fit is about understanding and tracking social precedent. It’s also about this idea of an opportunity for engagement. Think about what the word “engagement” actually means. It has a lot to do with attention, and broadly, you’re all giving your attention to me right now.
That’s a pretty amazing commodity, right? I want that stuff. I want that attention. When we think about product-market fit, there has to be room for people to give you their attention en masse. This is half of the story. The other half of the story is this idea of behavioral insight, and this is where we go from thinking communally, thinking really broad and abstractly, to thinking very locally.
Again, three main trends or issues that I see with behavioral insight. The first is this notion of value. I think, for a long time, we thought of value as utility. Did it allow me to do something that I couldn’t previously do? Increasingly, value is emotional.
Does it allow me to feel how I couldn’t previously feel? I think we’ve all talked to death the idea of experiential. Does it allow me to experience something that I couldn’t otherwise experience?
Value is pretty intimate, and it’s about a one-to-one exchange. I’m going to give you my money, time, attention, personal data to mine through and sell as you will, and in exchange I want some value, not necessarily utility.
There’s also this notion of identity, and I think increasingly, for products to have this behavioral-insight connection with people, it has to be tied to who they aspire to be. That means that you need to know who they aspire to be. This is not a focus group, a set of checklists, a series of statistics. This is pretty intimate. We’re talking about people’s hearts and minds and souls.
Third big idea here is that behavioral insights are about provocations. They’re about saying the world could be a certain way if only a certain thing existed. They provoke new behavior, and they change the way that we think about the world. They don’t simply respond to the world.
That’s very abstract. We have this idea of behavioral insight, idea of product-market, and all of this leads to this discipline or world or job or career or thing of product management. I think it’s useful, just quickly, to talk about how it came into our world, because traditionally, product management was an engineering thing, and companies like Google and Microsoft drove product management by adding new features and capability.
What could the technology do, and then how could we productize it? You can juxtapose that to a marketing approach that maybe a J-and-J or Procter and Gamble might take. How many extra blades can we add to a razor?
That’s a joke. Yeah.
Suddenly, we have this world of design-led product management. It’s like this new thing to us. It has a lot to do with, for me, three different trends, right? Three different, I don’t know, movements within culture that have enabled this.
The first, I think we can thank people like Bruce Nussbaum, but broadly, popular journals like “Businessweek,” for tying design to innovation, by directly correlating the idea of making things for people with driving new returns for companies. I suppose that’s good.
We could also blame, if you like, this stuff, because suddenly, what we do is not necessarily understood, but it’s desired in 18-minute increments by some of the most affluent people in the world. They want to know this stuff. They’re going to pay 5, 6, $7,000 to go to TEDGlobal to hear people talk about design. Cool, good for us.
I actually blame more of this stuff. This is where I think design-led product management starts to really, really make a lot of sense, because our world is now overwhelming. The first time you encounter one of these in your bathroom—let me see if my little laser pointer works. You’re like, “What does that do?” Or like, “What does that do?” Totally bizarre, right? The world is strange, and this stuff is starting to be everywhere.
This is not a “oh my God, technology is everywhere” talk, except to substantiate why design-led product management now makes a great deal of sense, and I think companies are starting to appreciate that.
What does that mean? Typically, for us, it has meant focusing on three things—aesthetics, usability, and information architecture, flow, interaction, however you want to frame that.
Most of us have spent our careers trying to get good at these things, and that gives us a very tactical skill set, but there’s a different set of things that happen when you’re talking about driving product strategy. These are things like innovation, positioning, engagement, and value.
These are just like tactical skills, things you can learn, do, apply in your job, and when you do these things and learn them and try them and practice them and do them well, design is strategic. You’ve moved up the value chain.
You can’t have one without the other, and for most of us, we’re sitting in this middle world going, “How do I bridge the gap between the two?” How do I actually take what I’ve honed as craft, the tactical skills that make me good at doing my job, and move up the value chain so that I can actually have a strategic impact? For me, I’ve found success in product management, found that to be the thing between.
Let’s talk about what the qualities of a good product manager might be, to see if maybe this is something for you.
Josh Elman, I think he’s a partner now—yeah, partner at Greylock, which is a VC. He’s had a number of prominent roles in different Internet start-ups that you’ve probably heard of, including Twitter. According to him, you need to be a great storyteller. You need to be able to tell good stories, OK?
We look at ourselves and we go, “We’re good at that. We can tell stories.” Most of the stories we tell are optimistic. We’re telling stories of a future that doesn’t exist yet, but damn, it’d be good if it did. Cool, check that box.
This was my boss at MyEdu, Frank. He had a previous life in EdTech and then at Procter and Gamble. He’s one of those “add the blades to the razor” kind of guy. He said, product manager, “No one has said they want something before. No one has done it before, but I’m looking at it and I think it will have an impact.” He’s starting to assess signals and trust his intuition.
For Frank, the ability to make sense of signals from people in the market, that’s a sign of a good product manager. Again, we’re going like, “Am I good at this stuff?” That’s what designers do. We trend-spot and we look at culture, and we try to understand where things are headed. Cool. I’m two for two. I should have one of those little quizzes that we had the other day. That’s much too technically advanced for me.
Remember Mark, the guy who pushed the rock uphill for seven years and zeroed out his credit score? He said, “To be a good product manager, you should be interested, not interesting.” What I hear from that is be really, really interested in listening to people. That’s what we do, right? That’s qualitative research. That’s our bread and butter. Three for three.
Gary, a friend of mine who I worked with at Trilogy, was the former GM at Union Square Ventures, a VC out of New York, and he said, on the suite of companies that they fund, the good product people are curious. It’s about if you are curious enough. There’s this desire to know everything about everything.
When I think about what designers are trained to do, at least in a consulting engagement, whatever the next product that comes across your desk is, it’s exciting. Tax? Cool. We’re doing banking? Awesome. Light fittings? Cool. Bring it on. Don’t care what it is. Four for four.
Alex, who runs product at Foursquare, said, for better or worse, product is the discipline that sits at the hub of the wheel. It’s a central idea. For me, that means you have to be affable. You have to be likable and be able to drive consensus. Again, we find ourselves in that situation over and over and over.
We have these traits that seem to point to you might be interested or good at or gel with this role, that we’re starting to understand as the hub of a wheel, where all the things come together and allow us to move from tactical to strategic.
We haven’t actually answered the question, what does a product manager actually do? We talked about establish vision, drive consensus, ship product. I want to talk about what a product manager actually does by means of example.
MyEdu was a company that I worked at for about 15 months as head of product innovation and design. We got acquired by Blackboard about three and a half months ago. MyEdu had 14 people in it, Blackboard has 2,000 people in it, and so, I can talk to you offline about what a culture clash that is at some point. For now, let’s talk a little bit about MyEdu, and I’ll use it as an example.
This is MyEdu. This is actually MyEdu sitting on an austere mantel. This is not how it’s intended to be used. MyEdu helps college students succeed in college, tell their story and get a job. You can think of it like LinkedIn for college students, but the problem with LinkedIn for college students is it makes them feel dumb, because they go and they create their profile and it said, “Started college,” and that’s the only thing on the profile.
Our profile looks like this. It always makes you look good, because it has things that you can be proud of even when you’re a freshman. Oh, I joined a club. Oh, I’m starting to learn French. Oh, I’m taking these courses. You can start to tell your story in a highly visual way.
I want to walk through four chunks of what a product manager does, by means of example through MyEdu. Now we’re going to get really tactical and really specific.
Let me start with this idea of research. I think some of this is familiar to you all, because this is how most of us start project engagements. Our goal here is to gain empathy. The way that we did that was spending a lot of time with college students in their natural habitat.
These are college students in their dorm rooms. We would go into their dorms, with their permission, look through their bags, take pictures of their desks, learn about what’s on their iPhones, comment on their fingernails. Can you see how bedazzled that is right there?
One of the students I talked to said that, “Your resume is like your life. It’s your golden ticket to the chocolate factory.” She said that, “I like to put customer service and management and stuff like that on my resume. Everyone has a business degree these days, I’ll always be able to get a good job.
I found out about international business from a guy at the Gap. I didn’t even know what it was. I googled it and it sounded better than regular business, I chose that. My life decisions are based on stupid things.” Samantha, 21-year-old international business major.
I say that not to ridicule Samantha, because actually this is how most college students pick their major. You can maybe go back in time and think about how you picked your major. We heard things during our research like, “I picked accounting, because it was the first one on the list.”
We also do research on the other side of our value proposition, which is with college recruiters. Does anybody recognize what that is? That’s a purple squirrel. That’s like the rest of the world’s version of a unicorn. We want unicorns, and everyone else is looking for the purple squirrel, which is obviously a very, very, very specific candidate with very, very, very specific skills.
We talked to a recruiter, who told us students say, “I could do anything. I think I could do this. I think I could do that.” You couldn’t say something worse to a recruiter. She said, “Don’t apply to five of my jobs, because you’re not going to get any of them.” OK? For a recruiter, they’re looking for that purple squirrel, and Meg was adamant that she’s not hiring somebody who does a buckshot approach to job application.
For us, contextual research kicks off a product cycle, and the way that we do that is actually extraordinarily easy. We go out and try to talk to people and watch them as they work, live, and play. For us, the key is watching real behavior. This is not an interview, as much as we can get it away from an interview. It’s watching them do their job.
What’s the job of a college student? Sometimes, it’s studying. Sometimes, it’s partying. Sometimes, it’s going to class. Sometimes, it’s procrastinating. Try to watch all of that stuff. First big element of what a product manager does is some of this stuff.
The next piece of this is making sense out of it. Because, when we go and talk to Samantha and we talk to Meg, we learned a lot, but we didn’t actually get an indication of what to build. We need to figure that out, and the process we used is synthesis. This probably looks a lot like your all’s war rooms.
There’s nothing really special about it, except the level of rigor that we went to maybe is a little extraordinary or out of the question. Every single one of these is an utterance from a user. Let me see if I can get a little closer. There’s the one from Meg—”Anything. I think I could do this. I think I could do that.” You couldn’t say something worse to a recruiter.
We transcribe all three or four hours of every single one of the inquiries we do with our users. Here’s the secret sauce. We use mail merge to get them—you don’t have to copy and paste. Just throw it in Excel and mail merge. Awesome. Worth the price of admission alone, right?
Then we put them all on the wall, and then we spend about two weeks marinading in the data and moving them around and finding patterns and anomalies and trends and really starting to build a mental understanding of the landscape of what it means to be a college student and what it means to be a recruiter.
The method here, again, is very easy. It’s time-consuming, but it’s easy. You transcribe this stuff, you get it on the wall, and then you live with it and move it around. Again, we probably talked to 20-something students, 20-something recruiters. I might be off by a couple. This probably took us two to three weeks of moving these little suckers around. It’s not a fast process, by any means, but it’s a rigorous process.
What pops out at the end of that are these behavioral insights. For me, this is actually the key to the whole thing.
Let me walk through what that looks like. Samantha said, “Your resume’s like your life. It’s your golden ticket to the chocolate factory.”
Samantha’s emphasizing bullets on a resume. For her, the resume’s key, not a portfolio. For her, she thinks she should have a broad but shallow set of abilities, so that she looks good to as many recruiters as possible. OK? For her, she’ll apply to every single job she sees.
We came up with an insight, after living with that data, that students think they have an idea of what employers want in a candidate, and they’re often wrong. This is not magic. Right? It was inevitable. You spend enough time thinking about the Samanthas of the world and you arrive at an insight.
I’ll give you a version of this over on the recruiting side. Meg said, “Don’t apply to five of my jobs, because you’re not going to get any of them.” She forms an opinion of a candidate in seconds.
She’s scanning for keywords on a resume, but she’s looking for skills and she wants competency of that skill, and she’ll create this fake narrative of what a candidate can and can’t do in her head, and then, she’ll leverage that fake narrative. Not fair, but that’s how she thinks about and assesses candidates.
The insight, for her, is that recruiters make these snap judgments, and that impacts a candidate’s chance of success. Again, it’s inevitable almost to arrive at an insight like that based on this data. You spend enough time marinading around Meg and you’re like, “Wow, this is how she goes about her job.”
This is the good stuff. These are insights. These are provocative statements of truth about human behavior, and we’re going to frame them as universal truths. We’re going to make an abductive leap of logic and say, “This is fact.”
All of a sudden, we just drove a whole bunch of risk into the process. Up until now, we’ve been pretty objective—do the research, synthesize the research. We just made a huge inferential leap, and that’s where innovation’s going to come from.
For us, the way that we got there is we grouped all of those sticky notes, those utterances, and then, we just asked why. We just drilled on the question why. Why are they saying that? Why do they do that? Why do they think that? Why do they perceive that? The trick here is you don’t know. You’re making guesses, but we’re going to answer them anyway. Ask why, and answer the question why.
Now, we can move on to this idea of value proposition. This is where it gets really exciting because, for me, as a competent interaction designer, this was the new stuff. This was the stuff where I was like, “Wow, this is actually what I think we’ve all been talking about. Now, I get to do it in a product role.”
We have our insights about students. We have our insights about recruiters. Let me tell you a story around that student. This is them, and they’re saying, “I don’t actually know how to show specific skills. I don’t even think I have specific skills.”
We would do some research, I remember, around an advertising major who was showing us her work, and like, “Oh, here’s a brochure, and here’s a packaged design, and here’s a go-to-market plan, but I don’t actually have any skills.” It blew my mind. What do you mean you don’t have any skills? You’re showing me that you’re skilled in these things. “Yeah, but this is just schoolwork. These are just projects.”
There’s a divide there, right? Mentally, she doesn’t think she’s skilled yet, how does she show her skills? She thinks it’s important to be viewed as having as broad a set of interests as possible and being open to anything—be really, really flexible.
She also thinks that the entire hiring process is an enigma, don’t really understand what happens in that black box, and the key to having a job is a good resume. This is her world view around getting a job.
Over here on the recruiter side, the recruiter’s saying, “I’m looking to match a really specific skill profile, really specific. I’m looking for that purple squirrel. I want the unicorn. I need to see evidence that you can do certain things.
Prove to me that you’re good at this, that you have that skill or capability or quality. I’m going to build a story about you based on these tiny, tiny details, and if that story’s not in your favor, tough for you. By the way, I’m very busy. I’m very busy.”
There seems like there’s a problem here, and we’re going to paint that problem as a what-if opportunity. What if we help students identify their skills and present them to employers in a credible way? Falls directly from these insights. Re-frame and rephrase that, and you get a value proposition. MyEdu helps students identify their skills and present them to employers in a credible way.
We just made a value promise. If we succeed, we will have achieved our value promise, right? We do this well, we’ll help students, we’ll help recruiters, we’ll have generated value.
The way that we did that was we told a story. We talked through our insights and told a story about the problem state, and we unpacked it a little bit, and then, we provoked what-if questions, to find a way to improve on the problem state that we had identified through research.
Then we framed it as a statement of value. We framed it as an aspirational statement. We don’t yet do this, but we’re saying we do.
Next step there, then, becomes a traditional software development process. This is probably where you’re going, “Yeah, I know how to do this part.” We identified features. If that’s our value proposition, we said, “OK, they got to be able to add skills to their profile.”
Here’s things that normally you would use to find skills, browse for them, search for them. You could find them from your friends. Once you have the skills, you could substantiate it. We could do a LinkedIn-style endorsement. You could associate it with a class, you could associate it with a professor, you could get a professor to recommend you, and then finally, display the thing on your tiles, on that profile that I showed you.
Then we said, “We don’t actually have a lot of time, and so let’s just pick what’s the hero flow through this? What’s the core flow that allows us to get to that value proposition?” If we kept taking stuff out, we wouldn’t actually achieve that value proposition, this is it, and that’s what we built.
From there, it goes through a very traditional, I don’t know what you want to call it, quasi-agile story-based points, JIRA. You’re writing user stories. You’re tracking them in some shitty tool. You’re doing standard design work, right? You’re building comps. Here’s a very early view of what the thing might look like.
You’re doing all the stuff that designers do, but you’re doing it in the context of a value proposition. We wrote the scenarios and flows and then we chunked them, and every single time we make a decision, we’re asking, does this still support the value proposition that we’re driving toward?
The next piece of this was, again, new to me as a product owner, as a product manager, because once you put the thing out in the world, you get data. This is where your mind explodes as a designer.
My developers made this little email that we all get internally every day. I’m starting to understand how my product is actually doing. I can find out, oh, here’s the top schools that are adding skills, top courses linked to skills. I get one of these in email every morning, I have a pulse on what’s happening. I don’t have to go into analytics or ComScore and look stuff up. It’s coming to me.
Then you can do stuff like this, with a CSV file that is included, and you go, “What skills are the students adding?” You find out that they’re adding Microsoft Word and Microsoft Excel and Microsoft PowerPoint, and Microsoft Windows, Microsoft Office. It’s a sad future that we’re looking at, isn’t it?
We’re learning what students are actually doing and what skills they think they have and how our colleges are actually performing. OK? It’s working.
The way that we got here is, prior to launch, we identified what are the metrics we actually want to track. Again, a new idea. Most designers aren’t thinking about this stuff, but we’re going, “Once we get this in the world, how are we going to judge success?” Sometimes these are called KPIs. For us, it was like, “What’s actually our barometer for success?”
We wrote them down, and then we tried to drive that through the development process as we drove the new feature definition. They were intimately tied together.
Then we iterated, and this is your standard designers going crazy. The difference is that we’re doing it in real time, launching a couple times a day, in order to get to this robust, blown-out value proposition. We added the blue ones next. We have this form that we send to students at the end of each quarter to say, “Hey, how are the classes that you took?”
We added to the bottom of it, “What skills did you learn?” Now we have a vehicle to remind students, at the end of each semester, “Hey, don’t forget to go add your skills to your profile.” Now, we’re gaining insight into what students think they’re learning in courses.
We started with a contained list of skills, an authoritative-source list. We allowed students to start to add free-form skills, and we did a, if three or four or five people add the same skill in the same class, we validate it and add it to our authoritative data source.
We added the ability to attach skills to work experiences, projects, and courses, and an easy way to connect those, and then, display them. In this case, I said I’m advanced in design, and I’ve substantiated that with a website creation, a couple jobs that I’ve had.
We learned stuff, too, like that girl that told me she didn’t have any skills? Well, she needed some prompting. OK? We added a series of instructions, and these are subtle things, right? This isn’t a brand-new product launch. These are minor enhancements that take pretty much no time to push live.
We also came up with this idea of doing something a little bit more magical. On the left, when you added a skill, this is what it looked like on your profile. We said, wait a second. If we know that most of the students are saying that they’re learning Microsoft Excel—good for them—let’s do something special when they put things that we know are popular. Like, communication with the big one. We made custom tiles for these. Look at the designers.
They get crazy for a while and come up with something a little more statically pleasing. The way that we got here was we made hypothesis based on usage. We prioritized our design changes based on how we thought they would drive that value proposition. I don’t have a lot of patience for lean as a way of finding innovation, but as a way of refining innovation.
I think it’s a really, really successful approach. Then finally, we arrived at this idea about communication strategy which again for me and my sort of naive, I’m a designer. People would just love the work, because it’s good.
I’ve discounted pretty much my entire career. We need a way to tell students about this. This was our brand child. We said, we frequently, lovingly send emails to our students or spam them to let them know what’s going on.
We said we can send them an email like this. We know that you took a class in general chemistry. John said he learned UVB spectrum [inaudible 0:31:49] whatever it is. We can then say, “Hey, did you learn that, too?”
We put up this mini email campaign. Instead we lovingly send it to 1.4 million of our students. We send a small test. We sent it to about 2,000 and it didn’t work. This is the email I got from the EA who is in charge of understanding a live in that work. 414 people out of 453 people clicked on it. Out of 2,000 emails, 53 clicks. That’s pretty shitty. Again, I have data.
The problem with this type of data is I don’t know why. This seems like a pretty logical thing. It does not work because the images did not load because our data was wrong, because no one actually learned UVB spectrum for photometry and general chemistry lab.
I’ve no idea. As you start to disseminate this new feature to the world, you actually learn that you don’t actually know as much as you think you know. Here’s our hypothesis about why it didn’t work, still not sure. The way that we got here to this communication strategy is we identified a way that we want to let the world know about this new feature or capability or practice. We tried it.
We tried a little test of it to a small but significant population of people to understand if it worked or not. The last piece of this journey is around feedback. I’ll tell you a quick anecdote.
Remember, I’m so proud of my custom tile badges that we made. One of the students wrote in this and send us ticket. He said, “Where can I see a list of skills that have their own dedicated images?”
I get all this, and these tickets and never know that I respond to him. He said, “In most CD like that. I found five on my own. I want to be sure there are more.” This is me.
I said, “Thanks for writing. We like the magic of the surprise but you’ll have to discover the other one.” I gave him a winky face. He rated it bad. He said, “I’m unsatisfied with the feedback I got.”
Jon: I thought I was being that clever with I thought of my magic moment. In this user, he said, “No, just tell me. Tell me what they are.” We did not make him a custom tile, whatever he was happy.
The key here I think is to make sure that you get all communication that touches the end user. In our case, that’s the end of support tickets. There are probably a number of different outlets in your organization that are producing user facing stuff. Get it all mailed to you. Just make the stuff just suck up.
We just sort of talked away through product. We talked about this idea of establishing a vision driving incenses and shipping our products in the middle. Like, “Maybe you’re going cool, I’m in. Sign me up. How do I get started?”
I think it’s actually really, really simple. Joe Gabbia said, “Throw yourself into an experience of making.” Levon, who runs product at kick starter said, “Yup, have some experience of building something.”
Gary said, “Go make something.” Seems pretty straight forward. I think if you make something, focus on product-market fit like we discussed, driven by behavioral insights. You’re actually able to ship something.
You will have a tremendous degree of success in moving to a product management by design role. Thank you very much for your time. I appreciate it.
Jon: I’m perfectly happy to answer questions if you have any. Yes, ma’am?
Audience Member: Did you actually dived back into that last segment where it didn’t work?
Jon: You have the company subsequently gone back to understand it.
Audience Member: Yeah, I know you had a communication with that one student.
Jon: There were two examples at the end. One was related to the tiles and he was just sort of unhappy. We never changed those.
I said, “You know what? I actually think I like the magic more than the fact that he can’t discover them.” I supposed a larger point is, once you learn all of these stuff, at what point do you want to go back and fix it?
That’s actually a frustration that I have within the entire sort of product management process, is that you have a constraint resource of your talent pull. You’re constantly juggling to want to build out the rest of the vision.
Remember, there are still a couple of squares that weren’t filled in? Or, do I want to go back and fix the stuff that isn’t working? We didn’t go back and fix the stuff that wasn’t working. We filled in new squares.
Audience Member: Yeah, another follow up. I worked on a, our product hasn’t been redesigned in many years. We’re spending a lot of time looking at what doesn’t work.
Of course, making decisions about what’s most important to fix. That’s our biggest challenges to go back in and find out why. If you’re not doing that, have fun [laughs] .
Jon: We do it on the things I think that are critical. I think both in those cases, they weren’t critical. Our focus analysis is on integrating this into the larger mother ship. Priorities are sort of drifted.
Audience Member: Thank you.
Jon: Other questions or comments? There’s one at the back. There are two, three. You’re going to get your work out here.
Audience Member: As you talked about product management, it sounded at least to me that the door to that is entrepreneurial. If you’re within a company, do have any advice for those who want to move into that role within the company?
Jon: Yeah, it’s a great question. Those feel entrepreneurial and that was certainly the background that I had here. I’m now been a black board for about four months as a result of the acquisition. They’re thirsty for this.
I think that they’re actually representative of most companies who have established sort of history in software, have a bunch of legacy success and a lot of problems. They’re actually really, really curios to be driven by these empathetic insights rather than by something like a competitive analysis or others going to mark a data.
I think my best suggestion is to leverage the unique parts of the process, which for us is going to be the behavioral insights stuff as your sort of your foot in the door and socialize it with many people as you can. Rather than trying to sort of do what they’ve traditionally done, but better from a user’s perspective due to the stuff that they haven’t traditionally done and really should have shine light on and get visibility to it.
Had a lot, a lot of success in taking all of that research, boiling it down to probably five-page deck and getting it in front of executive.
Audience Member: Hi, this might be an exception rather than norm. Increasingly, I’ve come across people who have been calling themselves product strategists.
Basically, even the discussion gets really deep about research and user experience design. They tend to take a step back and call themselves as ideators. What do I’m missing?
Jon: [laughs] I don’t know. It sounds like you’re talking about that split between those strategic view of the world than a tactical view of the world. I don’t actually think you can do one or the other.
I think the reason why I’m so drawn to product as a, I don’t know what you call it. Just have been a role because you have to do both successfully. Or, you have to enable teams to do both successfully.
It actually has a lot to do with the last question. I think in a lot of organizations that I have dealt with on the consulting side and now I’m in house for the first time in a large company. There’s such a desperate hierarchy split between people who are supposed to be having a strategic vision and people who are actually doing the work that there’s actually a knowledge gap.
It’s pretty well documented in research around how businesses work and are run. I think the ideation, the iteration in the making has to inform the strategy. What I heard most people run into is they don’t have time to do both. I don’t know if I buy that. I think there’s always time to do both. I’m not sure if this actually answers your question. I think that’s in that world of how do you bridge it between strategy and tactic.
Audience Member: I think it is fair for me to expect product strategies to know, at least, be informed by what’s happening in the way it’s…
Jon: I think it’s critical that they actually understand the market, the analyst, and the users. I don’t understand how you do your job if you don’t know how three of those things.
Audience Member: Thank you.
Audience Member: Hi, Jon. I heard you speak a couple of years ago at another conference. I just think you’re awesome, I just thought with that.
Jon: [laughs] Thank you.
Audience Member: I think my question is around working in an agile environment, where you’re used to working with product owners. Then, you come in as resource that use experience resource.
How do you keep from source stepping on the product owner’s toes when you go and take on that entrepreneurial spirit and go out there and see the data and do all those activities?
Jon: I’m not sure, I’ve never really understood the idea of territory. It seems so ancillary that actually getting product shipped and doing your job. I suppose people are aware of it.
My method and it’s not probably a good one has been to strive forward. Usually, that involves around rolling other people. Again, it’s probably not the recommended course of action. I make friends with adults who can do better at socializing and working on relationships stuff. What has worked for me has been to do what’s right to the product and user.
What I found is that, most other constituents see the value of that and don’t feel threatened on their own role, because I’m pretty overt that I don’t care about the role [laughter] . Again, your [inaudible 0:41:12] may vary there. I’m sure there are better ways to do that.
Audience Member: How formal is the process do you have for all these. A lot of these, one leads directly to the other. In the different organizations that I worked with, it’s a very loose amorphous movement forward without a lot of structure.
I heard some companies like Apple internally actually have like, check boxes they through the process. I’m just curios what’s worth for you?
Jon: It is much more amorphous and fuzzy that what was presented here. I’m pretty overt about tracking who really regulates process. Like, or do and this and it ends here. It’s always that easy.
Sometimes, that means applying artificiality that the dates. Like as an example, when you’re in synthesis and you’re moving post notes around on the wall, you do that forever. Very much arbitrarily based on experience.
I’d say we have two weeks for this team go, and then, I’ll track two weeks from there. Not in two weeks I have the answer, but because in two weeks, I’ve ran out of time. I’m pretty process driven.
Every time I’ve used this process, or parts of the process, it works. It works with varying degrees of success depending on the organizational context and the appetite for it. Appetite worked really well here, because I’ve got to do all of those things.
I think if you’re only able to do the research synthesis insight part, there’s enough value and substance there. That is all the rest of the process goes to shit, you’re still going to make forward momentum.
Audience Member: Thanks.
Jon: Thank you very much for your time and attention. I really appreciate you all.