It seems hard to take a ‘virtual conference’ as seriously as one where you’ve had to travel somewhere, wear a name tag, and drink bad coffee with strangers. But when we heard about The Product Management + User Experience virtual conference (which took place on February 3, 2016) we decided that the quality of the presentations is what really matters. The presenters lined up were all quite well-known figures so it seemed a good opportunity to hear some new ideas, especially since recordings of each seminar would be provided to watch at our leisure.
Jeff Gothelf
The first seminar I watched was by Jeff Gothelf, formerly of Neo and now an independent consultant, co author of "Lean UX" and the forthcoming book with Josh Seiden, “Sense and Respond”.
This seemed to be partly a semantic argument around the correct use of the word ‘strategy’. According to Jeff, strategy is about vision and should be comprehensive, not discipline-specific. His basic point was that UX needs to be well-integrated on the decision-making table, and used as a set of tactics to deliver the strategy of the business, not have a strategy of its own.
Fine, but this seems like an argument in search of a problem. If UX is breaking away and following its own strategic direction, rather than that of the business, then of course that sounds like a problem, but I’m not sure how common it is. As much as I agree with the sentiment, Jeff didn't really make me feel like he was onto something particularly interesting or important in the real world.
Christina Wodtke
Another speaker, however, helped to crystallise the power of having a common strategy for me. Christina Wodtke spoke about OKRs - and not specifically in regards to UX or PM roles. Her approach was quite simple, laying out the idea of the OKR in its standard form, except that she insists there should only be ONE OBJECTIVE for EVERYONE in the team. This chimes with the comprehensive strategy argument of Jeff Gothelf.
For Christina OKRs should unite the efforts within the company on one big thing. They put a timeframe onto the most important stuff, adding urgency so that everyone is working to get the key results done within a certain period - Wodtke says 3 months. Under the objective you should have 1, 2 or 3 key results that will be tracked to measure success at the goal:
- Only measure what you absolutely have to achieve in order to get to the goal.
- But don’t just pick things to attack, things to protect (e.g.: staff health) are also important (i.e.: cannot achieve at any cost).
- Check-in every week and discuss the measures - on or off track, things to do to achieve them over the next week.
- Fridays should offer a chance for bragging and celebrations each week.
At the end of the 3 month period review each measure and see how many you achieved. Failing several is fine, says Christina. It’s ok because they should have been a stretch after all. But then ask what you have learned and can do differently to achieve them in the next period.
As a plan for running OKRs I think this sounds pretty good. But one of the problems she didn’t address is the longevity of the approach. Like any motivational/strategic process, people can get bored and less motivated after a while, as soon as it becomes repetitive. And with a 3 monthly cycle, that could happen quite fast.
Jeff Patton
Jeff spoke at length about the process of “discovery and delivery”. According to Jeff, it’s not your job to make software, it’s your job to change the world (or your company’s little bit of it).
He emphasises that we should measure outcome in changed behaviour, not output in releases over time. Some of what we release has impact that really shows. Some is a disaster and need to be reversed or ignored. But most are just meh, delivering a tiny bit of extra impact. And Jeff says that really good PMs get it right about 30% of the time, with high-impact releases. 60% or so hit the floor and 10% must be reversed/removed.
As for the “discovery” work, there are always too many ideas, because everyone has them. A PM’s job is to make good choices of what to invest in, for maximum impact. PMs need to do things that are at the intersection of Valuable (makes money), Usable (desirable) and Feasible (viable to build). UX sits at the Usable side of the triangle and Engineer at the Feasible side, with the PM these people make up the triad of the Product Team.
Discovery starts with listening and learning, statement of problems, solutions, hypothesis - MVP to test them. Validating hypotheses is crucial. Here’s Jeff’s not telling us anything new, although he likes the idea of two-track continuous discovery and delivery. Ideally people engaged in one will also do the other. For example he encourages us to get the ‘triad’ product team out to interview people, watch users in real world. Sound advice.
Laura Klein
Planning Your User’s Path Together: Laura makes the point that working together on the same goal can result in design by committee: best if cross-functional teams work together but “not too together”, she says. Her presentation was dominated by an exercise she recommends teams go through together to question each step of the customer/visitor funnel and explain why people drop off at each stage:
- reflect individually for 2mins (not together)
- how do people 1. hear about your product, 2. learn about it, 3. become an active visitor, 4. convert, 5. how will you make money, 6. how will you predict users will come back?
This exercise brings out new ideas but also gets everyone to share in every problem along the flow of users. It works as a teaching exercise to ensure everyone understands the whole journey and the issues the business has to overcome together. Laura advises us to put all the ideas up as sticky-notes on the funnel board. This gives a high level customer journey map, and builds a shared understanding of success. It does not mean we then solve everything together - different people will go on to tackle different issues/ideas that are brought up. But with the shared understanding that the exercise offers there should be better alignment on the shared goals of traffic and conversion.
Marty Cagan
Good to Great: Product and Design Collaboration in High Performance Teams Marty reflected on a long list of don’t dos for Product teams, things that he sees as being key in holding good back teams from becoming great ones. Here it is - Problems preventing teams from being great:
- PMs not actually doing right jobs. PM must bring strategy to the team, and motivate and inspire. Build commitment to the problems that you are trying to solve.
- Too many PMs try to be designers.
- Too many fall into project management activities instead of product management - figuring out what is needed to make a great product/
- Confusing output with outcome - same point made by Jeff Patton
- Hiding behind process - do not get religious about process. It’s about culture and outcome.
- Just wanting someone to tell them what to design. It is a big part of the job to figure out what the product would be and designers need to be doing this.
- Thinking job is all about usability. Usability is the easiest part of the Usable/Valuable/Feasible triangle. Need brains of the designer to get the valuable side. Designers must work on this with PM.
- Not appreciating importance of integration design or visual design. Need this to be centre-stage.
- PMs or UX designers thinking it’s their job to protect the user. Everyone must do this.
- Spending too much time documenting every use case to constrain the FE devs
- Dedicated user researchers - discoveries must be in front of PMs to get used. Don’t do research that PMs are not onboard with.
Although the points made are sensible and could be helpful, I found his talk pretty uninspiring. A list like this is difficult to learn from unless it’s made concrete, and reflected on regularly. Maybe it could be stuck up on the office wall - but it seems too long and detailed for that.
Tomer Sharon
This final seminar from the day very specifically addressed issues of UXers and PMs working together. Tomer made the point that PMs and UXers have identical goals and users, so must work together. His presentation boiled down to 3 sets of advice:
UX researchers:
- be there early to shape the roadmap with the PM
- listen to PMs to identify knowledge gaps
- include PMs - let PMs do research if they want, let them talk to users, even moderate sessions with users
PMs:
- UXers can make you heroes through excellent insights, and through recognising and challenging assumptions
- Avoid surveys - they are prone to mislead. Get help from UX to find alternatives that give stronger results
- Make customer meetings happen and include the UXers
How to convince teams to do or listen to user research:
- You need empathy towards any stakeholders - from customers to PMs
- Find engaging ways to report the results
- Changes will happen if you are persistent and support what stakeholder need
There are a number of good points here, and it’s reassuring to see how many ideas align with the other presenters’ adivce on how teams should work together.
Overall I found a number of useful ideas to take away from this virtual conference. Tomer Sharon gave the most specific advice about how product management and user experience researchers should work together, which was useful. I liked the ideas around focussing the entire team on a single objective, sharing understanding by doing exercises together to build common understanding. I also agreed with Jeff Patton’s picture of how discovery - even when well done - results in only about 30% of releases having a high impact.
So despite being a virtual conference, I probably got as much out of this day of seminars as a real one. And having the recordings to watch and reflect on gave me more opportunities to learn from the presentations. With better coffee too.
Paul Sharp
Senior Product Manager at FOODit