Measuring Execution: Product Metrics, Part 2

In the previous stand-up we discussed setting a baseline around product metrics. This baseline simply maps out the current metrics that are gathered by the team, and begins to understand whether they are useful (not a vanity metric) and whether they are leading or lagging indicators.

After a current baseline has been found, metrics can be mapped to stages of the user’s journey. While not every product is one that results in a sale, the below chart can be used to understand what types of metrics can exist at each stage.

These example metrics are not exhaustive, and many more can be tailored to a specific product. Ideally, each product team can report on metrics from each of the user journey phases.

North Star Metrics
Many product teams subscribe to the idea of a North Star Metric. This metric is a single measurement that best captures the core value your product delivers to the user. The focus of a North Star is on retainable long-term user growth and satisfaction. This metric would be in your elevator-pitch about the success of your product.

Many successful product companies use a North Star Metric to keep their teams focused on their core value. For example:

• Google - clicking a search result

• AirBnB - nights booked

• Facebook - daily active users

• WhatsApp - number of messages a user sends

• Salesforce - average records created per account

• Slack - number of paid teams

How to figure out your North Star Metric

• something that indicates your user experienced the core value of your product (define your user's success moment)

• reflects the user's engagement and activity level

• something you have control over / can affect

• easily understood and communicated

• can be tied to product success / company success (aligned to your vision)

North Star metrics should not be swapped out frequently. They should meet the criteria above and then be given long enough to prove useful to measuring long term success.

---

STAND-UP EXERCISE

Ask your product team to map their baseline metrics to the user journey using the chart above. Is there one metric that stands out as the single best indicator of long-term value of your product? Can one of these metrics be your North Star Metric - both aligned to your vision and tied to company/product success?

Develop your North Star Metric and begin to watch it as a team. Is it something you have control over as you experiment and ship? If so, begin reporting on it to your stakeholders as your North Star, and holding yourself accountable to its outcome.

Measuring Execution: Product Metrics, Part 1

To consistently drive scalable and sustainable growth for your product, you are likely going to need to understand a set of useful metrics around your product or platform. At their core, product metrics are indicators that show how users interact with a product. But there are several types of metrics, and varying levels of utility.

There are two primary groups to metrics, leading and lagging indicators.

Leading indicators - tells you where your business is headed

• drive daily tactics

• measure frequently (and easily)

Lagging indicators - tells you if your actions were successful

• drives long-term strategy

• measure at a longer time interval (quarterly / annually)

Neither of these is “better”, they are simply used for different purposes on a different cadence. Your product team should be tracking metrics in both categories.

There are some metrics that are bad, however. These fall into two categories - vanity metrics, and metrics without context.

Vanity Metrics

• look good but don't measure meaningful results

• aren't actionable or controllable in a repeated way

• page views / "likes" / number of email subscribers

Metrics without context

• Often, running totals

• For example, "10,000 registered users" sounds good, but not if there are only "100 active monthly users"

To understand if you are working with vanity metrics, use this helpful worksheet from Amplitude.

Understanding the categories of metrics can help set a standard on gathering information about your product’s performance. Use the stand-up exercise below to help set that baseline.

---

STAND-UP EXERCISE

After reviewing definitions of leading / lagging indicators and understanding what types of metrics are bad, make a list of measurements the product team is currently using. Which category do these fall into? Are any of them vanity metrics or lacking necessary context to define success? Which of these measurements is used only internally to the team and which are shared out to stakeholders? How often have these measurements been used to inform a decision on the product?

Once a baseline is in place, the team can dive into making sure metrics correspond to each part of the user journey, as well as determining a primary North Star metric. More of that to come in part two.

Image by Freepik

Identifying Stakeholders

A key part of product management is managing stakeholders, as most teams require participation, guidance, and approval from a wide range of people across the organization. But oftentimes, product managers treat all stakeholders equally in terms of focus or time expended. A key component to effective stakeholder management is identifying your various stakeholders and grouping them by need. Having this knowledge will help your product team communicate effectively with these groups, and therefore gain early alignment on goals and plans, as well as address any conflict or risk early on.

Often stakeholders can be grouped by their levels of power and interest. A simple two by two can map these out - resulting in four groups: Players, Context Setters, Subjects, and Crowd. (This matrix was popularized by the book Making Strategy: Mapping out Strategic Success.)

The needs of each of these groups are different.

Players
High Interest, High Power
- need to be managed closely
- need high-quality data/insights regularly
- get buy-in on big decisions early
- ask for feedback often

Context Setters
High Power, Low Interest
- need to be kept satisfied
- they can influence the future overall context
- raise awareness with them
- could convert them to players?

Subjects
High Interest, Low Power
- need to be kept informed, "read only" stakeholders
- make use of their interest through low-risk areas of involvement
- "goodwill ambassadors"

Crowd
Low Interest, Low Power
- not worth time to actively manage
- inform via general communications
- aim to move into Subjects

How you interact with these groups in form of the cadence, information provided, and size of audience will all vary. But it is important to keep these general needs in mind, as the more you can tailor communication to gain support or approval from various stakeholders, the more likely your initiatives are to succeed.

---

STAND-UP EXERCISE

Have your team do a stakeholder analysis by first listing all the groups (or individuals) they know to be stakeholders for your product. Then work to sort these folks into the 2x2 matrix, paying attention to both the level of power and interest. After their needs are identified, the product manager and team can begin to create tailored communication plans, focusing on building and maintaining trust with each of the groups.

Types of Innovators

The job of a product team is to solve problems for its users and the business. We might not always think of it in this term, but that problem solving is innovation. Some problems require higher levels of innovation than others, but always that should be a goal of our product teams. A Harvard Business Review article explores the roles we take on during the innovation process, “to identify where specific people are needed and who should work together to generate new breakthrough ideas.”

HBR identified four innovation styles: Generator, Conceptualizer, Optimizer, Implementer

The research found that generally, these innovation styles are not evenly distributed. The most people were found to be implementers, folks who take action to put solutions into work. The least amount of people were found to be generators, those who find new problems and ideate based on their own experiences.

This is problematic for two reasons. One, the product teams you are working on are likely lack a balance of innovation styles to be insufficiently cognitively diverse. To solve the most wicked problems that rise on your product, you’ll need support from all four of the innovation arenas. And two, the style you are most likely missing is generators. If we aren’t identifying the problems, are we best serving our users?

Bringing this framework to your product team can help the team understand where they might have gaps, and which innovation styles they might need to lean into.

---

STAND-UP EXERCISE

After they learn about the innovator types (have them read the article, or walk them through the above graphic), ask your team which innovation style they lean toward. Is the distribution the same as the general population? Which areas are missing?

Ask them to think about how to encourage an even distribution of styles on the team, as well as how to create more generators within their team and the organization. Remember that innovation styles are cognitive state, not fixed personalities. How best can you use identifying these styles to create more true innovation on your team? How can you use this to smooth the pain points in your problem solving processes?

Exploring Polarity and Strategic Tension

A topic we often dive into as a product team is competing initiatives, resources, or stakeholders. We attempt to remedy through prioritization frameworks, gaining further insights, and negotiation. However, some tension, some competition in these realms, is good. Tension keeps your rope taut, able to understand the push and pull of what you are tethered to, keeping you aware of your surroundings and its forces.

If we think about our goals or initiatives in the same way, we can explore the opposing forces pushing or pulling the organization or team in several ways at the same time. Product teams that address just one of the poles in a tension are apt to miss opportunities and fail to deal with threats. Looking for the tension between opposing forces broadens the search for strategic responses and increases the prospects of taking appropriate action.

A few weeks ago we explored polarity through contradictory users. The exercise below broadens our exploration into several different forms of polarity or tension, in terms of short term and long term initiative planning.

---

STAND-UP EXERCISE

John Cutler wrote a list of prompts for exploring tension as a part of the annual planning process. Distill the list into a handful you believe your team will find most valuable. Add them to a white board or virtual Miro and ask each team member to choose one fill-in-the-blank prompt. Though reticent at first, my team quickly began filling in multiple prompts. Discuss what the team came up with.
And remember, strategic tensions are dynamic - they can change as the strategy is executed or the initiative develops. Revisit these statements to edit or add as needed.