"How would you measure success for this feature?" is asked in almost every product loop, and the failure mode is unusually consistent: candidates answer with a list.
Adoption, engagement, retention, satisfaction, maybe revenue. Five metrics, all defensible, none chosen — which tells the interviewer that if this feature shipped and two of the five went up while three went down, you would have no idea whether it worked.
The round is about choosing, and then about defining the thing you chose precisely enough that someone could build it.
The prompt, worked
"We've just shipped a way to share a saved report with someone outside your company. How would you measure whether it worked?"
Start with what it's for, not with metrics.
"Before picking anything — what was this meant to do? There are two very different versions. If it's a growth feature, the point is that the external person ends up as a user, and I'd measure it as an acquisition channel. If it's a retention feature, the point is that reports get used in more meetings and the account gets stickier. Those need completely different metrics and I'd want to know which one we sold internally.
I'll assume retention, because the feature is in the analytics product rather than the signup flow. Tell me if that's wrong."
That thirty seconds is the highest-scoring part of the answer. Two readings named, one chosen, the assumption made visible.
Then choose one, and say what it can't tell you.
"My primary would be the share of accounts where at least one report is shared externally and then opened again by the same recipient within a month. Not shares sent — shares that got used twice.
The reason for the second open is that a single share can be someone testing the button. A recipient coming back is the only cheap evidence that the report became part of how they work, which is the thing we actually claimed this feature would do."
Then the trap you're deliberately avoiding.
"What I'd refuse to use as the headline is total shares. It's the number that will look best, it'll go up the week we announce it, and it would be almost entirely internal curiosity. If someone puts it in a board pack I'd want the recipient-return number next to it."
Then the guardrail, briefly.
"One thing to watch rather than optimise: support tickets about permissions. The failure mode for this feature isn't low usage, it's someone sharing something they shouldn't have — and that would show up in support long before it showed up anywhere else."
Then how long you'd wait.
"I wouldn't read it for six weeks. The recipient-return window is a month by construction, and the first two weeks will be distorted by the announcement. If someone needs a signal sooner, the honest early indicator is what share of shares get opened at all — that's available in days and it tells you whether the emails are landing, which is a different question but a real one."
The sentence that does the most work is "not shares sent — shares that got used twice". Everything before it is process; that is a product judgement, and it's the only part of the answer the interviewer couldn't have written themselves. Every strong metrics answer contains one moment where a candidate refuses the obvious count and says what would make it misleading. Without that moment, a correct answer still sounds like a checklist.
The three definition traps
A metric can be the right choice and still be useless, because nobody agreed what it means. These come up as follow-ups constantly.
| Trap | The question that exposes it | What a good definition does |
|---|---|---|
| The denominator | "Per account, or per user?" | Say which and why — a per-user number hides that one account does everything |
| The window | "Active in what period?" | Tie the window to the natural rhythm of the job, not to a round number |
| The event | "What counts as 'used'?" | Name the specific action, not the category |
On windows specifically: monthly active is the default and it's usually wrong. If the underlying job happens weekly — a sprint report, a payroll run — a monthly window counts someone who did it once as identical to someone who did it four times. Matching the window to the job's real frequency is one of the cheapest ways to sound like you've done this.
On "active": the word means nothing on its own, and the strongest version of any usage metric replaces it with a verb. Not "monthly active users" but "accounts that ran at least one report in a calendar month". You can argue with the second one, which is the point.
When the data doesn't exist
A very common second half of this round: "we don't currently track that. Now what?"
The weak answer proposes a proxy immediately. The strong one is ordered:
What's already there. "Before instrumenting anything — do we have the share events and the recipient opens? If we have both with identifiers, I can build this from what exists and it's a query, not a project."
What the proxy costs. "If we only have shares sent, the proxy is shares per account, and I'd use it knowing it over-counts by exactly the thing I'm worried about. I'd rather have it and say so than wait six weeks for a clean number."
What you'd add, and its size. "The one event I'd add is recipient open with a share ID. One event, one afternoon, and it unlocks the real metric — which is worth arguing for because we'll want it for every sharing feature after this one."
That last framing — instrumentation as an investment rather than a request — is what senior product people sound like.
The list
“I'd look at adoption, engagement, retention, NPS and revenue impact, and build a dashboard so we could track them all together.”
Five metrics, no choice, and a dashboard nobody will read. If three go up and two go down, this candidate has no way to say whether the feature worked — which is the only question they were asked.
One number, and its limits
“Accounts where an external share gets opened twice in a month. Not shares sent — that'll spike on announcement week and mean nothing.”
One metric, defined precisely enough to build, with the tempting alternative explicitly rejected. The interviewer can now disagree with you, and a metrics answer you can argue with is worth more than one you can't.
The follow-ups
"What if it goes up and the business doesn't improve?" — checking whether you'd notice your metric was wrong. "Then the metric is measuring something real that doesn't matter, and I'd want to know which link in the chain broke — are the shares happening in accounts that were already safe? If so, this is a feature for people who didn't need it, and that's a targeting problem rather than a product one."
"How would you know if it's cannibalising something?" — the senior question. "I'd look at whether report exports fell in the same accounts. If people are sharing instead of exporting, the net effect on the account might be zero and the feature is a nicer version of something that already worked."
"Pick a north star for the whole product." — a different question, and a bigger one. It's the one covered in the existing list: product manager interview questions. Don't answer it here by widening this answer; say you'd treat it separately.
Preparing for this round
Take three features you've shipped and, for each, write the one sentence: the metric, with its denominator, its window and its event named. Most people discover their real metrics were never defined this tightly, which is exactly what the round is probing.
Then write the number you'd have refused to use, and why. That's the sentence that gets you the offer.
The frameworks that get reached for in this round — and why naming HEART out loud makes the answer worse — are covered here: product manager interview frameworks. And if the round turns into diagnosing a metric that already moved, that's a different shape, worked through here: data analyst case study.
Practical target: take any feature and answer out loud in ninety seconds, with a rule — you may name exactly one metric. The discipline of having no second choice forces the definition work to the front, and the definition work is where this round is actually decided. If ninety seconds feels short, notice how much of your usual answer was listing alternatives instead of committing to one.
