Data Scientist Interview Case Study: A Worked Example, Start to Recommendation

The product case is the round that decides most data science loops, and it's the one candidates practise least — partly because it's hard to practise alone, and partly because most written advice about it stops at "structure your thinking".

So here is one case, worked all the way through, with the numbers on the page. Read the brief first and try it yourself before the walkthrough. The interesting part isn't the answer; it's the point about two-thirds of the way down where the obvious conclusion turns out to be wrong.

The brief

Our subscription renewal rate fell from 71% to 63% last quarter. Find out why, and tell us what to do about it.

You have access to the usual: subscription records, product events, support tickets, and the CRM. You have about twenty-five minutes.

That's the whole prompt. It's deliberately thin, and the thinness is the first test.

The first ninety seconds

Before touching a decomposition, three things are worth establishing out loud. Interviewers score this part heavily and candidates skip it, because it feels like stalling.

Is the number real? "First I'd check that nothing about the measurement changed. A different definition of renewal, a change to when a subscription is counted as churned, a billing retry window that moved — any of those produce exactly this shape and none of them are a business problem. I'd want to reproduce last quarter's 71% with today's query before I trust the 63%."

Is it outside normal variation? "Eight points is large, but I'd want to see the last eight quarters rather than two. If this metric routinely moves four or five points, an eight-point drop is less remarkable than it sounds, and the answer might be that we're over-reading one quarter."

What decision does this feed? "Whether this is a pricing question, a product question or a go-to-market question changes what I'd spend the twenty-five minutes on. If someone's about to change the pricing page, that's a different investigation from a board question about the trend."

Assume all three come back clean: the measurement is stable, the historical range is 69–73%, and the question is real.

Decomposition one: where is it concentrated?

The first move is always the same. A metric this size is either broad or concentrated, and those have completely different explanations.

The weighted totals reconcile: roughly 71% previously, roughly 63% now.

This is the moment the case turns. Two segments barely moved. The entire eight-point drop is coming from one segment that represents 15% of renewals and fell by 47 points.

Saying it out loud: "So this isn't a general renewal problem — it's one segment, and a dramatic one. Which means anything I'd have said about pricing or the product experience broadly is now off the table. The question is no longer 'why did renewals fall', it's 'what happened to sales-assisted annual accounts'."

That reframing is worth more than everything after it. Candidates who never decompose end up recommending a pricing change to fix a problem that lives in fifteen percent of the base.

Decomposition two: what's inside the segment?

Now the obvious hypothesis. Sales-assisted accounts are managed by humans, so the natural theory is a people problem — a reorganisation, someone leaving, accounts going unmanaged.

Worth checking, and worth saying you'd check it. But before that, one cheaper question: who was actually up for renewal this quarter?

Annual contracts renew twelve months after they're signed. So this quarter's sales-assisted renewals are last year's sales-assisted signings — a specific cohort, not a random sample.

There it is — and there's a second thing in the table worth catching. The discount cohort renewed at 24% and made up nearly two-thirds of the renewals due, which explains most of the collapse. But the normally-signed accounts came in at 68%, against a segment history of 88%. That's a twenty-point drop too, in a group with nothing unusual about it.

So there are two effects stacked on top of each other, and they are not the same size or the same problem.

Saying it out loud: "Most of this is composition. A year ago we ran a discount campaign that brought in a lot of sales-assisted accounts, they're renewing at a quarter of the normal rate, and they happen to dominate this quarter's renewals — so the headline number is being driven by who was eligible to renew, with a twelve-month lag. But I don't want to explain the whole thing that way, because the normal accounts also came in twenty points below their history. That's a smaller effect and it's a real one, and it would have been invisible if I'd stopped at the first explanation."

What this means, and what it doesn't

The distinction matters enormously for what you recommend, and it's where candidates who found the right number still lose the round.

That middle column is the one to say out loud. The discount campaign is confounded with everything about the accounts it attracted — they may have been smaller, less committed, in different industries, or sold to under time pressure by a team hitting a quarterly target. The data shows the association, not the mechanism.

"What I can't tell you from this is whether the discount caused it or just selected for it. Those have opposite implications: if discounting attracts price-sensitive buyers who were never going to stay, the fix is qualification. If discounting anchors the price and they churn at renewal when it goes up, the fix is how we structure the renewal. I'd want to know which before recommending we change the discounting policy."

The recommendation

Three parts, in the order a stakeholder can act on them.

What I'd tell you today. Most of the headline number is composition and will partly recover on its own, so I'd reforecast the next two quarters from the cohort mix rather than extrapolating the trend — we already know who is up for renewal. But I would not call the whole thing a false alarm, because the normally-signed accounts dropped twenty points too, and that piece has no explanation yet.

What I'd do this week. Pull the discount cohort's characteristics — company size, industry, who sold them, what they paid — and compare against the normal cohort. That's a day's work and it separates the two explanations above. Also check whether these accounts ever activated properly, because an account that never onboarded is a different story from one that used the product and left.

What I'd want to change if the qualification theory holds. Not the discount itself — the criteria for who gets offered it, and a check at ninety days rather than at renewal. If a cohort renews at 24%, we knew something was wrong eleven months before the renewal and nobody was looking.

What the panel scored

None of those required a model, and that's typical. The case round is almost never won with technique.

Practising this alone

Take any metric you've worked with and write the brief yourself in one sentence, with no context. Then answer it out loud in twenty-five minutes, forcing yourself to decompose twice before you explain anything.

The habit that transfers is the second decomposition. Almost everyone does the first one — split by segment, find where it's concentrated. Far fewer ask who was eligible to be in this number at all, which is where the answer was hiding here and is where it hides surprisingly often.

For the rest of the loop — the SQL screen, the statistics round, the modelling round and the communication round — the question list is here: Data Scientist Interview Questions. And the presentation version of this round, where you have to defend the finding live, is covered in the complete technical interview communication guide.