Someone else's case study result is not your benchmark

Someone else’s case study result is not your benchmark

You finish a case study, the closing number is strong, and before you reach the bottom of the page you are already thinking about next quarter’s target. The percentage has quietly moved from describing what happened to someone else into being an expectation for you. Nothing in the document stopped you, because nothing in it was written to.

The number is real. The comparison isn’t.

Assume for this whole piece that the number is true: measured properly, reported honestly, workings available on request. Fabrication is a different problem and this is not it. The figure can be sound and still be the wrong thing to put in your plan.

What has gone wrong is a category error. Whether a number is accurate and whether it applies to you are separate questions, and answering the first well tells you nothing about the second. Verifying is checking. Adopting is forecasting.

It slips past careful readers because a percentage carries no visible attachment to the business it came from. “Up 34 percent” has no units, no currency, no audience size, no calendar. A spend figure announces its own context and you compare it to yours instinctively. A bare percentage announces nothing, so nothing prompts you to ask whether it transfers. Which is why how to read a case study without being misled is only half the skill. The other half is knowing what a correctly read result does not license you to conclude.

What actually has to match

A borrowed result is a fair benchmark only when the conditions behind it resemble yours. Five do most of the work.

  • Starting audience or account size. The same relative gain is a different amount of work at different scales, so a small base and a large base are not the same achievement.
  • Budget behind the result. A percentage says what moved, not what it cost to move, and spend several times yours is an input you do not have.
  • Baseline maturity of the metric. Neglect leaves easy gains lying around, so lifting an ignored metric yields a bigger percentage than lifting an optimised one.
  • Industry and purchase cycle. Where buyers decide over months rather than an afternoon, a real effect can fail to appear inside the same reporting period.
  • Time window and channel mix. A result built over eighteen months by paid, organic and partnerships together is not something to ask of one quarter on one channel.

A hypothetical: two businesses, one percentage

What follows is a hypothetical. No real company, no agency, no published result, and the figures were invented to show the arithmetic.

Business A starts the hypothetical year with 3,000 followers, Business B with 300,000. Both report the same hypothetical headline: followers up 30 percent over twelve months. A added 900 and finished at 3,900. B added 90,000 and finished at 390,000.

For A, one post travelling further than usual can account for most of the gain. For B, 90,000 additions is the entire audience of thirty businesses the size of A, and no lucky post gets you there. Reverse it and the asymmetry flips: B adding A’s 900 is a rounding error of 0.3 percent nobody writes up, while A adding B’s 90,000 would be a thirtyfold increase. The one thing the shared 30 percent never tells you is which result was hard.

Why the same percentage means different things

Relative change is a ratio, and a ratio stretches or compresses depending on where it started. Small baselines inflate percentages because any absolute movement looks dramatic against them, and large baselines deflate them for the same reason in reverse. Baselines also sit at different distances from a practical ceiling. A metric far below its ceiling has room, and lifting it by half is often a matter of repairing something obviously broken. A metric near the ceiling may not move by half at all, whatever the skill or spend.

Budget compounds it, because a percentage hides its own price. A result reached with sustained media spend is partly a statement about what money bought. Copy the percentage without the money and you have copied the ambition while leaving the mechanism behind.

The checklist before you adopt someone else’s number as a target

Condition Check about your business What a mismatch looks like
Audience or account size Follower count, list size or traffic on the account the target applies to Their base is an order of magnitude from yours, so the same percentage is a different number of humans
Budget or spend What you can commit over the same period, production included Their result rests on sustained paid support and your plan is organic, or the reverse
Baseline maturity Whether the metric has been worked on, and how near its ceiling it sits They lifted a neglected metric off the floor, yours has no easy gains left
Industry and purchase cycle How long buyers take to decide, against your reporting period Their category converts in days and yours takes months
Time window and channel mix The period you would measure and which channels actually run Their number covers a year of multi-channel work, yours a quarter on one

Three or more mismatched rows and the number is not a target. It is somebody else’s weather report.

When a borrowed number is still useful

A case study is good evidence that an approach produced an outcome under the conditions it describes, which shows you what is plausible in a category and what the work behind it looked like. What it cannot do is set a numeric expectation for a business it never measured. So use a borrowed number to decide what is worth testing, not what counts as success. Success is defined against your own baseline, over your own window, at your own spend.

Two cautions to close. A library of published work is a selected sample, an independent reason not to make any of it a default expectation, and every case study library is a selected sample covers why. Our editorial rule here is that every number in this library belongs to the agency that reported it, never restated as an industry benchmark, because a credited result that becomes a general figure stops describing anything real. Read it alongside the counterfactual a case study can’t show you.

Read more results, borrow fewer numbers

Read enough published work and the conditions start to stand out before the headline does. You can browse the case study library, each entry credited to the agency that reported it, and submit your own if you have published work.

FAQ

If the case study’s numbers are accurate, why can’t I use them as a benchmark?

Accuracy and applicability are separate questions. A result can be reported honestly and still describe a different audience size, budget, starting baseline and time window than yours. Confirming it is true tells you the agency measured correctly. It does not tell you the number would appear again under your conditions, which is what a benchmark promises.

What conditions actually need to match for a borrowed number to be a fair benchmark?

Five: starting audience or account size, the budget behind the result, how mature the metric already is and how near its practical ceiling it sits, the industry and its purchase cycle, and the time window and channel mix it was measured over. A mismatch on several means the number describes a business that is not yours.

Sources

Similar Posts

Leave a Reply

Your email address will not be published. Required fields are marked *