When a Case Study Is Really a Year in Review

When a Case Study Is Really a Year in Review

You have a case study open, and the headline number is strong. Before deciding whether it holds up, you need to know what it is actually a number of. Some case studies describe one campaign with a start and an end. Others quietly describe twelve months of work, folded into a single narrative that reads like one win. The two documents can use the same headline format. Only one of them tells you what a single effort achieved.

The tell is in the timeline, not the headline

Watch for a single hero metric with no matching date range beside it. Watch for phrases like “over the past year” next to a number with no stated start and end. Watch for month or quarter labels that appear only inside a chart, never in the prose. And watch for a client relationship described as ongoing rather than as one discrete engagement.

A genuine single-campaign case study almost always names a start date, an end date, or a duration in its first few paragraphs. To say “this campaign produced this result,” you need to say what “this campaign” covers in time. The absence of that detail is itself informative. A writer with a clean window to point to would usually point to it, because it strengthens the claim. When duration words replace dates, treat the silence as a fact about the document, not an oversight to fill in yourself.

Why the honest unit of analysis changes what the number measures

A “signature win” framing and a rollup framing look similar but make different claims. The signature win implies a single cause acting inside a single window: this campaign, this budget, produced this outcome. A rollup is really the sum or average across many separate efforts run over a year, some of which likely underperformed. Averaging is not dishonest by itself. Writing the copy as though an average describes one cause is where the claim stops matching the evidence.

This is a different honesty problem from a cherry-picked window, where a short, strong slice of a longer campaign stands in for the whole. A year in review lets a few strong months carry the weaker ones without saying so. The weak months are hidden through aggregation, not selection.

A hypothetical rollup, worked through

Here is a hypothetical, composite example, not a real agency or client, built only to illustrate the mechanism.

Picture a fictional agency reporting that a client’s engagement rate rose sharply “over the course of our partnership last year.” Now picture the twelve months underneath that figure: a slow first quarter during setup, a strong second quarter as a new format caught on, a flat third quarter, and one exceptional month in the fourth quarter tied to a single well-timed post. Averaged, those months could still produce an impressive annual number, carried almost entirely by that one month. The annual total alone would not tell you this happened.

The point is not that this pattern occurred anywhere. It is that the same annual number is compatible with a year of steady growth and with a year of one great launch month propping up eleven flat ones. A case study reporting only the annual figure cannot tell you which of those two years happened, because both produce the identical top line number.

What to check before you accept the number

  • Does the case study state a campaign start and end date, or only the date the client relationship began? A relationship start date is not a campaign window.
  • Does any chart break the total into smaller periods, and if so, does every period trend the same direction as the headline? A chart never referenced in the prose is worth a closer look.
  • Does the case study describe one engagement with a defined scope, or an ongoing retainer? A retainer is structurally more likely to be reported as a year in review, since there is no natural campaign boundary to write around.
  • Do the same client’s results appear elsewhere in smaller, dated increments you can compare against the annual claim?

How this differs from an evergreen program reported as one win

It is easy to conflate this with a related problem: an always-on program reported as a single win, framing an unbounded effort as if it were one discrete campaign moment. A year in review is a different shape of the same instinct, a bounded, usually annual document that compresses many separate, unevenly performing campaigns into one narrative arc.

The date range check works for both, but what you are checking for differs. An evergreen program falsely borrows the shape of a single moment. A year in review averages several distinct, nameable campaigns into one number without disclosure. One is a boundary problem, the other is an aggregation problem.

What a well disclosed version looks like

A well disclosed year in review names the reporting period as a full year up front, breaks the year into the campaigns or phases that made it up, and avoids implying a single cause for what is plainly an aggregate, phrasing the claim as “across the year’s work” rather than “this campaign delivered.”

The problem is never that a case study covers a year. Plenty of honest ones do. The problem is only when a document borrows the shape of a single campaign while reporting an aggregate that no single window produced. For the wider set of checks this piece fits inside, see the basics of reading a marketing case study without being misled. For the mirror image, a short window standing in for a long one, see a 90 day headline standing in for a 12 month campaign, and on the worked example above, a bad quarter tends to be followed by a better one anyway regardless of the campaign.

You can browse the case study library, and agencies with a case study worth adding are welcome to submit their own.

FAQ

What is the quickest way to tell a case study is really a year in review?

Look for a stated campaign start and end date near the headline number. If the only date is when the relationship began, or the copy leans on phrases like “over the past year” without a matching bounded window, treat the number as a rollup until shown otherwise.

Is a year in review case study automatically dishonest?

No. It can be entirely honest when it discloses the reporting period and breaks the year into the campaigns inside it. The problem is only when it reads like a single campaign win while actually being an aggregate, with the disclosure left out.

Sources

Similar Posts

Leave a Reply

Your email address will not be published. Required fields are marked *