The campaign launched in December, and so did everyone's sales

The campaign launched in December, and so did everyone’s sales

A case study reports the result in a clean sequence: a campaign ran, a number went up. Then you check the dates and see the window ran through November and December. The question is not whether the number went up. It is whether it went up more than it would have anyway.

The result and the calendar arrived at the same time

This is not the usual complaint about a missing baseline. These case studies often have one, and a reasonable one: the eight weeks before launch, same account, same method. The harder problem is that the baseline itself was moving for reasons unconnected to the campaign. Comparing December against October in a category where December is always the biggest month tells you what December does.

So a case study can be honest about every number it prints and still mislead, because the comparison that separates a campaign effect from a calendar effect is the one it does not show.

What counts as a category tailwind

Most are recognisable on sight once you know the shapes:

  • A retail holiday season. The end of year shopping period and the smaller peaks around it, when purchase intent rises across most consumer categories on its own.
  • A category-specific annual cycle. Back to school for stationery, wedding season for venues, tax season for accountants, the opening weeks of a sport’s season. A wave inside one category, invisible outside it.
  • A one-off, industry-wide event. A platform change that lifted organic reach for everyone at once, or a trend the whole category rode. Not annual, but it moves many accounts at once.
  • The company’s own recurring cycle. A brand that launches its main line at the same time each year, or repeats an annual sale, spikes in that window with or without an agency.

What unites them is predictability. A tailwind is something a competent person in that category could have anticipated, which is what makes it so easy to mistake for the campaign’s doing afterwards.

A hypothetical: the same reported number, two different stories

Here is a hypothetical, with invented numbers for illustration. A homeware brand’s agency reports a 27 percent increase in purchases over a six week window, against the six weeks before.

In the first version, the window sits in late February and March. Nothing in the homeware calendar peaks there: no holiday, no annual sale, no category-wide event. Whatever moved, moved while the category was ordinarily flat. That 27 percent has other questions to answer, but timing is not one of them.

In the second version, the same 27 percent comes from a window running from mid October through the biggest shopping weekend of that market’s year. The number is identical and the evidence much weaker, because a plausible alternative explanation now sits in plain view and the case study never rules it out. A competitor running no campaign may have posted a similar rise.

Both versions are hypothetical, and the same figure carries very different weight in each depending only on the calendar.

The one question that discounts for it

Ask this: what was this category doing during this same window in a typical year, independent of this specific campaign?

Three answers would settle it. The same brand in the same calendar window a year earlier, which turns a before and after comparison into a year over year one. A same-window comparison against accounts that did not run the campaign: a holdout group, a sibling market, comparable accounts the agency manages. Or a category-level seasonal pattern. Any of the three is decisive. Most show none.

You usually cannot get this data yourself: prior year figures are private, holdouts are rarely published. The move is to notice the absence and discount the claim rather than accept it. Same gap as the counterfactual a case study can’t show you, aimed at the calendar instead of the account’s history.

Not the same problem as a rebound, and not the same problem as a moved goalpost

Regression to the mean is an internal pattern: a campaign starts after an unusually bad stretch, the account drifts back to normal on its own, and the recovery is billed as a result. A seasonal tailwind is external, and would have lifted almost any account in that category running in that window. One is about where the account was before, the other about what everyone around it was doing at the same time.

A changed attribution window is a third thing: how credit is distributed inside a result, not what moved it at all.

Reading a case study reported over a holiday window

When the reporting period runs through a well known spike, three checks do most of the work:

  1. Does it state the exact reporting dates at all? “This quarter”, “over the holiday season” and “during our peak period” all name a stretch of calendar without committing to one, which leaves the comparison impossible to isolate from the season.
  2. Does it compare against the same window a year earlier, or only against the weeks immediately before launch? The second is the default, and the weaker whenever the window overlaps a known peak.
  3. Is this a category that spikes then anyway? Ask about the industry, not the brand. If a competitor doing nothing would also have had a good six weeks, the campaign has a higher bar to clear.

A holiday result is not worthless, it is a claim with a known gap. For the wider framework, see how to read a case study without being misled.

Try it on a real one

Browse the case study library and try this on the next few results you read: note the dates, the category, and whether a prior-year comparison appears. If your own published work names its window and shows what the category did anyway, submit your own.

FAQ

What is a seasonal or category tailwind in a case study?

A predictable, external driver of activity across a category, such as an end of year shopping season or an annual launch cycle, that would raise results for most accounts in that category during the same window regardless of any campaign. A lift reported over that window contains both effects together, and the case study rarely splits them.

Does a case study reported over the holidays automatically mean the result is fake?

No. This is not an accusation of dishonesty, it is a gap the case study may not have addressed, and plenty of strong campaigns run in peak season for good reasons. Ask for a same-window prior-year or category comparison, and hold the claim loosely until you see one.

Sources

Similar Posts

Leave a Reply

Your email address will not be published. Required fields are marked *