If the Number Never Comes Back, Ask Why It Disappeared

If the Number Never Comes Back, Ask Why It Disappeared

You are holding a case study with a strong headline number. Before deciding what that number is worth, there is one check you can run in about ten minutes: has the agency ever mentioned this client again, anywhere, since?

The habit: search for the number’s second appearance, not its first

Take the client’s name from the case study you are reading. Then look for that same name anywhere later in the agency’s published output: its own site, its case study index if it keeps one, its social proof or client list pages, and any later pitch materials that happen to be public. You are searching for a second mention, published after the first.

The search ends in one of two places. Either the client and the number show up again somewhere later, a portfolio refresh, a later roundup, an updated figure, in which case there is at least a thread to check. Or the client’s name never appears again anywhere, not once, in any later format the agency uses. That second outcome, total disappearance, is what the rest of this post is about.

Why this is a different question from the number getting stale

Keep this separate from a result going stale. A number goes stale because the world underneath it changed, a platform’s ranking logic shifts, an ad auction gets more competitive, and a lift measured two years ago might not reproduce today even with the same tactics. The number was true when reported and simply stopped generalising.

Total disappearance is narrower. It is about whether the agency has ever brought the client or the number back up at all, for a client it is presumably still able to talk about if the relationship held. A stale result can still have been honestly reported once. A vanished result raises a different question: not whether it would still be true, but whether the agency ever wanted to point back to it.

What disappearance can mean, and what it does not prove

The benign explanations are the largest group. The relationship might have ended for reasons unrelated to the result: a budget cut, a reorganisation, a roster change. The agency might have stopped publishing case studies in that format, or migrated its site and lost the archive. The client may have asked not to be named again. Or, most mundane of all, no one got around to writing a sequel, since case studies are not written on a schedule.

The less benign explanations exist too: the result may not have held up over a longer stretch, the relationship may have ended shortly after the case study ran, or the number may have been inflated in a way that made a follow up uncomfortable to publish.

The honest conclusion sits between those two lists. Total disappearance is a prompt to ask, not a verdict you get to reach alone. It shifts the burden reasonably onto the agency to explain the gap, and does not settle the question by itself.

A worked example of the check, with hypothetical numbers

Hypothetical example, not a real agency or client.

Picture a fictional agency, Agency H, that publishes a case study for a fictional client, Client K, a mid sized retailer. The case study cites a headline lift in engagement, reported as Agency H’s own client result and marked as specific to that account and period, not an industry figure.

Two years later, a reader runs the check above, searching Agency H’s site, case study index, and social channels for any mention of Client K. Nothing turns up: not a portfolio update, not a later “year in review” post, not a client list mention. The name that anchored a headline case study two years earlier is absent from everything published since.

That absence is the finding, and it is exactly as far as it goes on its own. The reader’s next step is not to conclude the number was fabricated. It is to ask Agency H directly whether the relationship with Client K continued and whether the result held.

What to actually ask the agency

Once you have confirmed a genuine disappearance, three questions cover the ground fairly:

  • Is the client relationship still active, and for how long did it run past the case study’s publication date?
  • Has the reported result been measured again since, and if so, what did that later measurement show?
  • If the relationship ended, was that connected to the result not holding up, or to something unrelated?

When the check does not apply

A case study published recently has not had time to reappear anywhere yet, so absence right after publication is not evidence of anything.

The check also breaks down for an agency that rarely publishes case studies, or never in a recurring format like a client list or yearly roundup. With no habit of revisiting any client in print, it will not show this pattern for any client specifically, which makes the absence uninformative for that agency as a whole.

Keep the scope narrow, too. This check is about one client and one number, not whether the agency is good or bad generally. A single vanished case study says nothing about its other clients.

Go check one

If you have a case study open right now with a number you have been taking at face value, search the agency’s own site and channels for that client’s name and see if it ever comes back. Browse the case study library for more to practice on, or submit your own if you have a result with a documented follow up.

FAQ

Does a case study result disappearing from later materials mean it was fake?

No. A client name might never resurface for benign reasons: a site migration, a change in format, a relationship ending for unrelated reasons, or simply the fact that no one writes a sequel to every case study. Less benign explanations exist too, but disappearance on its own only justifies asking the agency, not concluding anything.

How far back should I look for a later mention?

From the case study’s publication date through the present, not a fixed window like the last twelve months. The point is total absence over the whole period, not absence in some arbitrary slice of it.

Where should I actually look for a later mention?

Start with the agency’s own case study index or portfolio, then its client list pages, then any later year in review or roundup post that would plausibly reference a standout result. A general web search is not a substitute, since the agency’s own site is the most likely place a continuing relationship would surface.

Should I ask the agency directly if I can’t find a later mention?

Yes, framed as a fair question rather than an accusation, using the three questions above: whether the relationship is still active, whether the result has been measured again, and if the relationship ended, whether that was connected to the result.

Sources

Similar Posts

Leave a Reply

Your email address will not be published. Required fields are marked *