What to Ask an Agency for When a Case Study's Numbers Do Not Add Up

What to Ask an Agency for When a Case Study’s Numbers Do Not Add Up

You have already found the gap. Maybe the case study never shows a date range picker, only a summary card with a big number on it. Maybe the lift is real but you cannot tell if it used a short or long attribution window. Maybe a rate is stated without saying what it divides by what. Whatever it is, you now face a second problem: how to ask the agency for the thing that resolves it, not the thing that gets you a polite non-answer.

You already did the hard part

Spotting a specific gap in a case study is the harder half of this process, and if you are here, you have done it. This piece picks up where that diagnosis ends: you know which kind of doubt you have, and need to know what document or export would settle it.

A general request wastes the conversation. “Can you share your data behind this?” invites a general answer, usually a restated summary or reassurance. If you want a specific answer, name a specific artifact.

If you have not yet worked out which kind of gap you are looking at, that diagnosis lives elsewhere on this site, in posts like self-reported versus verifiable numbers, how to read a case study without being misled, and the attribution window is a choice. Read one of those first if you are still at “something feels off” rather than “I know exactly what feels off.”

Match the doubt to the artifact

Each row pairs a specific doubt with the artifact that resolves it. Ask for exactly what is named, not a paraphrase of it.

The doubt The artifact to request
No visible date range, only a summary card. A dashboard export or screenshot that includes the date range picker itself, not just the summary card below it.
The result could depend on a short or long attribution window, undisclosed. The platform’s attribution settings screen, or the report’s methodology footnote, showing which window produced the number.
A rate or percentage with no stated numerator or denominator. The platform-native report showing the raw numerator and denominator, not a pre-calculated percentage.
The number only appears in a slide or written summary the agency produced. A platform-native export (for example Meta Business Suite, TikTok Analytics, or GA4) that the agency did not type into a slide.
The “before” figure was chosen by the agency and the comparison looks uneven, as covered in how to read a results chart before the caption. The “before” period’s own dashboard export, on the same date range length as the “after” period, from the same source.

Every artifact in the right column is named by type: a dashboard export, a settings screen, a raw numerator and denominator, not “the data” or “proof.”

A worked request, phrased the way you would actually send it

The following is a hypothetical exchange with a fictional agency and a fictional client, invented only to show the contrast between a vague ask and a precise one.

Vague version: “Hi team, this case study mentions a big jump in results. Can you share the numbers behind this so we can see how it was measured?”

Precise version: “For the case study referencing [hypothetical] Client A’s Q2 campaign, could you export the Ads Manager report for that specific campaign, for the exact date range shown in the case study, including the attribution window setting on that report?”

The vague version can be answered with a slide screenshot, a reassurance, or a different number entirely. The precise version names the platform, the object (a campaign, not the whole account), the date range already printed in the case study, and the specific field in question, which is what makes it answerable in one reply instead of a back and forth.

What a real artifact looks like versus what a reassurance looks like

A screenshot of the platform’s own interface, with its native menu chrome and date picker visible, is a different object than a screenshot of a slide stating a number in a large font. Only one shows the system that produced the number. A verbal reassurance, “yes, that number is accurate,” produces no document you could independently check, however sincerely it is meant. A report generated specifically in response to your named request is worth more than one that already existed in a pitch deck, since the pitch deck version was built to persuade rather than to answer this question.

If the agency cannot produce it

Sometimes the artifact genuinely no longer exists. Some platforms limit how far back a dashboard view can be regenerated, so a live view from far enough in the past can be a real, mechanical limitation rather than evasion. [EVIDENCE NEEDED: a specific retention period stated by a named platform, not asserted here since it varies by platform and policy.]

What matters is what the agency does instead. Explaining, in plausible terms, why an export can no longer be regenerated differs from answering a request for a named campaign’s Ads Manager report with a company-wide summary instead. The first is a limitation you can evaluate. The second quietly substitutes a more convenient number for the one you asked about.

Where to go next

If the gap you are trying to name is not covered above, browse the case study library, or submit your own.

FAQ

What is the single most useful thing to ask for first?

Whatever artifact directly targets the specific doubt already diagnosed, a date range picker view, an attribution setting, or a raw numerator and denominator, rather than a generic request for “the data” or “proof.”

Is a screenshot ever good enough?

Only if it shows the platform’s own interface, including the date range control, rather than a slide or summary card restating a figure.

What if the agency says the number is accurate but won’t send anything?

A verbal assurance is not an artifact and does not resolve the doubt, however confidently it is stated. That differs from an agency explaining a genuine, retention-window style limitation on producing an export. This site stops at naming that distinction, not at what to do next about the relationship or any contract.

Does asking for raw numbers mean I distrust the agency?

No. The tone to bring is wanting the claim to hold up, not catching anyone out. A specific, well-named request is easier for a legitimate result to satisfy than a vague one.

Sources

Similar Posts

Leave a Reply

Your email address will not be published. Required fields are marked *