How Old Is the Youngest Cohort Behind That Result?
Conversions and refunds arrive late. Learn to spot a case study number cut off before its cohort matured, and what to ask the agency to see the final figure.
How to interpret social media results and the numbers behind them
Conversions and refunds arrive late. Learn to spot a case study number cut off before its cohort matured, and what to ask the agency to see the final figure.
When a test meant to be 50/50 reports very uneven groups, the result may be compromised. Here is the arithmetic check you can run from published counts.
A results chart whose axis starts above zero can make a small change look huge. See a hypothetical example, when it is fair, and how to read the real change.
A case study says top 1 percent or number one in its category. Three questions, pool, metric and period, show whether the ranking means anything at all.
A year-over-year gain is only as good as the year it beats. Learn to test the comparison year against a second reference year before you trust the percentage.
A case study says its winning variant was 97 percent confident. Here is what that figure actually measures, what it leaves out, and the questions to ask next.
A ‘gained X new followers’ headline can hide the people who left. Learn the difference between gross gain, net gain and ending count, and what to ask for.
A code count measures redemptions, not influence. See what it overstates, how codes leak beyond a campaign, and the one question that exposes the gap.
Self-reported and platform attribution answer different questions. Here is what each one sees, misses and inflates, and how to read both in a case study.
Two rising lines on separate y-axes will track each other, because the author picks both scales. Here is the swap test and indexed-axis check that expose it.