'Top 1 Percent' Is a Rank Within a Pool. Who Is in It?

‘Top 1 Percent’ Is a Rank Within a Pool. Who Is in It?

A case study says the account finished in the top 1 percent, or number one in its category, and then moves on to the next slide. You would like that to be true. Before you repeat it, you need to know what the number was counted against, because a rank means nothing until you know the group it was ranked inside.

A rank is a position, and positions need a group

A percentile rank describes where one value sits inside a defined group. In the standard statistical definition it is the percentage of scores in the group that fall below a given score, with ties handled by convention (Wikipedia, Percentile rank). The group is part of the number. Change the group and the same score gets a different percentile. Statisticians call the wider difficulty the reference class problem: deciding which class to use when calculating the probability that applies to a particular case (Wikipedia, Reference class problem).

In a published result, the claim is usually true as stated and uninformative, because the pool was chosen by the person making the claim. That is not necessarily dishonest. It means the sentence has a missing input. Three things define any rank: the pool, the metric and the period.

One account, two pools: a hypothetical worked example

This whole section is hypothetical. Every number is invented to show a mechanism. None of it is a reported result or a benchmark, and no real account or agency is involved. Picture one account with an engagement rate of 3.1 percent, ranked in two pools.

Hypothetical pool Pool size Position Arithmetic Result
Pool A: hand-picked small accounts in a quiet niche 60 1st 1 / 60 = 0.0167 Number one, top 1.7 percent
Pool B: broad set in the same category 5,000 2,000th 2,000 / 5,000 = 0.40 Top 40 percent

Invented numbers, for illustration only. Notice that 1st of 60 is the top 1.7 percent, so even “number one” in a small pool does not reach “top 1 percent” of anything large. A claimed top 1 percent of a pool of 60 would mean 0.6 of an account, which is less than one. Top 1 percent of the 5,000 pool would need rank 50 or better, and rank 1 of 5,000 would be 0.02 percent.

Question one: what pool?

Pools shrink quietly. Watch for these:

  • The agency’s own client roster.
  • Accounts of a similar size, where the writer decides what “similar” means.
  • A single region or language.
  • Accounts that use the same tool or programme.
  • Only accounts that stayed active for the whole period.

The last is survivorship. Ranking among accounts that still exist removes the ones that failed, as we argue at library level in every case study library is a selected sample. The same hidden-denominator problem hits other metrics, and share of voice depends on the competitor set behind it. This post is about rank statements specifically.

Then there is “in its category”. Category boundaries are drawn by whoever writes the sentence. An acceptable disclosure gives the pool size, how members were selected, and who selected them. “1st of 212 accounts, taken from a third-party list of every public account in the category” can be checked. “Top in category” cannot.

Question two: ranked on what metric?

A rank is on one metric, and different metrics reorder the same pool. Take two hypothetical accounts. Account X has 2,000 followers and a 6 percent engagement rate, which is 120 engagements per post. Account Y has 200,000 followers and a 1.2 percent rate, which is 2,400 engagements per post. Rank on rate and X wins. Rank on count and Y wins. All of it is invented, and the reversal is the point.

The reasoning carries over without needing a statistic. A small base can post a high rate on a few enthusiastic people, while a large base moves big totals at a modest rate. So a pool of small accounts flatters a rate ranking, and a pool with large accounts flatters a count ranking. The size mix of the pool decides which metric the writer reaches for.

If the metric is not named at all, the claim is a mood, not a measurement. “Top performer” is a feeling. “Highest engagement rate, 1st of 212” is a claim.

Question three: over what period?

Rankings are snapshots, and a period can be chosen to catch a peak. A rank for the best week is a different claim from a rank across the campaign. We cover how that choice gets made in how a reporting window gets picked, so the only point here is that a rank inherits the flattery of its window.

Ask whether the rank was computed once, at the end, by the agency, or reported by a third party on a schedule. Then run a one-line test: if the period were doubled in either direction, would the same account still be in that position?

A quick way to run the check

Claim as written What is missing What to ask for
“Top 1 percent”, “number one in category” Pool Size, members, who chose them
“Top performer” Metric Rate or count, and the formula
“During the campaign” Period Dates and who computed the rank

A trustworthy version reads like this: a named pool with a size, a named metric, stated dates, and a named source for the ranking. A stated rank at least gives you a pool to question, which is more than an industry average with no source offers.

What to do when the answers are not there

Treat an undefined rank as a description of how the agency feels about the result, not as evidence, and look at the underlying number instead: the actual rate, the actual count, the baseline it moved from. Do not swing to cynicism. A defined, honest pool can still make a good claim. Defining the pool does not make the claim true, only checkable, and the aim is to find the claims that survive being checked.

You can browse the case study library, and if you have a result worth reading closely, submit your own with the pool, metric and period stated.

FAQ

Does “top 1 percent” always mean the top 1 percent of all accounts?

No. It means the top 1 percent of whatever group the writer used, and that group is rarely stated. A defined pool gives a size, a selection method and who selected it.

Is “number one in its category” better than a percentile?

Not automatically. Rank 1 of a small or loosely drawn category is easy to reach, and the example above shows 1st of 60 is only the top 1.7 percent. The same three questions apply: pool, metric, period.

Sources

Similar Posts

Leave a Reply

Your email address will not be published. Required fields are marked *