A Brand Lift Percentage Without a Sample Size Isn't a Result

A Brand Lift Percentage Without a Sample Size Isn’t a Result

A case study says 34 percent of survey respondents reported a more favorable opinion of the brand after the campaign. It sounds like a result. It is not one, because the case study never says how many people answered the survey. Without that number, the 34 percent cannot be compared to anything.

The claim that hides its own denominator

A percentage is a ratio: some count of people divided by some other count, multiplied by a hundred. You cannot evaluate a ratio with only the top half visible, only repeat it, which is different from judging it. When a case study reports that 34 percent of respondents showed a favorable opinion shift, a rise in unaided awareness, or a stated purchase intent, that percentage is a fraction whose bottom half is missing.

This differs from a hard-count metric. Clicks, conversions, and followers come with a denominator that lives in the platform’s own reporting, even when the case study omits it. A brand lift percentage has no such backstop: its denominator exists only in a research vendor’s dataset, and nobody outside that room sees it unless someone chooses to publish it.

What a sample size actually buys you

A reported percentage sits inside a range of plausible true values, not a single fixed fact. That range is margin of error: how far the reported figure might sit from what a full census of the target audience would show. A small sample produces a wide range, a large one narrows it, but not in a straight line. The error shrinks roughly with the square root of the sample size, so the first few hundred respondents buy far more precision than the next few thousand do.

Here is a hypothetical, invented for illustration only and not a reported result from any real study or agency. A case study claims 34 percent of respondents reported a favorable shift in opinion. If that came from 20 people, the margin of error is enormous: the true figure could plausibly sit anywhere from a small minority to something close to a majority. The identical 34 percent from 2,000 people is a different claim entirely, with a much tighter range. Same headline, very different evidentiary weight, invisible unless the sample size is printed.

The questions a disclosed sample size still doesn’t answer

Publishing the sample size is a real improvement, but not the finish line. A stated sample size answers one question: how many completed responses fed the percentage. It says nothing about who those respondents were or how they were chosen, a problem covered elsewhere on this site. It also says nothing about response rate: 500 completed surveys out of 550 invited is a different claim than 500 out of 50,000 invited.

How this differs from a sales or conversion percentage

A brand lift percentage needs a different kind of skepticism than a conversion rate. A sales percentage is built on a count the platform logged automatically, a purchase or click that happened or did not. A brand lift percentage is built on people choosing to answer a question about their opinions, which introduces variables a hard count does not carry: who answered, how the question was worded, what mood the respondent was in.

The two should never sit side by side as though they carry the same evidence, even when the percentages look similar. One rests on a system logging behavior it can verify; the other, on people describing their own state of mind, filtered through a survey instrument nobody outside the research team gets to inspect.

What to look for before you accept the number

  • The sample size stated as completed responses, not the number of people invited or exposed to the survey.
  • A margin of error or confidence interval, stated outright or at least estimable from the sample size given.
  • Whether the surveyed population matches the audience the campaign actually targeted, rather than a broader or different group.
  • Whether the percentage is compared to a baseline survey using the same questions and a comparable sample.

Why agencies often leave the sample size out

There are ordinary reasons this number goes missing, and it is worth resisting the urge to assume bad faith. Brand lift surveys are often run through a third party research vendor, and the agency may simply reuse whatever summary format the vendor handed back. In other cases, the sample may be small enough that printing it would undercut the headline percentage. Neither is an accusation; this site cannot know which applies without the agency saying so. [EVIDENCE NEEDED: the sample size and response rate behind the specific survey figure being cited].

Read the next case study with this in hand

The library at Social Case Studies exists so you can practice this kind of reading against real published work, credited to the agencies that produced it. Browse the case study library and look for the sample size behind any survey-based claim. If you run agency work and have a case study worth featuring, get in touch about it.

FAQ

What is a brand lift survey?

A brand lift survey asks people about awareness, favorability, or purchase intent, typically before and after a campaign, to estimate whether the campaign shifted opinion among the people exposed to it. That makes it different from a satisfaction survey or Net Promoter Score, which measure how existing customers feel about an experience rather than whether advertising changed anyone’s mind.

How many survey respondents are enough to trust a percentage?

There is no universal number that makes a brand lift percentage automatically trustworthy. What matters is how large a shift is being claimed and how much precision that claim needs. Work back from the margin of error implied by the stated sample size and judge whether that range is tight enough to support what the case study argues.

Is a brand lift percentage the same as a conversion rate?

No. A conversion rate is built on a platform-logged count that requires no cooperation from the person being counted, while a brand lift percentage depends on people choosing to answer a survey question, and a number is only as good as what backs it up in a way a hard count is not. For the hard-count side of a similar problem, see a percentage built on a small counted denominator, and for the broader framework, see how to read a marketing case study without being misled.

Sources

Similar Posts

Leave a Reply

Your email address will not be published. Required fields are marked *