A reliable creative test has: a hypothesis (“a hook with price increases quote requests”), a single variable, a business metric (cost per qualified request, not clicks), enough budget and duration (one to two conversion cycles, according to Google), a decision rule written beforehand and a record. With AI generating variations (Meta speaks of up to 150 with Advantage+, and nearly 10 million advertisers used AI creative tools, according to Meta), the risk is testing everything and learning nothing. Meta’s case study of 20% more conversions is the agency’s data, with no control group.
The problem: more variations, less learning
With AI generating dozens of variations, testing everything seems easy. But testing everything at once, with a small budget, gives noisy results: each variation receives little data, and the system favours the one that looks better early, by chance. The result is the illusion that you found the winning creative.
Learning demands clear hypotheses and fewer variations with more data.
Start with a hypothesis, not a creative
A good hypothesis says what changes, why and what you expect: “Showing the starting price in the video reduces browsers’ contacts and raises booking rate”. Write it beforehand. Without a hypothesis, each result is interpreted after the fact, and any pattern looks like a lesson.
The hypothesis also protects the brand: it defines what can and can’t be said.
Reliable creative test
- Written hypothesis
- A single variable
- Business metric
- Sufficient sample and duration
- Decision rule and record
One variable at a time
If you change hook, price, image and call to action at once, you don’t know what caused the difference. Choose one variable (hook, format, proof, offer, call to action) and keep the rest equal. If the platform generates automatic variations, define what stays fixed and use the account’s testing tools when they exist.
First the highest-impact variables (offer and hook), then the details.
The right metric
Clicks and views are easy to optimise and misleading. Use the metric closest to the business that you can measure: cost per qualified request, booking rate, cost per sale, margin. In video, bear in mind Google began counting engagement after 10 seconds from 10 October 2026, according to PPC News Feed, so don’t compare engagement between periods with different definitions.
The creative that attracts browsers is the worst deal.
Inconclusive is information. A false winner is a cost.
Sample and duration: how much is enough
There’s no magic number, but there’s a practical rule: wait for enough conversions in each variation so the difference isn’t noise, and let it run one to two conversion cycles, which is Google’s guidance for bidding, according to PPC Land. Avoid concluding after three days. If volume is small, test big, evident differences, and accept that some tests will be inconclusive.
Inconclusive is information.
Decision rule written beforehand
Before starting, write: “If variation B has cost per request at least 15% lower with equal quality, we adopt B; if within 10%, we keep A; otherwise we repeat”. This avoids deciding by enthusiasm or pressure. Record the result, even when it fails. A sheet with hypothesis, variable, metric, duration, result and decision is enough.
Records form the account’s memory.
Platform claims: treat as hypotheses
Meta credits AI ads with 20% more conversions across seven UK brands in an agency case study, with no control group, according to PPC Land, and speaks of nearly 10 million advertisers using AI creative tools. They aren’t proof for your account. Test in yours, with your margin.
The same goes for any global “best practice”: it’s a hypothesis to test.
Responsibility and approval of what AI generates
According to PPC Land, from 30 October Meta makes the advertiser responsible for what AI creates or alters in ads. So each generated variation should be approved before publishing, and those promising what you can’t deliver should be removed. Record who approved. Production speed can’t exceed review capacity.
Fewer variations, more control.
Summary and questions for your manager
Summary: written hypothesis, one variable, business metric, sufficient duration, decision rule beforehand, record and approval of AI-generated variations. Questions: which tests are running now? What hypothesis does each have? What metric and decision rule? Who approves AI variations?
If you want to set up a creative testing plan with a record sheet, I can help structure it.
An illustrative well-designed test example
A shoe brand tests two versions of the same video: A opens with the product, B opens with the starting price. Same budget, same audience, same page, six weeks. The metric is cost per confirmed purchase. Rule: B only wins if it costs at least 15% less with equal returns. The example is hypothetical. Result: B costs 9% less, within noise; the brand keeps A, records the learning and tests another hook.
The test didn’t “fail”: it saved a wrong decision.
Frequently asked questions
How many variations should I test?
Few, with clear hypotheses. Many variations with small budget give noisy results.
How long should a test run?
At least one to two conversion cycles, per Google’s guidance cited by PPC Land.
Can I trust Meta's 20%?
It’s an agency’s data, with no control group. Test in your account.