Geo experiments with few regions: compensating for the lack of comparison points

Abstract orange illustration on a dark background: a location pin. Article topic: “Geo experiments with few regions: compensating for the lack of comparison points”
In brief

Geo tests compare regions with and without ads to measure the real effect. They work best with many comparable regions. In small countries, or with businesses concentrated in a few areas, there are fewer comparison points. Even so, there are ways to design useful tests, with caution in the conclusions.

The essentials in a definition

Geo experiments with few regions are tests where ad exposure is changed in some areas and results compared. With few regions, statistical power is lacking and there is a risk of comparing different areas. It is offset by careful selection of comparable regions, long series of data before the test, synthetic-control methods and prudent reading. Models such as Meridian, measurement and Think with Google studies help.

The problem of having few regions

With two or three regions, any difference may be caused by local factors (weather, events, competition) rather than the ads. Natural variation between regions is large compared with the effect you want to measure. Results become fragile.

This does not prevent testing, but calls for more prudence.

Ways to compensate

Extend the test period and use several phases (on, off, on again), to see whether the effect repeats. Use smaller units (districts, municipalities, postcodes) to have more points. Compare each region with itself, before and after, and with similar control regions. Adjust for seasonality.

Record everything that changed simultaneously.

Illustrative diagram with 4 numbered steps joined by an orange line, the last one marked as done. It represents: Geo tests with few regions

Geo tests with few regions

  1. Use smaller geographic units
  2. Repeat on and off phases
  3. Compare before and after with controls
  4. Describe the result as a range

Alternatives

Tests by audience segment (excluding part of the users), by hours or by products, instead of by region. Simple time-series models that estimate what would have happened without ads, from earlier data. Combine with surveys and qualitative data.

The more sources agree, the greater the confidence.

Large orange quotation marks on a dark background with abstract lines of text, illustrating the quote: “With few regions, repetition replaces volume.”
With few regions, repetition replaces volume.

Reading and deciding

Describe the result as a range, not an exact value. If the effect is clear and consistent across different phases, act. If ambiguous, repeat or use another method. Record assumptions and limitations. Avoid big decisions based on a single fragile test.

Plan periodic repeats.

Decision table

ProblemCompensationNote
Few regionsUse several weeks and comparable pairsLower power; cautious reading
Different regionsChoose regions with similar historyCheck trend before the test
Leakage between regionsPrefer separate regionsThere may be cross-exposure
NoiseRepeat and combine with other evidenceAvoid strong conclusions

Common mistakes to avoid

  • Choosing regions for convenience. It can bias the result.
  • Concluding from one test. Uncertainty is large.
  • Ignoring local events. Promotions and holidays distort.

How to measure the result

Record the trend before the test, the estimated effect, the uncertainty and local events that may have influenced.

Limits and exceptions

With few comparison points, it is hard to distinguish effect from noise, and any regional difference may have another cause. Results should be used with caution and alongside other evidence. If the decision is high-impact, consider more robust methods or specialist measurement support.

Illustrative example

Illustrative example: a brand with four relevant districts chooses the two with the most similar sales history, cuts remarketing in one for four weeks and compares with the other and with its earlier behaviour. The estimated effect is small, but with great uncertainty. The brand repeats the test in another period before deciding.

In practice: checkpoints

Choose regions with similar histories, use longer tests, repeat when possible and combine the results with other methods. Report the uncertainty honestly, and avoid running other big changes in the same regions during the test.

Frequently asked questions

Can I run geo tests in a small country?

You can, with smaller units and several phases.

What alternatives exist?

Audience-exclusion tests, by hours or time-series models.

How do I deal with uncertainty?

By reporting ranges and repeating.

Is it worth testing with few regions?

Yes, as an indication, provided uncertainty is accepted and the test repeated.

What alternatives exist?

Audience holdout tests, time series with control and mix models.

Summary to quote

With few regions, choose comparable pairs, use long series before the test, repeat and read the result as an indication with uncertainty, not proof.

Sources and recommended reading

Last editorial review: October 2026. Informational content; always confirm current conditions in the official sources before deciding.

Keep reading

Quick contact

Want to talk about your campaigns?

Leave a short note. I'll reply within one working day.