A/B test sample size calculator

How long should your test run?

Drag the sliders to see how many visitors you need, and how many days that takes at your traffic.

Your test

%
%
Detects a move from2.0% → 2.30%
Advanced: confidence and power

Confidence level

How sure you want to be that a winner is real. At 95%, there is a 5% chance of calling a win when there is no real difference.

Statistical power

The chance your test catches a real lift of this size. At 80%, a real lift is missed 1 time in 5.

Run your test for

74 days

73,382 visitors total, 36,691 per group.

2-week minimum12 weeks

Smaller lifts take much longer to detect

Duration for every lift size at your traffic. The dot is your test.

1 wk1 mo3 mo1 yr10 yr+1%5%10%25%50%100%

Planning estimate for two equal groups using a normal approximation. It does not adjust for peeking, multiple outcomes, or changes in traffic.

How the sample size is calculated

The calculator uses the standard two-proportion formula for each group:

n = (zα/2 + zβ)² × (p₁(1−p₁) + p₂(1−p₂)) / (p₂ − p₁)²

  • p₁ is your baseline conversion rate and p₂ is the rate after the lift.
  • zα/2 comes from the confidence level (1.96 at 95%) and zβ from power (0.84 at 80%).
  • Duration is both groups combined, divided by daily visitors.

A worked example

With 1,000 daily visitors, a 2% baseline and a 15% lift to detect (2% → 2.3%), you need about 36,700 visitors per group at 95% confidence and 80% power. That is roughly 74 days. Aim for a 30% lift instead and it drops to about 20 days.

Wondering whether the lift is worth the wait? Put a dollar figure on it with the A/B testing ROI calculator.

Questions, answered

How long should an A/B test run?

Until it reaches the sample size for the lift you want to detect, and for at least one or two full weeks so every weekday is covered. Stopping as soon as a result looks significant inflates false positives.

What is a minimum detectable effect?

The smallest relative lift the test is designed to reliably detect. At a 2% baseline, a 15% minimum detectable effect means detecting a move to 2.3%. Halving it roughly quadruples the sample needed.

What confidence level and power should I use?

95% confidence and 80% power are the common defaults. Raise power to 90% when missing a real win is costly, and raise confidence to 99% when a false win is costly. Both increase the sample size.

What if I do not have enough traffic?

Test bigger changes, test closer to the conversion where the baseline rate is higher, or accept a longer test. Tiny tweaks on low-traffic pages rarely reach significance.

Does this count visitors across both variants?

Yes. Daily visitors is the total traffic split evenly between the control and the variant, and the duration covers both groups.