1. Home
  2. Tools
  3. Statistics and testing
  4. A/B Test Significance Calculator

A/B Test Significance Calculator

Enter visitors and conversions for two variants to get conversion rates, lift, confidence and a clear verdict on whether to ship.

Free, no sign-upRuns in your browserStatistics and testing

Variant A (control)

Variant B (test)

Result

-

Confidence level

Variant A conversion rate-
Variant B conversion rate-
Relative lift-
Z-score-
P-value (two-sided)-

How the calculator works

Enter the visitors and conversions for your control (A) and your test variant (B). The calculator works out each conversion rate, the relative lift of B over A, and runs a two-proportion z-test to tell you how confident you can be that the difference is real and not chance.

How to read the result

  • 95% confidence or more: the result is significant. Ship the winner.
  • 90% to 95%: the result is trending. Keep testing.
  • Below 90%: not significant. Run the test longer or send it more traffic.

Sample size matters

The most common A/B testing mistake is calling a winner too early. With 200 visitors per variant you cannot reliably detect a 10% lift. Decide your sample size before you start, let the test run for at least one full week to cover weekday and weekend behaviour, and avoid stopping the moment the numbers look good.

Frequently asked questions

What is the difference between confidence and the p-value?

They describe the same test from two angles. The p-value is the chance of seeing a difference this large if the variants were really the same. A two-sided p-value under 0.05 is the usual bar for significance.

Can I check the results every day?

You can look, but do not stop the test as soon as it crosses 95%. Checking repeatedly and stopping early inflates false positives. Stick to the sample size you planned.

Does a significant result guarantee the lift will hold?

No. The measured lift is an estimate, and real-world results often settle lower. Treat big lifts on small samples with caution and retest important changes.

Related tools

All tools
Year-over-Year Growth CalculatorCompare this year to the same period last year to get YoY growth, the absolute change and a quick verdict on the result.Month-over-Month Growth CalculatorCalculate month-over-month growth from this month and last month, with the absolute change and an annualized rate.7-Day Rolling Average CalculatorSmooth a noisy daily series into a 7-day rolling average so the real trend in your traffic, sales or spend shows through.Standard Deviation CalculatorPaste a list of numbers to get the mean, variance, and sample and population standard deviation in one place.Sample Size CalculatorFind how many visitors each A/B test variant needs from your baseline rate, minimum detectable effect, confidence and power.