A/B Test Significance Calculator
Whether the difference in conversion rate between two versions of a page, email or ad is likely to be real or just chance, with the p-value and z statistic.
Results
The dashed line marks the value you entered.
What this tool does
A test where B converts at 13 % and A at 10 % looks like a clear win, but with a few hundred visitors that gap appears by chance surprisingly often. The two-proportion z-test asks how likely a gap at least this big would be if both versions were really the same; below 0.05 the usual convention calls it significant. Decide the sample size before you start and do not stop the test the moment it crosses the line — peeking repeatedly inflates false positives. The chart shows the p-value for other conversion counts in B.
Formula
z = (p_B − p_A) ÷ √(p̂(1 − p̂)(1/n_A + 1/n_B)) · p-value = 2(1 − Φ(|z|))
Variables
| Symbol | Meaning | Unit |
|---|---|---|
na | Visitors in version A | — |
ca | Conversions in version A | — |
nb | Visitors in version B | — |
cb | Conversions in version B | — |
CA | Conversion rate A | % |
CB | Conversion rate B | % |
UP | Relative change from A to B | % |
Z | z statistic | — |
PV | p-value (two-sided) | — |
SG | Significant at 95 %? | — |
Worked example
- Visitors in version A1000
- Conversions in version A100
- Visitors in version B1000
- Conversions in version B130
- Conversion rate A10.000 %
- Conversion rate B13.000 %
- Relative change from A to B30.00 %
- z statistic2.1027
- p-value (two-sided)0.03549
- Significant at 95 %?Yes
Limitations
- Standard deviation and variance are calculated for a sample (dividing by n − 1). For a full population, divide by n instead.
- The result is an estimate based only on the values you type. Real situations often include factors this calculator does not know about.