Payment & Metrics
Z-Test
A statistical significance test used to confirm whether an A/B test result is reliable.
A Z-test is a statistical significance test used to determine whether the difference between two proportions — such as the conversion rates of two landers in an A/B test — is real or plausibly just random noise. It produces a confidence level; results are conventionally trusted at 95% or higher, meaning under 5% probability the observed gap is chance.
Buyers need this because small samples lie constantly: lander B 'winning' 12 conversions to 9 means nothing, and acting on it burns money on coin-flips. Free significance calculators run the Z-test from your visitor and conversion counts in seconds. The discipline is deciding sample size and test duration before launching, then not peeking and stopping early when a variant temporarily leads — early stopping is the most common way teams ship losers dressed as winners.
In buyer speech
“Don't kill the control yet — the Z-test says we're only at 87% confidence, give it two more days of traffic.”