> ## Documentation Index > Fetch the complete documentation index at: https://docs.abtestly.com/llms.txt > Use this file to discover all available pages before exploring further. # Confidence intervals > Four different intervals appear on a results page, computed four different ways. The formulas, why the rate interval is Wilson and the lift interval is Wald, and why every one of them is printed next to its n. A point estimate is a number without a scale. "Plus 12.5 %" from 40 visitors and "plus 12.5 %" from 40,000 are the same characters describing completely different states of knowledge, and the only thing that separates them is the interval and the denominator underneath it. Four intervals appear on a results page. They are computed four different ways, on purpose, and this page says which is which. ## 1. The conversion rate of one variation Wilson score interval, `wilsonCI` in `dashboard/src/lib/wilson.ts`, at `z = 1.96`. ``` p = conversions / visitors denom = 1 + z^2/n center = (p + z^2/(2n)) / denom half = z * sqrt( p*(1-p)/n + z^2/(4n^2) ) / denom CI = [ max(0, center - half), min(1, center + half) ] ``` Wilson rather than the textbook Wald interval, because Wald breaks exactly where a CRO test spends its first day. At zero conversions Wald collapses to \[0, 0], which reads as certainty that the rate is zero. Wilson does not: ``` 0 of 40 -> [0.0000, 0.0876] 3 of 40 -> [0.0258, 0.1986] ``` The first interval says what you actually know from 40 visitors and no conversions, which is that the rate is probably under 9 %. One Wilson implementation serves the per-variation cards, the CI bar and the goal-breakdown table, so those three never disagree. ## 2. The relative lift against control `relLiftCI` in `dashboard/src/lib/report-export.ts`. A normal approximation on the difference of two proportions, divided through by the control rate. ``` SE = sqrt( p0*(1-p0)/n0 + p1*(1-p1)/n1 ) // unpooled diff = p1 - p0 lo = max(-1, (diff - 1.96*SE) / p0) hi = (diff + 1.96*SE) / p0 ``` The lower bound is floored at minus one. A non-negative metric cannot fall by more than its whole baseline, so a lift under minus 100 % is not a conservative reading, it is an impossible one, and the normal approximation will produce it on low-conversion goals. The upper bound is not floored, because a lift can genuinely exceed plus 100 %. The function returns nothing at all when the control rate is zero (the relative lift is undefined, not zero) or when either arm has no visitors. Both rates are clamped to at most 1 before the arithmetic, so a converters-above-visitors data artifact produces a wide interval rather than a NaN. The same interval, under the name `liftConfidenceInterval`, is what the public [A/B test calculator](https://abtestly.com/ab-test-calculator) runs. ## 3. The absolute difference, at your confidence level The first two intervals are display objects, fixed at 95 %. The verdict carries its own interval, and that one honours the experiment's configuration. From `frequentistVerdict` in `worker/src/lib/stats-engine.ts`: ``` familyAlpha = (100 - confidenceLevel) / 100 // 95 or 99 comparisonAlpha = familyAlpha, split by the correction z = PHI_INV(1 - comparisonAlpha/2) SE = sqrt( pv*(1-pv)/nv + pc*(1-pc)/nc ) interval = (pv - pc) +/- z * SE ``` The correction is one of none, Bonferroni or Sidak, and it narrows the per-comparison alpha when several variations are tested against one control. The default configuration is 95 % with no correction. One detail a statistician should know before quoting these numbers. The significance test uses a **pooled** standard error, computed under the null where both arms share one rate. The displayed interval uses an **unpooled** standard error, computed under the alternative where they do not. That is the standard pairing, and it is deliberate: the test statistic and the interval are answering different questions. It also means that in a narrow band around the threshold the p-value and the "does the interval clear zero" reading can disagree slightly. Neither is wrong. They are different estimators. ## 4. Revenue per visitor Revenue is not a proportion, so none of the above applies. From `meanConfidenceInterval` in `worker/src/lib/stats.ts`, working from running moments rather than stored observations: ``` mean = sumX / n var = max(0, sumXX/n - mean^2) se = sqrt(var / n) CI = mean +/- 1.959964 * se ``` `n` is visitors, not orders, because the metric is revenue per visitor. The function returns nothing when `n < 2`: one observation carries no dispersion, so any interval built from it would be invented. The variance is clamped at zero so catastrophic cancellation and genuinely degenerate arms both collapse to an honest zero-width interval instead of a NaN. The bounds themselves are not clamped, and a negative lower bound on a noisy revenue arm is a true statement about your data. The difference of two revenue arms uses a Welch (unpooled) standard error in `meanDifferenceCI`: ``` SE = sqrt(seA^2 + seB^2) diff = meanA - meanB CI = diff +/- z * SE ``` For revenue there is no separate significance test. The interval excluding zero **is** the frequentist signal, which is why a revenue lift changes colour on the results table exactly when its interval clears zero. A documented simplification. When an experiment has refunds, the revenue interval is centred on the refund-adjusted revenue per visitor that the card displays, while its **width** still comes from the raw order-value moments. Refund variance is not modelled. It is a second-order effect on the width, and we would rather name it than let you assume otherwise. ## A worked results row Control converts 384 of 12,000. The variation converts 432 of 12,000. | | Visitors | Conversions | Rate | Rate CI (Wilson) | | --------- | -------- | ----------- | ------ | ---------------- | | Control | 12,000 | 384 | 3.20 % | 2.90 % to 3.53 % | | Variation | 12,000 | 432 | 3.60 % | 3.28 % to 3.95 % | ``` lift = (0.036 - 0.032) / 0.032 = +12.50% unpooled SE = sqrt(.036*.964/12000 + .032*.968/12000) = 0.00233952 lift CI = (0.004 +/- 1.96 * 0.00233952) / 0.032 = -1.83% to +26.83% absolute CI = 0.004 +/- 1.959964 * 0.00233952 = -0.059 pp to +0.859 pp pooled z-test p = 0.0873 ``` The lift is positive and the interval that describes it straddles zero, so there is no verdict here. That is the whole point of printing both. Note what the rate intervals do **not** settle. They overlap, and people routinely read overlapping per-arm intervals as proof of no difference. That inference is not valid in general: two intervals can overlap while the interval on their difference excludes zero. The comparison is answered by rows 2 and 3, never by eyeballing row 1. ## Why the denominator is always next to the interval The interval is a deterministic function of the counts. Print it without its `n` and a reader cannot tell a wide interval caused by a small sample from a wide interval caused by a genuinely noisy metric, and those call for opposite responses. The results table therefore carries visitors, conversions, rate, rate CI, lift, lift CI and confidence on one row, and the CSV export carries the same columns. Two floors back that up. The significance verdict is withheld unless both the variation and its control have at least 100 visitors and at least 5 conversions, so the confidence column stops quoting a percentage off a handful of visitors. The rate, interval and lift are never withheld, because they are honest at any sample size and the interval is the thing that says so. The revenue-per-visitor interval uses the same 100-visitor and 5-conversion bar, since a revenue interval is at its noisiest exactly where that bar is not met. ## What a confidence interval is not It is not the probability that the true value lies inside it. Under the frequentist reading, the true value is fixed and the interval is what is random. A 95 % interval is a procedure that brackets the truth in 95 % of repetitions. The sentence people actually want, of the form "there is a such-and-such probability that this variation is better", is a posterior probability with a credible interval beside it, and that comes from the [Bayesian engine](/results/engines). It is also not a guard against bias. An interval describes sampling noise and nothing else. A test with a [sample ratio mismatch](/methodology/sample-ratio-mismatch), a goal that only fires on one variation, or an instrumentation gap will produce intervals that are perfectly well formed and centred on the wrong number. ## Sources The interval construction here is textbook. The reason we show the denominator beside every interval is not. * [Kohavi, R., Tang, D. and Xu, Y. (2020), *Trustworthy Online Controlled Experiments: A Practical Guide to A/B Testing*, Cambridge University Press](https://www.cambridge.org/core/books/trustworthy-online-controlled-experiments/D97B26382EB0EB2DC2019A7A7B518F59). The standard treatment of intervals, variance and what a confidence interval does and does not claim. * [Deng, A., Xu, Y., Kohavi, R. and Walker, T. (2013), *Improving the Sensitivity of Online Controlled Experiments by Utilizing Pre-Experiment Data*, WSDM '13](https://exp-platform.com/cuped/). On tightening intervals with pre-experiment data. ABTestly does not implement CUPED today; it is included because it is the usual next question once the width of an interval starts to matter. Every link above was checked on 6 September 2026. ## Related The precondition an interval cannot check for you. Confidence interval, confidence sequence and credible interval are three different objects. Where each of these lands on the page. The same lift interval and the same z-test, on your own numbers.