> ## Documentation Index
> Fetch the complete documentation index at: https://docs.abtestly.com/llms.txt
> Use this file to discover all available pages before exploring further.

# The numbers look wrong

> Conversion count higher than visitor count. Two arms with wildly different sizes. Lift that flip-flops. A ladder for each.

Weird results are almost always a data issue, not a statistics one.
Walk this list before rejecting a verdict.

## Conversions higher than visitors

You see 200 visitors and 250 conversions on one variant.

**Cause:** the goal counts fires instead of unique converters. That is
usually a revenue goal (revenue *is* summed across fires) or a custom
event set to "count all fires".

**Fix:** check the goal's definition. Switch the goal type if you
want unique converters.

## Two arms have wildly different sizes

Split is 50/50 but the results show 60/40 or worse.

**Almost always** a Sample Ratio Mismatch. See [SRM](/results/srm) for
the full write-up. Short version:

* Preview links poisoning the data.
* A click activation with a selector that only exists in one variant.
* A redirect variant firing an exposure then navigating away before
  the beacon lands.
* An analytics adapter blocking the beacon on one arm's page.

## Lift flip-flops daily

Yesterday Variant B was +2 %. Today Variant A is +1 %.

Almost always **too small a sample**. The CI is wide; the point
estimate is bouncing around inside it. A converging test stops
flip-flopping.

**Fix:** wait longer, or accept that the true effect is very small
and probably not worth calling.

## Verdict says "no effect" but I see a real change on-site

Two possibilities:

1. **The change is not what the primary goal measures.** You
   redesigned the hero, but the primary goal is checkout completion
   six pages downstream. Downstream goals move slowly and are hard
   to attribute.
2. **The change genuinely does not move the primary.** A pretty
   change that does not affect behavior. Happens more than people
   admit.

## Revenue does not match my analytics

GA4 shows $10,000 in the same period. ABTestly shows $8,000.

Neither is wrong; they measure different things.

* ABTestly counts revenue tagged to an experiment's exposed
  visitors. Non-bucketed visitors are not in the total.
* GA4 counts all revenue, regardless of experiment membership.

Trust ABTestly for **experiment-attributable** revenue and GA4 for
**total** revenue.

## Results page shows "Processing"

The [Exact Ledger](/results/exact-ledger) is catching up on a burst
of events. Dashboard live counters are current; the ledger-driven
verdict is held while the queue drains. Usually seconds.

**Fix:** wait a minute. If it does not clear in ten minutes, email
support.

## Everything looks fine but I do not believe the winner

Sensible skepticism:

1. Check the CI width vs the effect size. A "9 % lift" with a CI of
   `[-4 %, +22 %]` could easily be zero.
2. Check the guardrails. A win with a broken guardrail is usually
   not a real win.
3. Re-run the test. Winners that survive replication are winners;
   ones that do not, were not.