Skip to main content
For a revenue goal, “did the variant win?” has four honest answers. The results page lets you pick between them with a metric toggle on the goal card.

​
The four metrics

​
Conversion rate (CR)

Of the visitors who saw this variant, what fraction converted at least once? CR=visitors who convertedvisitors exposed\text{CR} = \frac{\text{visitors who converted}}{\text{visitors exposed}} Best for: the classic “more people bought” question. Ignores how much each buyer spent.

​
Revenue per visitor (RPV)

Of the visitors who saw this variant, what is the average revenue across all of them (including zeros for those who did not buy)? RPV=total revenuevisitors exposed\text{RPV} = \frac{\text{total revenue}}{\text{visitors exposed}} Best for: the “is this net-positive?” question. The one number a business person usually wants. Confidence intervals on RPV are non-trivial because most visitors are zero and the distribution is heavy-tailed. ABTestly ships a proper mean-difference CI for RPV, see The engines page for the specific method.

​
Average order value (AOV)

Of visitors who converted, what is their average order? AOV=total revenueorders\text{AOV} = \frac{\text{total revenue}}{\text{orders}} Best for: the “did this move upsells?” question. Ignores the change in conversion rate.

​
Orders per visitor (OPV)

Of the visitors who saw this variant, how many orders per visitor? OPV=total ordersvisitors exposed\text{OPV} = \frac{\text{total orders}}{\text{visitors exposed}} Best for: subscriptions or repeat-purchase products where a single visitor can buy more than once. Behaves like CR when almost every buyer buys once, but tells a different story when repeat purchase matters.

​
Picking a metric

Rule of thumb: RPV is the honest default when the test could plausibly change either conversion rate or average order value.
  • CR is fine when you know AOV cannot move, a homepage headline test rarely moves what people put in their cart.
  • AOV alone is misleading, a test that halves conversions but doubles AOV wins on AOV and loses on RPV. Report AOV only alongside CR or RPV.
  • OPV is the right pick for subscription products, or for services where one visitor generates multiple orders per session.

​
The confidence numbers

Each metric gets its own confidence interval. Switching the toggle recomputes them for the metric you picked; the underlying data is the same. RPV and OPV get mean-difference intervals because both are averages of a per-visitor quantity. CR and AOV get proportion-style and mean-of-orders intervals respectively.

​
Currency

ABTestly does not do currency conversion. Revenue is reported in the site’s configured currency (Settings → General). If your visitors pay in multiple currencies, convert to a single currency on your end before firing the goal.

​
Dedup

Revenue values are deduplicated by orderId, a beacon retried after a network blip does not double-count. See the trackGoal API for the shape.

​
Refunds

Refunds subtract from the visitor’s revenue for that experiment. Fire the same goal with refund: true and the same orderId:
The results page nets refunds against original revenue automatically.