Skip to main content
Every experiment measures at least two things: the thing you want to improve, and the things you cannot afford to break. The first is the primary goal. The others are guardrails.

​
Primary

The primary goal decides the verdict. When the results page says “Variant B wins”, it means “wins on the primary goal”. Only one goal can be primary; pick it in the experiment editor under What to measure. Pick the primary goal that maps to the thing you actually want to change. For a checkout test, primary is usually a purchase or a checkout-completion event. For a landing page test, it’s usually the CTA click that leads into the funnel, not the eventual purchase, which happens far downstream and adds noise. Rule of thumb: the primary should fire on a large-enough share of visitors that the confidence interval closes in a reasonable time. Very-downstream events with tiny fire rates take forever to call.

​
Guardrails

Guardrails are goals you do not want to win, you want them to not break. A win on the primary goal is checked against every guardrail before the results page calls a winner. Common guardrails:
  • Increase Engagement and Decrease Bounce Rate, attached automatically. See System defaults.
  • Revenue per visitor, for tests near the funnel, to catch a “clicks more but buys less” pattern.
  • A downstream conversion, for early-funnel tests, to catch the same pattern one step later.
On the results page, each guardrail gets its own row. The verdict copy calls out any guardrail that got worse, even if the primary won.

​
The math is the same

Guardrails use the same statistics engine as the primary. Frequentist, sequential, or Bayesian, whichever engine you picked for the experiment applies to all its goals. The confidence bar for a guardrail is read as “can we rule out a bad effect?”, not “is variant beating baseline?”. A guardrail that says “no meaningful effect” is a good outcome, it means the primary win did not come at that guardrail’s expense.

​
What to do with a guardrail-breaking win

Three options:
  1. Ship anyway if you understand the tradeoff. A test that lifts revenue by 5 % but drops engagement by 1 % might still be net positive, depends on what engagement means for your business.
  2. Iterate. Take the winning variant, fix the guardrail problem, re-test.
  3. Do not ship. The lift did not survive contact with a guardrail that matters.
The results page will call it out. It does not decide for you.

​
Do I always need guardrails?

For a first pass, no. The two system guardrails auto-attach and cover the two most common failure modes. If you never add another guardrail, you still catch “the variant is quietly bad for the site” most of the time. Add task-specific guardrails when you know a specific downstream metric is what actually matters, a purchase for a hero test, a return rate for a shipping-info test, a support-ticket rate for a documentation change.

​
Detaching the system guardrails

You can detach them per-experiment (they do not disappear from the library). Detach when you have a good reason:
  • A test on a page that is intentionally a dead end. A “thank you” page has a bounce rate of nearly 100 % by design.
  • A single-page app where the “second page” concept does not fit. Bounce rate defined as time-on-page still works, but you may prefer a task-specific engaged-session goal.
Otherwise, leave both attached. They are cheap and they catch real mistakes.