Skip to main content
Errors in variant code, in ABTestly’s own runtime, or in things like Global JS are captured and reported. The design goal is to give you enough signal to fix a broken variant without flooding the ingest pipeline with duplicates.

​
The /err beacon

Every error goes to POST /err. The payload is:

​
First-occurrence-per-session dedup

The snippet keeps a per-session set of (phase, experimentId) tuples that have already reported. The first occurrence sends. The second, third, and Nth occurrences in the same session do not. This exists because a broken variant that runs on every page load would otherwise flood the endpoint. One report per session is enough signal for diagnosis without the flood.

​
The phases

  • apply, an error inside onApply.
  • cleanup, inside onCleanup.
  • route-handler, inside the runtime’s SPA route change handler.
  • variant-js, inside the variant’s top-level JS (not wrapped in a handler).
  • async, a rejected Promise the variant kicked off with no .catch.
  • global-js, inside the Global JS block.

​
Where errors show up on your side

  • Sentry, if you have SENTRY_DSN configured (Enterprise), errors land in your Sentry project.
  • Dashboard health panel, the site’s Snippet health panel shows a count of recent errors and lets you drill into the most recent ones with their stacks.
  • Dev debugger, window.__abtestly.debug() on the affected page shows the error in the events buffer alongside the events that preceded it. Best for reproducing.

​
What happens after an error

An error in apply or variant-js does not stop the runtime. The runtime catches, logs, and moves on. Other experiments on the page still get evaluated and applied. The specific variant that threw simply did not apply. An error in global-js also does not stop the runtime. The variants that were expecting Global JS to define something might throw themselves as a consequence, but each is caught independently.

​
Errors from your own analytics adapter calls

If your GA4 or Segment or Amplitude SDK is missing or throws, the analytics adapters catch it and report as an error under phase variant-js (the wrapping is conservative). One first-occurrence report per session.

​
What is not in /err

  • Bucketing decisions. “Visitor did not enter test” is not an error; it is a normal decision. Debug those with the dev debugger.
  • Failed beacons. The /e beacon uses sendBeacon with a keepalive fetch fallback. If both fail, the runtime does not self-report, that would require the same broken network to succeed at /err.
  • User-perceived issues. “The layout looks wrong” or “the button is off-center”, those are QA problems, not runtime errors. Use the QA overlay.

​
Rate limiting

/err is not user-authenticated, so the dedup on our side is the only rate limit. If you push a variant that throws on every page load, expect one report per (session, experiment), a burst of new visitors in the first minute after a bad publish will produce a burst of reports. If you know you are pushing something risky, republish quickly when you notice, the dedup resets per session, so subsequent visitors will not report against the fixed version.