August 2026 — coverage sweep
This month closed the gap between what ABTestly ships in the product and what the docs actually describe. Twenty-one areas got real pages.Added
Documentation site map + trust page + REST API
- Documentation index. One-page site map, linked from Getting started, for readers who know what they want and do not want to scan the sidebar.
- Trust and compliance. Subprocessors, retention grid, security posture, GDPR/CCPA rights, uptime commitment. Sourced from privacy.html and terms.html so nothing overclaims.
- REST API reference. Every
/api/*endpoint behind api.abtestly.com, extracted from worker source. Auth is Clerk JWT; there is no long-lived API key today. - Sample-size calculator cross-links from reading a result and time to decision.
Ledger, results, and integrations deep dives
- The three statistics engines. Frequentist, sequential, Bayesian. Method IDs. When to pick which.
- Revenue metrics. CR, RPV, AOV, OPV. Each with a proper CI. Refunds and orderId dedup handled.
- Exact Experiment Ledger. The canonical event store your final numbers come from. Not a sample.
- Sample Ratio Mismatch. Chi-square on every experiment. Warns before a bad split becomes a false winner.
- Speed guardrail. Per-device p75 LCP. Ship a winner, not a broken page.
- Business impact projection. Turn a lift into currency at your current traffic.
- Time to decision. Honest runway estimate; not a promise.
- Baseline. Compare treatments head-to-head.
- Segments. Slice the whole report by device, source, geo.
- Exports. CSV for a stakeholder deck, or async raw-event dump.
Full developer reference track
- window.abtestly API. trackGoal, onApply, onCleanup, waitFor, activate, debug.
- Beacon protocol. Wire format for every public endpoint the snippet hits.
- Bucketing. MurmurHash3, two-step allocation, per-experiment salt. Reproducible in Node.
- Runtime bindings. Every name available inside variant JS.
- SPA lifecycle. Cleanup, re-eval, apply. ApplyContext, routeRevision.
- Errors and telemetry. What lands in /err, first-occurrence dedup, where to see it.
- Rate limits. 100 requests per minute per authenticated user on /api. Beacons are not rate-limited.
Mutual exclusion groups get their own section
- Overview. Why groups exist, when to reach for one.
- How allocation works. The 10,000-slot math, the salt, the generation counter.
- Long-term holdouts. Measure the cumulative effect of every test you have shipped.
- Managing membership. Add, retire, reorder. What generation bumps do to running experiments.
Team and access, in the depth this feature deserves
- Members and roles. Admin, Dev, Viewer. Per-site scope. Twenty per-site permission keys per member. Four presets. Companion enforcement rules. Common team shapes.
- Permissions catalog. Every key, what it unlocks, which role it requires.
- Activity log. Every meaningful change, grouped by day.
Site settings, assets, targeting, preview and QA
- Site settings: general, snippet panel, analytics adapters, SPA panel, Global JS, Blocked IPs, Danger zone.
- Assets: upload, serving, recycle bin.
- Targeting: overview, URL rules, audiences, geo, saved lists, activation triggers.
- Preview and QA: preview modal, forced variation, QA overlay, dev debugger.
Changed
Retention alignment across docs and privacy policy
- results/exact-ledger and billing/refunds corrected from 24 months to 90 days for raw events, matching the privacy policy commitment. Summary numbers (verdicts, per-day rollups) still persist for the life of the experiment.
- trust-and-compliance added a row for the 24-month team activity log retention window, and corrected support conversations to 3 years to match privacy policy §7.
- The public privacy policy at abtestly.com/privacy gained an explicit bullet for team activity log retention.
Snippet size and cryptographic wording, corrected
- Introduction now cites the actual snippet sizes: 0.8 KB inline loader plus 15 KB runtime, both gzipped. Cold-edge first-fetch caveat added.
- Quickstart rewrote the “cryptographically tied to one domain” line to the honest version: the snippet carries a site-specific key and is validated against its configured domain at runtime.
- exports corrected a stale reference to a nonexistent long-lived API key.
Look and feel
Reading-comfortable theme
- Light background moved from stark white to a warm cream (#f2ede1).
- Dark background moved from near-black to a softer blue-gray (#262935).
- Both meant to be less harsh for long documentation reads.
Older entries
This changelog started in August 2026. Before that, the docs were a smaller flat set. Nothing older is worth re-listing here; the current documentation index is the canonical map.Something missing or wrong
Emailsupport@abtestly.com with the URL and what you expected to
see. That is the fastest way for us to fix it.