---
title: "A/B testing: definition and meaning · Countly"
description: "Showing two variants to comparable groups and measuring which performs better, on the same event data the rest of your analytics uses."
url: https://countly.com/glossary/a-b-testing
last_modified: 2026-11-11
---

[Glossary](https://countly.com/glossary) · Growth

# A/B testing

Showing two variants to comparable groups and measuring which performs better, on the same event data the rest of your analytics uses.

The value of a test is decided before it runs: a hypothesis, one variant, a metric agreed in advance, and a sample size that makes the result mean something. Tests fail most often not because the variant lost but because the team could not say beforehand what winning looked like. Running the experiment on the same event store as the analytics matters here: if the test tool and the analytics disagree on what a conversion is, the result is unarguable in the worst sense.

More in Growth

- [AARRR framework Acquisition, Activation, Retention, Referral, Revenue: five stages used to decide which metric a team should be arguing about this quarter.](https://countly.com/glossary/aarrr-framework)
- [Activation The moment a new user first gets the value the product promised. Everything before it is setup; everything after it is retention.](https://countly.com/glossary/activation)
- [Activation rate The share of new users who reach that moment. The single number most onboarding work is trying to move.](https://countly.com/glossary/activation-rate)
- [Attribution Assigning an install or a conversion to the campaign, channel or referrer that produced it.](https://countly.com/glossary/attribution)
- [Average Revenue Per User (ARPU) Revenue in a period divided by active users in that period. Useful next to retention; misleading on its own.](https://countly.com/glossary/average-revenue-per-user-arpu)
- [Churn rate The share of users or revenue lost in a period. Its shape over time matters more than the number in any single month.](https://countly.com/glossary/churn-rate)
[All 91 terms](https://countly.com/glossary)

Further reading

- [Using Product Analytics Insights for Mobile A/B Testing and Conversion Rate Optimization 19 July 2023 · 8 min read](https://countly.com/blog/ab-testing-cro)
- [A Guide on How to Benefit from Mobile A/B Testing 28 November 2022 · 4 min read](https://countly.com/blog/mobile-ab-testing)
- [The Complete Guide to Events Tracking In Digital Analytics 24 February 2025 · 16 min read](https://countly.com/blog/event-tracking-digital-analytics)
