In this demo car app, clustered charging pins beat the classic map. The winner is one click from being what every driver sees.
Experiments
Test it on some.
Ship the winner to everyone
An experiment serves different values of your app’s variables to a share of your users, measures which one converts better on goals you define from your own data, and turns the winner into the value everyone gets. No release, and no separate testing tool.
Part of engagement
A test you can
take to the decision
Every experiment compares each variant with a control on the goals you set, and says how sure it is.
- How many ideas at once?
8
groups per experiment, the control included
Test up to seven variants against what you serve today. Each one sets the same variables, up to eight, to its own values.
- What counts as better?
3
goals, one of them primary
A goal is a conversion you define from profile properties, cohort membership, or the events people trigger during the test. Each is compared variant by variant.
- Is it really better?
95%
the bar for calling a winner
A variant wins when it is at least 95% likely to beat the baseline and its improvement range clears your target. Otherwise the page says no winner, plainly.
Set up in minutes
Four steps
to a running test
A short wizard, saved as a draft until you start it.
- 01
Basics
Name it and choose when it stops: by hand, after a number of days, or once a variant reaches your target improvement.
- 02
Targeting
Enroll a share of your users, everyone or only those matching profile properties, cohorts or profile groups.
- 03
Goals
Up to three conversions to measure, from who people are and what they do during the test.
- 04
Variants
The variables to test and a value for each variant, with the control set to what you serve today.
From result to release
The winner becomes the value.
No ticket, no deploy.
Roll out the winner, or any variant you choose, straight from the results. The values your app fetches change for the people you decide.
Works with
Experiments, with the
rest of Countly
Every feature reads the same live data, so an answer in Experiments doesn’t stop there. These are the handoffs built in.
All featuresData ownership
Tested on your users,
measured on your data
Enrollment, values and conversions all run on the server that holds your analytics, so there are no experiment events to send anywhere else.
Data privacyOne less vendor
Experiments use the same SDK, profiles and cohorts as your analytics and your variables.
Self-hosted, or dedicated
Run Countly in your own cloud, your data center or an air-gapped network. Or let us run a single-tenant instance in the Google Cloud region you choose.
No default vendor access
On Enterprise, Countly has no access unless you grant it. On Flex, access is limited, granted by you and logged.
*Countly Ltd. is a TISAX® Participant. TISAX® is a registered trademark of and governed by ENX Association. Learn more at enx.com/tisax