A/B tests
An A/B test shows half of a page's visitors the page as it is (Control) and the other half a version with different words (Variant), then counts which one gets more of them to the step you care about. Set tests up in Website → A/B tests. A/B tests are included in every Echo plan with the Website package.
How a test works
- Pick a page. In Website → A/B tests, use + → New A/B test… and choose the page. You can also right-click a page in Website → Content and choose Create A/B test. A test needs a page whose text lives in the CMS (most pages of a site on the Allsite CMS).
- Write the Variant. Edit variant lists the page's text. Change the headline, a button, a price note or any other text you want to test; what you leave alone stays the same as Control. The preview shows both versions.
- Add conversion steps. Add step sets what counts as a success. A step is one of: - Click: a click on an element with a tracking id, such as a Book demo button marked
data-echo-event="book-demo". - Page view: a visit to another page, such as/demo/success. - Form submit: a form being sent, by its name. Steps run in order, so two or three steps make a funnel (for example View /pricing, then Start free, then Sign up). The last step is the conversion the test is judged on. - Start the test, then publish. Start test adds the test to the site's settings as a change. Publish puts it live; from then on every new visitor is sent to one variant and stays on it.
- Read the results and Stop test when you have your answer. Stopping and publishing shows every visitor Control again. To keep the winning words, change the page itself in Website → Content.
What you see
- Status: Pending (not started), Running or Stopped.
- Winner: Pending while there isn't enough evidence, the winning variant once one is clearly better, or No clear winner when a test stopped without one.
- Distribution: how visitors are split. Even sends half to each; Edit sends more or fewer to the Variant.
- Summary: the result in one sentence.
- The funnel: how many visitors saw the page and how many reached each step, with the share that went on from the step before.
- The table: per variant its views, events (visitors who reached the last step), conversion, lift (how much better or worse than Control) and Best, the chance that this variant really converts best.
- Over time: each variant's conversion so far, with the range its true rate is likely in.
- Filter… narrows everything to a device or a country.
How Echo decides
Echo compares the two conversion rates the Bayesian way: from the visitors and conversions so far it works out the chance that each variant has the better true rate. That is the Best column. Echo names a winner when one variant reaches a 95% chance and each variant has had at least 100 visitors. Small differences need many visitors, so let a test run at least a week, through quiet and busy days, before you act on it.
What it needs
- The Website package on your plan. A/B tests are included in every tier that has it.
- The Echo tag on the site (Tracking → Tags), which counts the views and steps. See The Echo tag.
- The SEO website right to make, change and start tests, and the Publish right to put them live. See Edit your website.
Good to know
- The Variant changes the page's words and the text of its buttons, not its layout. For a different layout, build a second page and test it with a page view step.
- Every visitor keeps their variant on later visits. Visitors who ask not to be tracked (Do Not Track, Global Privacy Control) always see Control and aren't counted.
- Search engines and AI crawlers see Control, so a test doesn't change what they read.
- Add
?acms-ab=bto the page's address to see the Variant yourself, or?acms-ab=afor Control.
Set it up from Claude or ChatGPT
With Echo connected to Claude or ChatGPT, you can ask in words, for example "Create an A/B test on /pricing with a Book demo click step". The test is saved as pending, and you start it in Echo. See Echo in Claude and ChatGPT.
Something wrong or out of date? Write to hello@echo-aeo.com and we'll fix it.