How to set up an AB test using the Didomi platform?

Didomi offers an AB test feature which allows you to test two different notices on two groups of users in order to compare the results obtained. 

In the light of the recommendations by the French data protection authority (CNIL), obtaining the user’s consent is required to use these tests. The same is true in other European countries.

Therefore, in order to comply with legal recommendations, we recommend that you consult your legal department before using our AB test functionality, to find out if you have their authorization.

What is an AB test?

An AB test allows you to compare consent rates of two groups of user population, the control group and the test group. For instance, your control group will see a pop-in format banner on your website, whereas your test group will see a bottom banner.

The Didomi SDK allows you to run AB tests to compare the consent rates and other metrics of two groups of populations (control and test group). We support running AB tests through your preferred vendor or through the Didomi platform.

How to set up an AB test using the Didomi platform?

In order to set up an AB test using the Didomi platform, you need to go to the "CUSTOMIZATION" step of the creation of your consent notice and then click on the "Advanced settings" section. You will be able to add some code in the custom JSON window.

You need to add an experiment in this window. An experiment is composed of several

parts : 

• The ID : You need to name your test in order to be able to distinguish it (use lower cases letters and only dashes as special characters).
• The size : You need to indicate the desired size of your test group (percentage between 0 and 1). Keeping in mind that the size of the control group will be (1 - size).
• The configuration : You have to indicate which extra configuration will be applied to your test group.
• The start date : It has to appear with this example format "2019-03-06T23:38:50Z". It is important to indicate the start date to ensure that you are showing the new configuration to new users.

The control group corresponds to the group that sees your default configuration, whereas the test group is the one seeing the default configuration PLUS the specific one for them.

📑 The developers documentation will give you examples of experiments. The examples are in Javascript, for customers not using the Didomi console. But if you are using the "custom json" field of the console to implement your AB test, this is an example in Json language :

{
  "app": {
    "apiKey": "<Your API Key>",
    "name": "My website",
    "vendors": {
      "iab": {
        "all": true
      }
    }
  },
  "notice": {
    "position": "bottom"
  },
  "experiment": {
    "id": "test-notice-format",
    "size": 0.2,
    "startDate": "2019-03-06T23:38:50Z",
    "config": {
      "notice": {
        "position": "popup"
      }
    }
  }
}

B- What can you AB test?

With the Didomi console, you can AB test :

• The format of your banner (sticky vs pop-in)
• The buttons
• The design of your cookie banner
• etc

We advise you not to test too many variations at the same time as you will not be able to attribute the performance results of your test group to a specific variable. Multivariate tests are not possible in the Didomi console.

Performance results of your AB test(s)

You will be able to see the results of your AB test(s) in the "Analytics" section of the Didomi console.