Skip to main content

Documentation Index

Fetch the complete documentation index at: https://docs.x.com/llms.txt

Use this file to discover all available pages before exploring further.

title: AB Testing keywords: [“AB testing”, “A/B testing”, “split testing”, “ad testing”, “campaign testing”, “test campaigns”] description: A/B Testing allows advertisers to segment the users they’re reaching on X so that they can understand how best to optimize for campaign performance and …---

​
Overview

​
Introduction

A/B Testing allows advertisers to segment the users they’re reaching on X so that they can understand how best to optimize for campaign performance and gather learnings to inform their marketing strategies. These segments—referred to as user group splits—are randomized and mutually exclusive. With randomization, factors that influence outcomes are equally distributed. In other words, there are no inherent differences between groups or their expected behaviors. Because of this, when a single variation is applied to a user group and not the others, the difference in campaign performance can be attributed to that variation. While it’s possible to test many variations at once, we strongly recommend testing a single variation at a time. This isolates the causal factor for the observed difference in campaign performance. Variations are set at the campaign level. For example, if the advertiser wants to test the efficacy of a new creative, they should create two identical campaigns where the only difference is the creative. In the future, we plan to support variations at the line item level.

​
Use Cases

A/B testing is most often used to support (1) optimization use cases for performance customers who want to understand what works best on X in order to optimize their investment and (2) learning use cases for brand advertisers who want to use learnings to inform their marketing strategy.  The API will support A/B testing for any campaign variable, including: * Creative  * Targeting  * Bid type * Bid unit

​
A/B Testing

A/B Testing allows advertisers to segment the users they’re reaching on X so that they can understand how best to optimize for campaign performance and gather learnings to inform their marketing strategies. These segments—referred to as user group splits—are randomized and mutually exclusive. With randomization, factors that influence outcomes are equally distributed. In other words, there are no inherent differences between groups or their expected behaviors. Because of this, when a single variation is applied to a user group and not the others, the difference in campaign performance can be attributed to that variation. While it’s possible to test many variations at once, we strongly recommend testing a single variation at a time. This isolates the causal factor for the observed difference in campaign performance. Variations are set at either the campaign level or the ad group level. The ad group is set through line item in Ads API,  As an example of an ad group level variation, if the advertiser wants to test the efficacy of a new creative, they should create one campaign with 2 identical ad groups where the only difference is the creative.

​
Use Cases

A/B testing is most often used to support (1) optimization use cases for performance customers who want to understand what works best on X in order to optimize their investment and (2) learning use cases for brand advertisers who want to use learnings to inform their marketing strategy.  The API will support A/B testing for any campaign variable, including: * Creative * Targeting * Bid type * Bid unit

​
Attributes

A/B Tests are represented as nested structures. There are top level fields for the A/B Test itself and an array of user group objects, each with a set of fields describing it. At a high level, every A/B Test must include the following information. * The test duration, represented by the start_time and end_time fields * The level at which the split will occur, represented by the entity_type field * At least two (and at most 30) user groups, each represented as an object in the user_groups array Each user group must include the following information. * The percentage of users that should be allocated to the given user group, represented by the size field * The campaign IDs/line item IDs that should make up the pool of users for the given user group, represented by the entity_ids array Optionally, name and description values can be set for A/B Tests and for user groups. Information about validation rules and other constraints can be found below. Other metadata, such as the ID or the created at time, are also included, but are automatically set by X. An example A/B Test entity for campaign level is shown below. 
{
  "created_at": "2020-12-01T00:00:00Z",
  "created_by": {
    "user_id": "756201191646691328",
    "username": "apimctestface"
  },
  "deleted": false,
  "description": "documentation example",
  "end_time": "2020-12-05T01:00:00Z",
  "entity_type": "CAMPAIGN",
  "id": "hr7l0",
  "name": "first AB test",
  "start_time": "2020-12-01T01:00:00Z",
  "status": "SCHEDULED",
  "user_groups": [
    {
      "id": "p1bcx",
      "name": "first group",
      "description": null,
      "size": "50.0",
      "entity_ids": [
        "f2qcw",
        "f2tht"
      ]
    },
    {
      "id": "p1bcy",
      "name": "second group",
      "description": "second AB test group",
      "size": 50,
      "entity_ids": [
        "f2rqi",
        "f2tws"
      ]
    }
  ],
  "updated_at": "2020-12-01T00:00:00Z",
  "updated_by": {
    "user_id": "756201191646691328",
    "username": "apimctestface"
  }
}
An example A/B Test entity for line item level is shown below. 
{
   "created_by":{
      "user_id":"756201191646691328",
      "username":"apimctestface"
   },
   "name":"Test2e",
   "start_time":"2022-08-15T00:00:00Z",
   "updated_by":{
      "user_id":"756201191646691328",
      "username":"apimctestface"
   },
   "description":"My Second AB test",
   "entity_type":"LINE_ITEM",
   "end_time":"2022-08-30T00:00:00Z",
   "id":"1ul",
   "user_groups":[
      {
         "name":"first group",
         "size":"50.0",
         "description":"first group description",
         "entity_ids":[
            "ij9dh"
         ],
         "id":"4xe"
      },
      {
         "name":"second group",
         "size":"50.0",
         "description":"second group description",
         "entity_ids":[
            "ihng8"
         ],
         "id":"4xf"
      }
   ],
   "status":"SCHEDULED",
   "created_at":"2022-08-11T00:10:50Z",
   "updated_at":"2022-08-11T00:10:50Z",
   "deleted":false
}

​
Usage

The subsections below describe creating and updating A/B Tests. Reading and deleting work like they do with all other Ads API endpoints.

​
Creating

Create an A/B Test using the POST accounts/:account_id/ab_tests endpoint. The endpoint only accepts JSON POST bodies. The Content-Type must be set to application/json. After the advertiser sets up two or more campaigns, an A/B Test can be created. As stated above, A/B Tests must include: test duration, split level, and at least two user groups. Each user group must declare the percentage of users that should be allocated to it as well as the campaign IDs that should make up its pool of users. Each of these is described in further detail below. Test duration: * The start_time and end_time values must * Be in the future (relative to when the A/B Test is created) * Overlap with the campaign/line item flight dates * The test must last at least one day for non-app-based campaigns and last at least five days for app-based campaigns Split level: * The entity_type can be set to CAMPAIGN or LINE_ITEM User groups: * Each user group is represented as an object in the user_groups array * A minimum of two user groups is required * A maximum of 30 user groups is allowed * The size for each user group is set using a string representation of a numeric value between 1.00 and 99.00 * Note: The size values across objects must add up to 100.00 * The campaign IDs should be specified in each user group’s entity_ids array Optionally, set the name and description for the A/B Test or for one of more user groups. The following request creates an A/B Test on the campaign level that lasts four days and has two user groups with 50% of users in each group. The first user group is based on campaigns f2qcw and f2tht; the second user group is based on campaigns f2rqi and f2tws. The request also adds names and descriptions to some parts of the entity. twurl -X POST -H ads-api.x.com “/8/accounts/18ce54d4x5t/ab_tests” -d ’{“end_time”: “2020-12-05T01:00:00Z”, “entity_type” : “CAMPAIGN”, “start_time”: “2020-12-01T01:00:00Z”, “user_groups”: [{“entity_ids”: [“f2qcw”, “f2tht”], “size”: “50.00”, “name”: “first group”},{“entity_ids”: [“f2rqi”, “f2tws”], “size”: “50.00”, “name”: “second group”, “description”: “second AB test group”}], “name”: “first AB test”, “description”: “documentation example”}’ 
twurl -X POST -H ads-api.x.com "/8/accounts/18ce54d4x5t/ab_tests" -d '{"end_time": "2020-12-05T01:00:00Z", "entity_type" : "CAMPAIGN", "start_time": "2020-12-01T01:00:00Z", "user_groups": [{"entity_ids": ["f2qcw", "f2tht"], "size": "50.00", "name": "first group"},{"entity_ids": ["f2rqi", "f2tws"], "size": "50.00", "name": "second group", "description": "second AB test group"}], "name": "first AB test", "description": "documentation example"}'

{
  "request": {
    "params": {
      "account_id": "18ce54d4x5t",
      "end_time": "2020-12-05T01:00:00Z",
      "entity_type": "CAMPAIGN",
      "start_time": "2020-12-01T01:00:00Z",
      "user_groups": [
        {
          "entity_ids": [
            "f2qcw",
            "f2tht"
          ],
          "size": "50.0",
          "name": "first group"
        },
        {
          "entity_ids": [
            "f2rqi",
            "f2tws"
          ],
          "size": "50.0",
          "name": "second group",
          "description": "second AB test group"
        }
      ],
      "name": "first AB test",
      "description": "documentation example"
    }
  },
  "data": {
    "created_at": "2020-12-01T00:00:00Z",
    "created_by": {
      "user_id": "756201191646691328",
      "username": "apimctestface"
    },
    "deleted": false,
    "description": "documentation example",
    "end_time": "2020-12-05T01:00:00Z",
    "entity_type": "CAMPAIGN",
    "id": "hr7l0",
    "name": "first AB test",
    "start_time": "2020-12-01T01:00:00Z",
    "status": "SCHEDULED",
    "user_groups": [
      {
        "id": "p1bcx",
        "name": "first group",
        "description": null,
        "size": "50.0",
        "entity_ids": [
          "f2qcw",
          "f2tht"
        ]
      },
      {
        "id": "p1bcy",
        "name": "second group",
        "description": "second AB test group",
        "size": "50.0",
        "entity_ids": [
          "f2rqi",
          "f2tws"
        ]
      }
    ],
    "updated_at": "2020-12-01T00:00:00Z",
    "updated_by": {
      "user_id": "756201191646691328",
      "username": "apimctestface"
    }
  }
}
For A/B Testing on line item level The major difference between A/B testing on the campaign and the line item levels is the entity_type. We need to set it to ‘entity_type’ = ‘LINE_ITEM’ for A/B tests on the line item level. This applies to all the actions on an already created A/B test below.  Requirements: 1. All line items of the A/B testing campaign must be included in the split test. 2. Only equal splitting is allowed for Line item level. 3. Number of user groups line items allowed in 1 split test must be less than or equal to 5.  4. Only 1 line item per user group.

​
Updating

Update an A/B Test using the PUT accounts/:account_id/ab_tests/:ab_test_id endpoint. The endpoint requires that a JSON blob be sent in the request. The Content-Type