Skip to content

Running switchback tests

Info

Learn the concepts and fundamentals of switchback tests in the Explanation page.

A switchback test is an experiment that runs in the background and compares the results of two instances: baseline vs. candidate. The experiment switches back and forth between the two instances, when a run is made to the app, hence the name.

This how-to guide explains how to interact with switchback tests using the /v1/applications/{application_id}/experiments/switchback endpoints. Go to the reference section to see all the available parameters for each endpoint.

Create a switchback test

Use the POST /v1/applications/{application_id}/experiments/switchback endpoint to create a new switchback test for an application. A switchback test requires you to define the number of units that it is going to run for, via the units field of the generate_random_plan object. Each unit is a time duration in minutes, defined by the unit_duration_minutes field. Lastly, you must define two instances to compare against each other, using the comparison field: a baseline_instance_id and a candidate_instance_id.

The endpoint also requires an id and name in the request payload, so generate one yourself if you don't want to choose custom values:

SWITCHBACK_TEST_ID="switchback-$(openssl rand -hex 4)"

curl -s -X POST "https://api.cloud.nextmv.io/v1/applications/uncanny-rodent/experiments/switchback" \
  -H "Authorization: Bearer ${NEXTMV_API_KEY}" \
  -H "Content-Type: application/json" \
  -d "{\"id\": \"${SWITCHBACK_TEST_ID}\", \"name\": \"${SWITCHBACK_TEST_ID}\", \"comparison\": {\"baseline_instance_id\": \"production\", \"candidate_instance_id\": \"staging\"}, \"generate_random_plan\": {\"unit_duration_minutes\": 2, \"units\": 5}}" \
  | jq '.'

{
  "id": "switchback-1a51bae6",
  "name": "switchback-1a51bae6",
  "description": "",
  "created_at": "2026-07-29T15:07:10.009201462Z",
  "updated_at": "2026-07-29T15:07:10.009201462Z",
  "status": "draft",
  "comparison": {
    "baseline_instance_id": "production",
    "candidate_instance_id": "staging"
  },
  "plan": {
    "units": [
      {
        "duration_minutes": 2,
        "instance_id": "staging",
        "index": 0
      },
      {
        "duration_minutes": 2,
        "instance_id": "staging",
        "index": 1
      },
      {
        "duration_minutes": 2,
        "instance_id": "staging",
        "index": 2
      },
      {
        "duration_minutes": 2,
        "instance_id": "production",
        "index": 3
      },
      {
        "duration_minutes": 2,
        "instance_id": "staging",
        "index": 4
      }
    ]
  }
}

The call above uses a randomly generated ID, and the same identifier for the switchback test’s name. The name of the test is used as a human-readable label. You can pass different values for the id and name fields in the payload to specify a custom ID and name for the switchback test. For example:

curl -s -X POST "https://api.cloud.nextmv.io/v1/applications/uncanny-rodent/experiments/switchback" \
  -H "Authorization: Bearer ${NEXTMV_API_KEY}" \
  -H "Content-Type: application/json" \
  -d '{"id": "fluffy-switchback-test", "name": "Fluffy Switchback Test", "comparison": {"baseline_instance_id": "production", "candidate_instance_id": "staging"}, "generate_random_plan": {"unit_duration_minutes": 2, "units": 5}}' \
  | jq '.'

{
  "id": "fluffy-switchback-test",
  "name": "Fluffy Switchback Test",
  "description": "",
  "created_at": "2026-07-29T15:02:55.634178557Z",
  "updated_at": "2026-07-29T15:02:55.634178557Z",
  "status": "draft",
  "comparison": {
    "baseline_instance_id": "production",
    "candidate_instance_id": "staging"
  },
  "plan": {
    "units": [
      {
        "duration_minutes": 2,
        "instance_id": "staging",
        "index": 0
      },
      {
        "duration_minutes": 2,
        "instance_id": "staging",
        "index": 1
      },
      {
        "duration_minutes": 2,
        "instance_id": "production",
        "index": 2
      },
      {
        "duration_minutes": 2,
        "instance_id": "production",
        "index": 3
      },
      {
        "duration_minutes": 2,
        "instance_id": "staging",
        "index": 4
      }
    ]
  }
}

The POST /v1/applications/{application_id}/experiments/switchback endpoint creates a switchback test in draft mode. Once the test is created, you must start it.

Start a switchback test

As mentioned above, when a switchback test is created, it is in draft mode. A switchback test must be started so that it can begin executing runs and collecting data. There are two ways to start a switchback test:

As an example, here is how to start a switchback test using the endpoint:

curl -s -X PUT "https://api.cloud.nextmv.io/v1/applications/uncanny-rodent/experiments/switchback/fluffy-switchback-test/start" \
  -H "Authorization: Bearer ${NEXTMV_API_KEY}"

Once a switchback test has started, you can stop it at any time or wait for it to complete based on the termination events defined when the test was created.

To generate data for the test, submit runs to the application using the baseline_instance_id (production in this example) as the instance_id query parameter, the same way described in the Run Cloud applications guide. As explained in the Explanation page, Nextmv Cloud automatically routes each run to whichever instance the currently active plan unit specifies, and links the resulting run to this switchback test.

Get a switchback test

Info

The best way to view and interact with switchback test results is in the Nextmv Console.

Use the GET /v1/applications/{application_id}/experiments/switchback/{switchback_id}/metadata endpoint to retrieve the metadata for a switchback test, using the switchback test ID.

curl -s -X GET "https://api.cloud.nextmv.io/v1/applications/uncanny-rodent/experiments/switchback/fluffy-switchback-test/metadata" \
  -H "Authorization: Bearer ${NEXTMV_API_KEY}" \
  | jq '.'

{
  "id": "fluffy-switchback-test",
  "name": "Fluffy Switchback Test",
  "description": "",
  "app_id": "uncanny-rodent",
  "created_at": "2026-07-29T15:02:55Z",
  "updated_at": "2026-07-29T15:02:56Z",
  "status": "started"
}

Once the status of the switchback test is completed, you can get its plan and comparison details using the GET /v1/applications/{application_id}/experiments/switchback/{switchback_id} endpoint. If the test hasn't completed, you can still get partial results.

A switchback test that has already started can be stopped at any time, or it will stop automatically when the termination events are fulfilled. Once a switchback test is stopped, then its status will move to completed.

curl -s -X GET "https://api.cloud.nextmv.io/v1/applications/uncanny-rodent/experiments/switchback/fluffy-switchback-test" \
  -H "Authorization: Bearer ${NEXTMV_API_KEY}" \
  | jq '.'

{
  "id": "fluffy-switchback-test",
  "name": "Fluffy Switchback Test",
  "description": "",
  "created_at": "2026-07-29T15:02:55.634178557Z",
  "updated_at": "2026-07-29T15:02:56.174454Z",
  "started_at": "2026-07-29T15:02:56.174454Z",
  "status": "started",
  "comparison": {
    "baseline_instance_id": "production",
    "candidate_instance_id": "staging"
  },
  "plan": {
    "start": "2026-07-29T15:02:56.174454Z",
    "units": [
      {
        "duration_minutes": 2,
        "instance_id": "staging",
        "index": 0
      },
      {
        "duration_minutes": 2,
        "instance_id": "staging",
        "index": 1
      },
      {
        "duration_minutes": 2,
        "instance_id": "production",
        "index": 2
      },
      {
        "duration_minutes": 2,
        "instance_id": "production",
        "index": 3
      },
      {
        "duration_minutes": 2,
        "instance_id": "staging",
        "index": 4
      }
    ]
  },
  "grouped_distributional_summaries": [
    {
      "group_keys": [
        "instanceID",
        "versionID"
      ],
      "group_values": [
        "staging",
        "v0.0.2"
      ],
      "indicator_keys": [
        "value"
      ],
      "indicator_distributions": {
        "value": {
          "min": 1.23,
          "max": 1.23,
          "count": 1,
          "mean": 1.23,
          "std": "nan",
          "shifted_geometric_mean": {
            "value": 1.2300000000000022,
            "shift": 10
          },
          "percentiles": {
            "p01": 1.23,
            "p05": 1.23,
            "p10": 1.23,
            "p25": 1.23,
            "p50": 1.23,
            "p75": 1.23,
            "p90": 1.23,
            "p95": 1.23,
            "p99": 1.23
          }
        }
      },
      "number_of_runs_total": 1
    }
  ]
}

Use the GET /v1/applications/{application_id}/experiments/switchback/{switchback_id}/runs endpoint to retrieve the runs that were made for the test. This endpoint is paginated, so the snippet below always uses pagination: it passes pagereturn=true to receive a next_page_token in the response, and keeps requesting pages by passing that token back as pagetoken until no token is returned.

RUNS="[]"
PAGE_TOKEN=""

while :; do
  RESPONSE=$(curl -s -G "https://api.cloud.nextmv.io/v1/applications/uncanny-rodent/experiments/switchback/fluffy-switchback-test/runs" \
    -H "Authorization: Bearer ${NEXTMV_API_KEY}" \
    --data-urlencode "pagereturn=true" \
    --data-urlencode "pagetoken=${PAGE_TOKEN}")

  RUNS=$(jq -n --argjson existing "$RUNS" --argjson page "$(echo "$RESPONSE" | jq '.runs')" '$existing + $page')
  PAGE_TOKEN=$(echo "$RESPONSE" | jq -r '.next_page_token // empty')

  [ -z "$PAGE_TOKEN" ] && break
done

echo "$RUNS" | jq '.'

[
  {
    "id": "staging-It0LqQEDg",
    "user_email": "sebastian@nextmv.io",
    "name": "",
    "description": "",
    "created_at": "2026-07-29T15:03:36.174454Z",
    "application_id": "uncanny-rodent",
    "application_instance_id": "staging",
    "application_version_id": "v0.0.2",
    "run_type": {
      "type": "",
      "definition_id": "",
      "reference_id": ""
    },
    "execution_class": "6c9500mb870s",
    "queuing_priority": 6,
    "queuing_disabled": false,
    "runtime": "python-3_11",
    "status": "succeeded",
    "status_v2": "succeeded",
    "metrics": {
      "status": "succeeded",
      "indicators": [
        {
          "name": "value",
          "value": 1.23
        },
        {
          "name": "metadata.duration",
          "value": 4.534
        }
      ]
    },
    "options": {
      "details": "true"
    },
    "options_summary": [
      {
        "name": "details",
        "value": "true",
        "source": "version"
      }
    ],
    "switchback_experiment": {
      "experiment_id": "fluffy-switchback-test",
      "plan_unit_index": 0
    }
  }
]

You can list all switchback tests in the application using the GET /v1/applications/{application_id}/experiments/switchback endpoint. This endpoint is paginated, so the snippet below always uses pagination.

ITEMS="[]"
PAGE_TOKEN=""

while :; do
  RESPONSE=$(curl -s -G "https://api.cloud.nextmv.io/v1/applications/uncanny-rodent/experiments/switchback" \
    -H "Authorization: Bearer ${NEXTMV_API_KEY}" \
    --data-urlencode "pagereturn=true" \
    --data-urlencode "pagetoken=${PAGE_TOKEN}")

  ITEMS=$(jq -n --argjson existing "$ITEMS" --argjson page "$(echo "$RESPONSE" | jq '.items')" '$existing + $page')
  PAGE_TOKEN=$(echo "$RESPONSE" | jq -r '.next_page_token // empty')

  [ -z "$PAGE_TOKEN" ] && break
done

echo "$ITEMS" | jq '.'

[
  {
    "id": "fluffy-switchback-test",
    "name": "Fluffy Switchback Test",
    "description": "",
    "created_at": "2026-07-29T15:02:55.634178557Z",
    "updated_at": "2026-07-29T15:02:56.174454Z",
    "status": "started",
    "started_at": "2026-07-29T15:02:56.174454Z",
    "comparison": {
      "baseline_instance_id": "production",
      "candidate_instance_id": "staging"
    }
  },
  {
    "id": "switchback-330prr2m",
    "name": "switchback-330prr2m",
    "description": "",
    "created_at": "2026-07-26T23:14:52.918111217Z",
    "updated_at": "2026-07-26T23:14:52.918111217Z",
    "status": "draft",
    "comparison": {
      "baseline_instance_id": "production",
      "candidate_instance_id": "staging"
    }
  }
]

Stop a switchback test

A switchback test will continue to execute until it stops, which is equivalent to it being completed. There are two ways to stop a switchback test:

When using the stop endpoint, you must pass the intent field in the request payload to track if you want to complete or cancel the switchback test. As an example, here is how to stop a switchback test and mark it as completed:

curl -s -X PUT "https://api.cloud.nextmv.io/v1/applications/uncanny-rodent/experiments/switchback/fluffy-switchback-test/stop" \
  -H "Authorization: Bearer ${NEXTMV_API_KEY}" \
  -H "Content-Type: application/json" \
  -d '{"intent": "complete"}'

Update a switchback test

You can update attributes of a switchback test with the PATCH /v1/applications/{application_id}/experiments/switchback/{switchback_id} endpoint, such as its:

  • Name
  • Description

The endpoint returns the updated switchback test attributes. You cannot update the ID of a switchback test.

curl -s -X PATCH "https://api.cloud.nextmv.io/v1/applications/uncanny-rodent/experiments/switchback/fluffy-switchback-test" \
  -H "Authorization: Bearer ${NEXTMV_API_KEY}" \
  -H "Content-Type: application/json" \
  -d '{"name": "Updated Switchback Test Name", "description": "Updated description for the switchback test"}' \
  | jq '.'

{
  "id": "fluffy-switchback-test",
  "name": "Updated Switchback Test Name",
  "description": "Updated description for the switchback test",
  "created_at": "2026-07-29T15:02:55.634178557Z",
  "updated_at": "2026-07-29T15:04:10.858089182Z"
}

Delete a switchback test

Warning

Deleting a switchback test is irreversible. All the data associated with the switchback test will be permanently deleted.

Delete a switchback test using the DELETE /v1/applications/{application_id}/experiments/switchback/{switchback_id} endpoint.

curl -s -X DELETE "https://api.cloud.nextmv.io/v1/applications/uncanny-rodent/experiments/switchback/fluffy-switchback-test" \
  -H "Authorization: Bearer ${NEXTMV_API_KEY}"