Running switchback tests¶

Info

Learn the concepts and fundamentals of switchback tests in the Explanation page.

A switchback test is an experiment that runs in the background and compares the results of two instances: baseline vs. candidate. The experiment switches back and forth between the two instances, when a run is made to the app, hence the name.

Included in this how-to guide are instructions on how to interact with the actions under the nextmv cloud switchback command tree. Go the reference section or use the --help option to see all the available options for each command.

Create a switchback test¶

Use the nextmv cloud switchback create command to create a new switchback test for an application. A switchback test requires you to define the number of units that it is going to run for. Each unit is a time duration in minutes. Lastly, you must define two instances to compare agains each other: a baseline and a candidate.

Here is an example command that creates a new switchback test for an application.

BashOutput

nextmv cloud switchback create \
    --app-id uncanny-rodent \
    --baseline-instance-id production \
    --candidate-instance-id staging \
    --unit-duration-minutes 2 \
    --units 5

⏳ Creating switchback test in draft mode...
{
  "id": "switchback-330prr2m",
  "name": "switchback-330prr2m",
  "description": "",
  "created_at": "2026-07-26T23:14:52.918111Z",
  "updated_at": "2026-07-26T23:14:52.918111Z",
  "status": "draft",
  "comparison": {
    "baseline_instance_id": "production",
    "candidate_instance_id": "staging"
  },
  "plan": {
    "units": [
      {
        "duration_minutes": 2.0,
        "instance_id": "staging",
        "index": 0
      },
      {
        "duration_minutes": 2.0,
        "instance_id": "production",
        "index": 1
      },
      {
        "duration_minutes": 2.0,
        "instance_id": "staging",
        "index": 2
      },
      {
        "duration_minutes": 2.0,
        "instance_id": "staging",
        "index": 3
      },
      {
        "duration_minutes": 2.0,
        "instance_id": "staging",
        "index": 4
      }
    ]
  }
}

The command above will create a random ID, and use the same identifier for the switchback test’s name. The name of the test is used as a human-readable You can use the --switchback-test-id and/or --name options to specify a custom ID and name for the switchback test. For example:

BashOutput

nextmv cloud switchback create \
    --app-id uncanny-rodent \
    --baseline-instance-id production \
    --candidate-instance-id staging \
    --unit-duration-minutes 2 \
    --units 5 \
    --switchback-test-id fluffy-switchback-test \
    --name "Fluffy Switchback Test"

⏳ Creating switchback test in draft mode...
{
  "id": "fluffy-switchback-test",
  "name": "Fluffy Switchback Test",
  "description": "",
  "created_at": "2026-07-26T23:16:33.505270Z",
  "updated_at": "2026-07-26T23:16:33.505270Z",
  "status": "draft",
  "comparison": {
    "baseline_instance_id": "production",
    "candidate_instance_id": "staging"
  },
  "plan": {
    "units": [
      {
        "duration_minutes": 2.0,
        "instance_id": "staging",
        "index": 0
      },
      {
        "duration_minutes": 2.0,
        "instance_id": "production",
        "index": 1
      },
      {
        "duration_minutes": 2.0,
        "instance_id": "staging",
        "index": 2
      },
      {
        "duration_minutes": 2.0,
        "instance_id": "production",
        "index": 3
      },
      {
        "duration_minutes": 2.0,
        "instance_id": "production",
        "index": 4
      }
    ]
  }
}

The nextmv cloud switchback create command creates a switchback test in draft mode. Once the test is created, you must start it.

Start a switchback test¶

As mentioned above, when a switchback test is created, it is in draft mode. A switchback test must be started so that it can begin executing runs and collecting data. There are two ways to start a switchback test:

Using the --start option with the nextmv cloud switchback create command. This will start the switchback test at the scheduled time.
Using the nextmv cloud switchback start command. This will start the switchback test as soon as the command is executed.

As an example, here is how to start a switchback test using the nextmv cloud switchback start command:

BashOutput

nextmv cloud switchback start --app-id uncanny-rodent --switchback-test-id fluffy-switchback-test

⏳ Starting switchback test...
✅ Switchback test fluffy-switchback-test started successfully in application uncanny-rodent.

Once a switchback test has started, you can stop it at any time or wait for it to complete based on the termination events defined when the test was created.

Get a switchback test¶

Info

The best way to view and interact with switchback test results is in the Nextmv Console.

Use the nextmv cloud switchback metadata command to retrieve the metadata for a switchback test, using the switchback test ID.

BashOutput

nextmv cloud switchback metadata --app-id uncanny-rodent --switchback-test-id fluffy-switchback-test

⏳ Getting switchback test metadata...
{
  "id": "fluffy-switchback-test",
  "name": "Fluffy Switchback Test",
  "description": "",
  "app_id": "uncanny-rodent",
  "created_at": "2026-07-26T23:16:33Z",
  "updated_at": "2026-07-26T23:19:32Z",
  "status": "started"
}

Once the status of the switchback test is completed, you can get the results using the nextmv cloud switchback get command. The output includes the runs that were made for the test. If the test hasn't completed, you can still get partial results.

A switchback test that has already started can be stopped at any time, or it will stop automatically when the termination events are fulfilled. Once a switchback test is stopped, then its status will move to completed.

BashOutput

nextmv cloud switchback get --app-id uncanny-rodent --switchback-test-id fluffy-switchback-test

⏳ Getting switchback test...
{
  "id": "fluffy-switchback-test",
  "name": "Fluffy Switchback Test",
  "description": "",
  "created_at": "2026-07-26T23:16:33.505270Z",
  "updated_at": "2026-07-26T23:19:32.224173Z",
  "status": "started",
  "started_at": "2026-07-26T23:19:32.221931Z",
  "comparison": {
    "baseline_instance_id": "production",
    "candidate_instance_id": "staging"
  },
  "plan": {
    "start": "2026-07-26T23:19:32.221931Z",
    "units": [
      {
        "duration_minutes": 2.0,
        "instance_id": "staging",
        "index": 0
      },
      {
        "duration_minutes": 2.0,
        "instance_id": "production",
        "index": 1
      },
      {
        "duration_minutes": 2.0,
        "instance_id": "staging",
        "index": 2
      },
      {
        "duration_minutes": 2.0,
        "instance_id": "production",
        "index": 3
      },
      {
        "duration_minutes": 2.0,
        "instance_id": "production",
        "index": 4
      }
    ]
  },
  "runs": [
    {
      "id": "staging-pgJeZaEvg",
      "user_email": "sebastian@nextmv.io",
      "name": "",
      "description": "",
      "created_at": "2026-07-26T23:21:18.605281Z",
      "application_id": "uncanny-rodent",
      "application_instance_id": "staging",
      "application_version_id": "v0.0.2",
      "run_type": {
        "definition_id": "",
        "reference_id": ""
      },
      "execution_class": "6c9500mb870s",
      "runtime": "python-3_11",
      "status_v2": "succeeded",
      "queuing_priority": 6,
      "queuing_disabled": false,
      "metrics": {
        "status": "succeeded",
        "indicators": [
          {
            "name": "value",
            "value": 1.23
          },
          {
            "name": "metadata.duration",
            "value": 5.951
          }
        ]
      },
      "options": {
        "details": "true"
      },
      "options_summary": [
        {
          "name": "details",
          "value": "true",
          "source": "version"
        }
      ]
    },
    {
      "id": "staging-cTp6Z-PDg",
      "user_email": "sebastian@nextmv.io",
      "name": "",
      "description": "",
      "created_at": "2026-07-26T23:21:14.988314Z",
      "application_id": "uncanny-rodent",
      "application_instance_id": "staging",
      "application_version_id": "v0.0.2",
      "run_type": {
        "definition_id": "",
        "reference_id": ""
      },
      "execution_class": "6c9500mb870s",
      "runtime": "python-3_11",
      "status_v2": "succeeded",
      "queuing_priority": 6,
      "queuing_disabled": false,
      "metrics": {
        "status": "succeeded",
        "indicators": [
          {
            "name": "value",
            "value": 1.23
          },
          {
            "name": "metadata.duration",
            "value": 5.869
          }
        ]
      },
      "options": {
        "details": "true"
      },
      "options_summary": [
        {
          "name": "details",
          "value": "true",
          "source": "version"
        }
      ]
    },
    {
      "id": "staging-i2veZ-PDR",
      "user_email": "sebastian@nextmv.io",
      "name": "",
      "description": "",
      "created_at": "2026-07-26T23:21:10.850735Z",
      "application_id": "uncanny-rodent",
      "application_instance_id": "staging",
      "application_version_id": "v0.0.2",
      "run_type": {
        "definition_id": "",
        "reference_id": ""
      },
      "execution_class": "6c9500mb870s",
      "runtime": "python-3_11",
      "status_v2": "succeeded",
      "queuing_priority": 6,
      "queuing_disabled": false,
      "metrics": {
        "status": "succeeded",
        "indicators": [
          {
            "name": "value",
            "value": 1.23
          },
          {
            "name": "metadata.duration",
            "value": 7.63
          }
        ]
      },
      "options": {
        "details": "true"
      },
      "options_summary": [
        {
          "name": "details",
          "value": "true",
          "source": "version"
        }
      ]
    },
    {
      "id": "staging-69neZaEvg",
      "user_email": "sebastian@nextmv.io",
      "name": "",
      "description": "",
      "created_at": "2026-07-26T23:21:07.231581Z",
      "application_id": "uncanny-rodent",
      "application_instance_id": "staging",
      "application_version_id": "v0.0.2",
      "run_type": {
        "definition_id": "",
        "reference_id": ""
      },
      "execution_class": "6c9500mb870s",
      "runtime": "python-3_11",
      "status_v2": "succeeded",
      "queuing_priority": 6,
      "queuing_disabled": false,
      "metrics": {
        "status": "succeeded",
        "indicators": [
          {
            "name": "value",
            "value": 1.23
          },
          {
            "name": "metadata.duration",
            "value": 7.453
          }
        ]
      },
      "options": {
        "details": "true"
      },
      "options_summary": [
        {
          "name": "details",
          "value": "true",
          "source": "version"
        }
      ]
    }
  ]
}

You can list all switchback tests in the application using the nextmv cloud switchback list command.

BashOutput

nextmv cloud switchback list --app-id uncanny-rodent

⏳ Listing switchback tests...
[
  {
    "id": "fluffy-switchback-test",
    "name": "Fluffy Switchback Test",
    "description": "",
    "created_at": "2026-07-26T23:16:33.505270Z",
    "updated_at": "2026-07-26T23:19:32.224173Z",
    "status": "started",
    "started_at": "2026-07-26T23:19:32.221931Z",
    "comparison": {
      "baseline_instance_id": "production",
      "candidate_instance_id": "staging"
    }
  },
  {
    "id": "switchback-330prr2m",
    "name": "switchback-330prr2m",
    "description": "",
    "created_at": "2026-07-26T23:14:52.918111Z",
    "updated_at": "2026-07-26T23:14:52.918111Z",
    "status": "draft",
    "comparison": {
      "baseline_instance_id": "production",
      "candidate_instance_id": "staging"
    }
  }
]

Stop a switchback test¶

A switchback test will continue to execute until it stops, which is equivalent to it being completed. There are two ways to stop a switchback test:

Using the --unit-duration-minutes and --units options with the nextmv cloud switchback create command. Termination events will cause the switchback test to stop.
Using the nextmv cloud switchback stop command. This will stop the switchback test as soon as the command is executed.

When using the nextmv cloud switchback stop command, you must use the --intent flag to track if you want to complete or cancel the switchback test. As an example, here is how to stop a switchback test and mark it as completed:

BashOutput

nextmv cloud switchback stop --app-id uncanny-rodent --switchback-test-id fluffy-switchback-test --intent complete

⏳ Stopping switchback test...
✅ Switchback test fluffy-switchback-test stopped successfully in application uncanny-rodent.

Update a switchback test¶

You can update attributes of a switchback test with the nextmv cloud switchback update command, such as its:

Name
Description

Please use the --help option on the nextmv cloud switchback update command to see all the available options for updating a switchback test.

You cannot update the ID of a switchback test.

BashOutput

nextmv cloud switchback update \
    --app-id uncanny-rodent \
    --switchback-test-id fluffy-switchback-test \
    --name "Updated switchback test Name" \
    --description "Updated description for the switchback test"

⏳ Updating switchback test...
✅ Switchback test fluffy-switchback-test updated successfully in application uncanny-rodent.
{
  "id": "fluffy-switchback-test",
  "name": "Updated switchback test Name",
  "description": "Updated description for the switchback test",
  "created_at": "2026-07-26T23:16:33.505270Z",
  "updated_at": "2026-07-26T23:26:24.483401Z"
}

Delete a switchback test¶

Warning

Deleting a switchback test is irreversible. All the data associated with the switchback test will be permanently deleted.

Delete a switchback test using the nextmv cloud switchback delete command. The CLI will prompt you to confirm deletion but you can override it with the --yes option.

BashOutput

nextmv cloud switchback delete --app-id uncanny-rodent --switchback-test-id fluffy-switchback-test

Are you sure you want to delete switchback test fluffy-switchback-test from application uncanny-rodent? This action cannot be undone.
💡 Confirm Yes
✅ Switchback test fluffy-switchback-test deleted successfully from application uncanny-rodent.