Skip to content

Running scenario tests

Info

Learn the concepts and fundamentals of scenario tests in the Explanation page.

Scenario tests are offline tests used to compare the output from one or more scenarios.

Included in this how-to guide are instructions on how to interact with the actions under the nextmv cloud scenario command tree. Go the reference section or use the --help option to see all the available options for each command.

Create a scenario test

Use the nextmv cloud scenario create command to create a new scenario test for an application. You can create multiple scenarios with the --scenarios flag by:

  • Using the --scenarios flag multiple times, or
  • Providing an array of objects in valid JSON format.

Each scenario is defined as a JSON object with the following attributes:

  • instance_id: ID of the instance to use for this scenario (required).
  • scenario_input: Object containing the scenario input (required), with:
    • scenario_input_type: Type of the scenario input (required). Allowed values: input_set, input, and new.
    • scenario_input_data: Data for the scenario input (required).
      • For input_set: a str (the input set ID).
      • For input: an array of str (list of input IDs).
      • For new: an array of objects (raw data).
  • scenario_id: ID of the scenario (optional). The default value will be set as scenario-<index> if not set.
  • configuration: An array of configuration objects (optional). Use this attribute to configure variation of options for the scenario. Each scenario configuration object requires:
    • name: Name of the configuration option.
    • values: List of values for the configuration option.

Consider the following example where two scenario are defined, using an input set and a managed input, respectively.

[
  {
    "instance_id": "production",
    "scenario_input": {
      "scenario_input_type": "input_set",
      "scenario_input_data": "burrowing-hares"
    },
    "configuration": [
      {
        "name": "details",
        "values": ["true", "false"]
      }
    ]
  },
  {
    "instance_id": "production",
    "scenario_input": {
      "scenario_input_type": "input",
      "scenario_input_data": ["baxter-burrow"]
    },
    "configuration": [
      {
        "name": "details",
        "values": ["true", "false"]
      }
    ]
  }
]

Here is an example command that uses the scenarios defined above to create a new scenario test for an application with repetitions.

SCENARIOS='[
  {
    "instance_id": "production",
    "scenario_input": {
      "scenario_input_type": "input_set",
      "scenario_input_data": "burrowing-hares"
    },
    "configuration": [
      {
        "name": "details",
        "values": ["true", "false"]
      }
    ]
  },
  {
    "instance_id": "production",
    "scenario_input": {
      "scenario_input_type": "input",
      "scenario_input_data": ["baxter-burrow"]
    },
    "configuration": [
      {
        "name": "details",
        "values": ["true", "false"]
      }
    ]
  }
]'
nextmv cloud scenario create \
    --app-id uncanny-rodent \
    --scenarios "$SCENARIOS" \
    --repetitions 3

{
  "scenario_test_id": "scenario-4nfrhc3v"
}

The command above will create a random ID, and use the same identifier for the scenario test's name. The name of the test is used as a human-readable You can use the --scenario-test-id and/or --name options to specify a custom ID and name for the scenario test. For example:

nextmv cloud scenario create \
    --app-id uncanny-rodent \
    --scenarios "$SCENARIOS" \
    --scenario-test-id fluffy-scenario-test \
    --name "Scenario test for a fluffy bunny"

{
  "scenario_test_id": "fluffy-scenario-test"
}

Get a scenario test

Info

The best way to view and interact with scenario test results is in the Nextmv Console.

Use the nextmv cloud scenario metadata command to retrieve the metadata for a scenario test, using the scenario test ID.

nextmv cloud scenario metadata --app-id uncanny-rodent --scenario-test-id fluffy-scenario-test

⏳ Getting scenario test metadata...
{
  "id": "fluffy-scenario-test",
  "name": "Scenario test for a fluffy bunny",
  "created_at": "2026-07-25T16:03:01Z",
  "updated_at": "2026-07-25T16:03:16Z",
  "status": "completed",
  "description": "",
  "number_of_requested_runs": 16,
  "number_of_runs": 16,
  "number_of_completed_runs": 16,
  "type": "scenario",
  "app_id": "uncanny-rodent"
}

Once the status of the scenario test is completed, you can get the results using the nextmv cloud scenario get command. The output includes the runs that were made for the scenario test.

nextmv cloud scenario get --app-id uncanny-rodent --scenario-test-id fluffy-scenario-test

⏳ Getting scenario test...
{
  "id": "fluffy-scenario-test",
  "name": "Scenario test for a fluffy bunny",
  "created_at": "2026-07-25T16:03:01.817797Z",
  "updated_at": "2026-07-25T16:03:16.051611Z",
  "status": "completed",
  "description": "",
  "number_of_requested_runs": 16,
  "number_of_runs": 16,
  "number_of_completed_runs": 16,
  "type": "scenario",
  "option_sets": {
    "scenario-1_0": {
      "details": "true"
    },
    "scenario-1_1": {
      "details": "false"
    },
    "scenario-2_0": {
      "details": "true"
    },
    "scenario-2_1": {
      "details": "false"
    }
  },
  "input_set_id": "",
  "instance_ids": [
    "production"
  ],
  "grouped_distributional_summaries": [
    {
      "group_keys": [
        "instanceID",
        "versionID"
      ],
      "group_values": [
        "production",
        "v0.0.2"
      ],
      "indicator_keys": [
        "value"
      ],
      "indicator_distributions": {
        "value": {
          "min": 1.23,
          "max": 1.23,
          "count": 16,
          "mean": 1.2300000000000002,
          "std": 2.293266818639604e-16,
          "shifted_geometric_mean": {
            "value": 1.230000000000011,
            "shift": 10
          },
          "percentiles": {
            "p01": 1.23,
            "p05": 1.23,
            "p10": 1.23,
            "p25": 1.23,
            "p50": 1.23,
            "p75": 1.23,
            "p90": 1.23,
            "p95": 1.23,
            "p99": 1.23
          }
        }
      },
      "number_of_runs_total": 16
    },
    {
      "group_keys": [
        "inputID",
        "instanceID",
        "versionID"
      ],
      "group_values": [
        "baxter-burrow",
        "production",
        "v0.0.2"
      ],
      "indicator_keys": [
        "value"
      ],
      "indicator_distributions": {
        "value": {
          "min": 1.23,
          "max": 1.23,
          "count": 8,
          "mean": 1.2300000000000002,
          "std": 2.3737566748887e-16,
          "shifted_geometric_mean": {
            "value": 1.2300000000000022,
            "shift": 10
          },
          "percentiles": {
            "p01": 1.23,
            "p05": 1.23,
            "p10": 1.23,
            "p25": 1.23,
            "p50": 1.23,
            "p75": 1.23,
            "p90": 1.23,
            "p95": 1.23,
            "p99": 1.23
          }
        }
      },
      "number_of_runs_total": 8
    },
    {
      "group_keys": [
        "inputID",
        "instanceID",
        "versionID"
      ],
      "group_values": [
        "input.json-8MFSJpPDg",
        "production",
        "v0.0.2"
      ],
      "indicator_keys": [
        "value"
      ],
      "indicator_distributions": {
        "value": {
          "min": 1.23,
          "max": 1.23,
          "count": 8,
          "mean": 1.2300000000000002,
          "std": 2.3737566748887e-16,
          "shifted_geometric_mean": {
            "value": 1.2300000000000022,
            "shift": 10
          },
          "percentiles": {
            "p01": 1.23,
            "p05": 1.23,
            "p10": 1.23,
            "p25": 1.23,
            "p50": 1.23,
            "p75": 1.23,
            "p90": 1.23,
            "p95": 1.23,
            "p99": 1.23
          }
        }
      },
      "number_of_runs_total": 8
    },
    {
      "group_keys": [
        "inputID"
      ],
      "group_values": [
        "baxter-burrow"
      ],
      "indicator_keys": [
        "value"
      ],
      "indicator_distributions": {
        "value": {
          "min": 1.23,
          "max": 1.23,
          "count": 8,
          "mean": 1.2300000000000002,
          "std": 2.3737566748887e-16,
          "shifted_geometric_mean": {
            "value": 1.2300000000000022,
            "shift": 10
          },
          "percentiles": {
            "p01": 1.23,
            "p05": 1.23,
            "p10": 1.23,
            "p25": 1.23,
            "p50": 1.23,
            "p75": 1.23,
            "p90": 1.23,
            "p95": 1.23,
            "p99": 1.23
          }
        }
      },
      "number_of_runs_total": 8
    },
    {
      "group_keys": [
        "inputID"
      ],
      "group_values": [
        "input.json-8MFSJpPDg"
      ],
      "indicator_keys": [
        "value"
      ],
      "indicator_distributions": {
        "value": {
          "min": 1.23,
          "max": 1.23,
          "count": 8,
          "mean": 1.2300000000000002,
          "std": 2.3737566748887e-16,
          "shifted_geometric_mean": {
            "value": 1.2300000000000022,
            "shift": 10
          },
          "percentiles": {
            "p01": 1.23,
            "p05": 1.23,
            "p10": 1.23,
            "p25": 1.23,
            "p50": 1.23,
            "p75": 1.23,
            "p90": 1.23,
            "p95": 1.23,
            "p99": 1.23
          }
        }
      },
      "number_of_runs_total": 8
    }
  ],
  "runs": [
    {
      "id": "production-hBv0UTEvg",
      "user_email": "sebastian@nextmv.io",
      "name": "",
      "description": "",
      "created_at": "2026-07-25T16:03:01.902036Z",
      "application_id": "uncanny-rodent",
      "application_instance_id": "production",
      "application_version_id": "v0.0.2",
      "run_type": {
        "definition_id": "",
        "reference_id": ""
      },
      "execution_class": "6c9500mb870s",
      "runtime": "python-3_11",
      "status_v2": "succeeded",
      "queuing_priority": 6,
      "queuing_disabled": false,
      "experiment_id": "fluffy-scenario-test",
      "metrics": {
        "status": "succeeded",
        "indicators": [
          {
            "name": "value",
            "value": 1.23
          },
          {
            "name": "metadata.duration",
            "value": 4.58
          }
        ]
      },
      "input_id": "baxter-burrow",
      "option_set": "scenario-2_1",
      "options": {
        "details": "false"
      },
      "options_summary": [
        {
          "name": "details",
          "value": "false",
          "source": "run"
        }
      ],
      "scenario_id": "scenario-2",
      "repetition": 3,
      "input_set_id": "inpset-scenario-2-dvupef6b"
    },
    {
      "id": "production-HfvAUTEvg",
      "user_email": "sebastian@nextmv.io",
      "name": "",
      "description": "",
      "created_at": "2026-07-25T16:03:01.896126Z",
      "application_id": "uncanny-rodent",
      "application_instance_id": "production",
      "application_version_id": "v0.0.2",
      "run_type": {
        "definition_id": "",
        "reference_id": ""
      },
      "execution_class": "6c9500mb870s",
      "runtime": "python-3_11",
      "status_v2": "succeeded",
      "queuing_priority": 6,
      "queuing_disabled": false,
      "experiment_id": "fluffy-scenario-test",
      "metrics": {
        "status": "succeeded",
        "indicators": [
          {
            "name": "value",
            "value": 1.23
          },
          {
            "name": "metadata.duration",
            "value": 5.827
          }
        ]
      },
      "input_id": "baxter-burrow",
      "option_set": "scenario-2_1",
      "options": {
        "details": "false"
      },
      "options_summary": [
        {
          "name": "details",
          "value": "false",
          "source": "run"
        }
      ],
      "scenario_id": "scenario-2",
      "repetition": 2,
      "input_set_id": "inpset-scenario-2-dvupef6b"
    },
    {
      "id": "production-zBDAUTEvR",
      "user_email": "sebastian@nextmv.io",
      "name": "",
      "description": "",
      "created_at": "2026-07-25T16:03:01.889970Z",
      "application_id": "uncanny-rodent",
      "application_instance_id": "production",
      "application_version_id": "v0.0.2",
      "run_type": {
        "definition_id": "",
        "reference_id": ""
      },
      "execution_class": "6c9500mb870s",
      "runtime": "python-3_11",
      "status_v2": "succeeded",
      "queuing_priority": 6,
      "queuing_disabled": false,
      "experiment_id": "fluffy-scenario-test",
      "metrics": {
        "status": "succeeded",
        "indicators": [
          {
            "name": "value",
            "value": 1.23
          },
          {
            "name": "metadata.duration",
            "value": 4.446
          }
        ]
      },
      "input_id": "baxter-burrow",
      "option_set": "scenario-2_1",
      "options": {
        "details": "false"
      },
      "options_summary": [
        {
          "name": "details",
          "value": "false",
          "source": "run"
        }
      ],
      "scenario_id": "scenario-2",
      "repetition": 1,
      "input_set_id": "inpset-scenario-2-dvupef6b"
    },
    {
      "id": "production-j-DAUoPDR",
      "user_email": "sebastian@nextmv.io",
      "name": "",
      "description": "",
      "created_at": "2026-07-25T16:03:01.885337Z",
      "application_id": "uncanny-rodent",
      "application_instance_id": "production",
      "application_version_id": "v0.0.2",
      "run_type": {
        "definition_id": "",
        "reference_id": ""
      },
      "execution_class": "6c9500mb870s",
      "runtime": "python-3_11",
      "status_v2": "succeeded",
      "queuing_priority": 6,
      "queuing_disabled": false,
      "experiment_id": "fluffy-scenario-test",
      "metrics": {
        "status": "succeeded",
        "indicators": [
          {
            "name": "value",
            "value": 1.23
          },
          {
            "name": "metadata.duration",
            "value": 4.434
          }
        ]
      },
      "input_id": "baxter-burrow",
      "option_set": "scenario-2_1",
      "options": {
        "details": "false"
      },
      "options_summary": [
        {
          "name": "details",
          "value": "false",
          "source": "run"
        }
      ],
      "scenario_id": "scenario-2",
      "repetition": 0,
      "input_set_id": "inpset-scenario-2-dvupef6b"
    },
    {
      "id": "production-yavAUTEvg",
      "user_email": "sebastian@nextmv.io",
      "name": "",
      "description": "",
      "created_at": "2026-07-25T16:03:01.879727Z",
      "application_id": "uncanny-rodent",
      "application_instance_id": "production",
      "application_version_id": "v0.0.2",
      "run_type": {
        "definition_id": "",
        "reference_id": ""
      },
      "execution_class": "6c9500mb870s",
      "runtime": "python-3_11",
      "status_v2": "succeeded",
      "queuing_priority": 6,
      "queuing_disabled": false,
      "experiment_id": "fluffy-scenario-test",
      "metrics": {
        "status": "succeeded",
        "indicators": [
          {
            "name": "value",
            "value": 1.23
          },
          {
            "name": "metadata.duration",
            "value": 5.952
          }
        ]
      },
      "input_id": "baxter-burrow",
      "option_set": "scenario-2_0",
      "options": {
        "details": "true"
      },
      "options_summary": [
        {
          "name": "details",
          "value": "true",
          "source": "run"
        }
      ],
      "scenario_id": "scenario-2",
      "repetition": 3,
      "input_set_id": "inpset-scenario-2-dvupef6b"
    },
    {
      "id": "production-baD08oPvR",
      "user_email": "sebastian@nextmv.io",
      "name": "",
      "description": "",
      "created_at": "2026-07-25T16:03:01.874341Z",
      "application_id": "uncanny-rodent",
      "application_instance_id": "production",
      "application_version_id": "v0.0.2",
      "run_type": {
        "definition_id": "",
        "reference_id": ""
      },
      "execution_class": "6c9500mb870s",
      "runtime": "python-3_11",
      "status_v2": "succeeded",
      "queuing_priority": 6,
      "queuing_disabled": false,
      "experiment_id": "fluffy-scenario-test",
      "metrics": {
        "status": "succeeded",
        "indicators": [
          {
            "name": "value",
            "value": 1.23
          },
          {
            "name": "metadata.duration",
            "value": 5.839
          }
        ]
      },
      "input_id": "baxter-burrow",
      "option_set": "scenario-2_0",
      "options": {
        "details": "true"
      },
      "options_summary": [
        {
          "name": "details",
          "value": "true",
          "source": "run"
        }
      ],
      "scenario_id": "scenario-2",
      "repetition": 2,
      "input_set_id": "inpset-scenario-2-dvupef6b"
    },
    {
      "id": "production-5aD0UoEDg",
      "user_email": "sebastian@nextmv.io",
      "name": "",
      "description": "",
      "created_at": "2026-07-25T16:03:01.868585Z",
      "application_id": "uncanny-rodent",
      "application_instance_id": "production",
      "application_version_id": "v0.0.2",
      "run_type": {
        "definition_id": "",
        "reference_id": ""
      },
      "execution_class": "6c9500mb870s",
      "runtime": "python-3_11",
      "status_v2": "succeeded",
      "queuing_priority": 6,
      "queuing_disabled": false,
      "experiment_id": "fluffy-scenario-test",
      "metrics": {
        "status": "succeeded",
        "indicators": [
          {
            "name": "value",
            "value": 1.23
          },
          {
            "name": "metadata.duration",
            "value": 5.85
          }
        ]
      },
      "input_id": "baxter-burrow",
      "option_set": "scenario-2_0",
      "options": {
        "details": "true"
      },
      "options_summary": [
        {
          "name": "details",
          "value": "true",
          "source": "run"
        }
      ],
      "scenario_id": "scenario-2",
      "repetition": 1,
      "input_set_id": "inpset-scenario-2-dvupef6b"
    },
    {
      "id": "production-I-vA8oEDR",
      "user_email": "sebastian@nextmv.io",
      "name": "",
      "description": "",
      "created_at": "2026-07-25T16:03:01.863045Z",
      "application_id": "uncanny-rodent",
      "application_instance_id": "production",
      "application_version_id": "v0.0.2",
      "run_type": {
        "definition_id": "",
        "reference_id": ""
      },
      "execution_class": "6c9500mb870s",
      "runtime": "python-3_11",
      "status_v2": "succeeded",
      "queuing_priority": 6,
      "queuing_disabled": false,
      "experiment_id": "fluffy-scenario-test",
      "metrics": {
        "status": "succeeded",
        "indicators": [
          {
            "name": "value",
            "value": 1.23
          },
          {
            "name": "metadata.duration",
            "value": 6.027
          }
        ]
      },
      "input_id": "baxter-burrow",
      "option_set": "scenario-2_0",
      "options": {
        "details": "true"
      },
      "options_summary": [
        {
          "name": "details",
          "value": "true",
          "source": "run"
        }
      ],
      "scenario_id": "scenario-2",
      "repetition": 0,
      "input_set_id": "inpset-scenario-2-dvupef6b"
    },
    {
      "id": "production-iaDAUoPvR",
      "user_email": "sebastian@nextmv.io",
      "name": "",
      "description": "",
      "created_at": "2026-07-25T16:03:01.858326Z",
      "application_id": "uncanny-rodent",
      "application_instance_id": "production",
      "application_version_id": "v0.0.2",
      "run_type": {
        "definition_id": "",
        "reference_id": ""
      },
      "execution_class": "6c9500mb870s",
      "runtime": "python-3_11",
      "status_v2": "succeeded",
      "queuing_priority": 6,
      "queuing_disabled": false,
      "experiment_id": "fluffy-scenario-test",
      "metrics": {
        "status": "succeeded",
        "indicators": [
          {
            "name": "value",
            "value": 1.23
          },
          {
            "name": "metadata.duration",
            "value": 6
          }
        ]
      },
      "input_id": "input.json-8MFSJpPDg",
      "option_set": "scenario-1_1",
      "options": {
        "details": "false"
      },
      "options_summary": [
        {
          "name": "details",
          "value": "false",
          "source": "run"
        }
      ],
      "scenario_id": "scenario-1",
      "repetition": 3,
      "input_set_id": "burrowing-hares"
    },
    {
      "id": "production-Cbv08oPvg",
      "user_email": "sebastian@nextmv.io",
      "name": "",
      "description": "",
      "created_at": "2026-07-25T16:03:01.853535Z",
      "application_id": "uncanny-rodent",
      "application_instance_id": "production",
      "application_version_id": "v0.0.2",
      "run_type": {
        "definition_id": "",
        "reference_id": ""
      },
      "execution_class": "6c9500mb870s",
      "runtime": "python-3_11",
      "status_v2": "succeeded",
      "queuing_priority": 6,
      "queuing_disabled": false,
      "experiment_id": "fluffy-scenario-test",
      "metrics": {
        "status": "succeeded",
        "indicators": [
          {
            "name": "value",
            "value": 1.23
          },
          {
            "name": "metadata.duration",
            "value": 5.778
          }
        ]
      },
      "input_id": "input.json-8MFSJpPDg",
      "option_set": "scenario-1_1",
      "options": {
        "details": "false"
      },
      "options_summary": [
        {
          "name": "details",
          "value": "false",
          "source": "run"
        }
      ],
      "scenario_id": "scenario-1",
      "repetition": 2,
      "input_set_id": "burrowing-hares"
    },
    {
      "id": "production-sbD08oPvR",
      "user_email": "sebastian@nextmv.io",
      "name": "",
      "description": "",
      "created_at": "2026-07-25T16:03:01.847375Z",
      "application_id": "uncanny-rodent",
      "application_instance_id": "production",
      "application_version_id": "v0.0.2",
      "run_type": {
        "definition_id": "",
        "reference_id": ""
      },
      "execution_class": "6c9500mb870s",
      "runtime": "python-3_11",
      "status_v2": "succeeded",
      "queuing_priority": 6,
      "queuing_disabled": false,
      "experiment_id": "fluffy-scenario-test",
      "metrics": {
        "status": "succeeded",
        "indicators": [
          {
            "name": "value",
            "value": 1.23
          },
          {
            "name": "metadata.duration",
            "value": 5.935
          }
        ]
      },
      "input_id": "input.json-8MFSJpPDg",
      "option_set": "scenario-1_1",
      "options": {
        "details": "false"
      },
      "options_summary": [
        {
          "name": "details",
          "value": "false",
          "source": "run"
        }
      ],
      "scenario_id": "scenario-1",
      "repetition": 1,
      "input_set_id": "burrowing-hares"
    },
    {
      "id": "production-1bD08oEvR",
      "user_email": "sebastian@nextmv.io",
      "name": "",
      "description": "",
      "created_at": "2026-07-25T16:03:01.841994Z",
      "application_id": "uncanny-rodent",
      "application_instance_id": "production",
      "application_version_id": "v0.0.2",
      "run_type": {
        "definition_id": "",
        "reference_id": ""
      },
      "execution_class": "6c9500mb870s",
      "runtime": "python-3_11",
      "status_v2": "succeeded",
      "queuing_priority": 6,
      "queuing_disabled": false,
      "experiment_id": "fluffy-scenario-test",
      "metrics": {
        "status": "succeeded",
        "indicators": [
          {
            "name": "value",
            "value": 1.23
          },
          {
            "name": "metadata.duration",
            "value": 6.037
          }
        ]
      },
      "input_id": "input.json-8MFSJpPDg",
      "option_set": "scenario-1_1",
      "options": {
        "details": "false"
      },
      "options_summary": [
        {
          "name": "details",
          "value": "false",
          "source": "run"
        }
      ],
      "scenario_id": "scenario-1",
      "repetition": 0,
      "input_set_id": "burrowing-hares"
    },
    {
      "id": "production-pbv08oEvg",
      "user_email": "sebastian@nextmv.io",
      "name": "",
      "description": "",
      "created_at": "2026-07-25T16:03:01.837266Z",
      "application_id": "uncanny-rodent",
      "application_instance_id": "production",
      "application_version_id": "v0.0.2",
      "run_type": {
        "definition_id": "",
        "reference_id": ""
      },
      "execution_class": "6c9500mb870s",
      "runtime": "python-3_11",
      "status_v2": "succeeded",
      "queuing_priority": 6,
      "queuing_disabled": false,
      "experiment_id": "fluffy-scenario-test",
      "metrics": {
        "status": "succeeded",
        "indicators": [
          {
            "name": "value",
            "value": 1.23
          },
          {
            "name": "metadata.duration",
            "value": 6.126
          }
        ]
      },
      "input_id": "input.json-8MFSJpPDg",
      "option_set": "scenario-1_0",
      "options": {
        "details": "true"
      },
      "options_summary": [
        {
          "name": "details",
          "value": "true",
          "source": "run"
        }
      ],
      "scenario_id": "scenario-1",
      "repetition": 3,
      "input_set_id": "burrowing-hares"
    },
    {
      "id": "production-7bv08TEDR",
      "user_email": "sebastian@nextmv.io",
      "name": "",
      "description": "",
      "created_at": "2026-07-25T16:03:01.829967Z",
      "application_id": "uncanny-rodent",
      "application_instance_id": "production",
      "application_version_id": "v0.0.2",
      "run_type": {
        "definition_id": "",
        "reference_id": ""
      },
      "execution_class": "6c9500mb870s",
      "runtime": "python-3_11",
      "status_v2": "succeeded",
      "queuing_priority": 6,
      "queuing_disabled": false,
      "experiment_id": "fluffy-scenario-test",
      "metrics": {
        "status": "succeeded",
        "indicators": [
          {
            "name": "value",
            "value": 1.23
          },
          {
            "name": "metadata.duration",
            "value": 7.482
          }
        ]
      },
      "input_id": "input.json-8MFSJpPDg",
      "option_set": "scenario-1_0",
      "options": {
        "details": "true"
      },
      "options_summary": [
        {
          "name": "details",
          "value": "true",
          "source": "run"
        }
      ],
      "scenario_id": "scenario-1",
      "repetition": 2,
      "input_set_id": "burrowing-hares"
    },
    {
      "id": "production-61vA8TEDR",
      "user_email": "sebastian@nextmv.io",
      "name": "",
      "description": "",
      "created_at": "2026-07-25T16:03:01.823596Z",
      "application_id": "uncanny-rodent",
      "application_instance_id": "production",
      "application_version_id": "v0.0.2",
      "run_type": {
        "definition_id": "",
        "reference_id": ""
      },
      "execution_class": "6c9500mb870s",
      "runtime": "python-3_11",
      "status_v2": "succeeded",
      "queuing_priority": 6,
      "queuing_disabled": false,
      "experiment_id": "fluffy-scenario-test",
      "metrics": {
        "status": "succeeded",
        "indicators": [
          {
            "name": "value",
            "value": 1.23
          },
          {
            "name": "metadata.duration",
            "value": 6.093
          }
        ]
      },
      "input_id": "input.json-8MFSJpPDg",
      "option_set": "scenario-1_0",
      "options": {
        "details": "true"
      },
      "options_summary": [
        {
          "name": "details",
          "value": "true",
          "source": "run"
        }
      ],
      "scenario_id": "scenario-1",
      "repetition": 1,
      "input_set_id": "burrowing-hares"
    },
    {
      "id": "production-a_JDAUoEvg",
      "user_email": "sebastian@nextmv.io",
      "name": "",
      "description": "",
      "created_at": "2026-07-25T16:03:01.818033Z",
      "application_id": "uncanny-rodent",
      "application_instance_id": "production",
      "application_version_id": "v0.0.2",
      "run_type": {
        "definition_id": "",
        "reference_id": ""
      },
      "execution_class": "6c9500mb870s",
      "runtime": "python-3_11",
      "status_v2": "succeeded",
      "queuing_priority": 6,
      "queuing_disabled": false,
      "experiment_id": "fluffy-scenario-test",
      "metrics": {
        "status": "succeeded",
        "indicators": [
          {
            "name": "value",
            "value": 1.23
          },
          {
            "name": "metadata.duration",
            "value": 6.07
          }
        ]
      },
      "input_id": "input.json-8MFSJpPDg",
      "option_set": "scenario-1_0",
      "options": {
        "details": "true"
      },
      "options_summary": [
        {
          "name": "details",
          "value": "true",
          "source": "run"
        }
      ],
      "scenario_id": "scenario-1",
      "repetition": 0,
      "input_set_id": "burrowing-hares"
    }
  ]
}

You can list all scenario tests in the application using the nextmv cloud scenario list command.

nextmv cloud scenario list --app-id uncanny-rodent

⏳ Listing scenario tests...
[
  {
    "id": "fluffy-scenario-test",
    "name": "Scenario test for a fluffy bunny",
    "created_at": "2026-07-25T16:03:01.817797Z",
    "updated_at": "2026-07-25T16:03:16.051611Z",
    "status": "completed",
    "description": "",
    "number_of_requested_runs": 16,
    "number_of_runs": 16,
    "number_of_completed_runs": 16,
    "type": "scenario",
    "option_sets": {
      "scenario-1_0": {
        "details": "true"
      },
      "scenario-1_1": {
        "details": "false"
      },
      "scenario-2_0": {
        "details": "true"
      },
      "scenario-2_1": {
        "details": "false"
      }
    }
  },
  {
    "id": "scenario-4nfrhc3v",
    "name": "scenario-4nfrhc3v",
    "created_at": "2026-07-24T19:22:04.319026Z",
    "updated_at": "2026-07-24T19:22:17.988950Z",
    "status": "completed",
    "description": "",
    "number_of_requested_runs": 16,
    "number_of_runs": 16,
    "number_of_completed_runs": 16,
    "type": "scenario",
    "option_sets": {
      "scenario-1_0": {
        "details": "true"
      },
      "scenario-1_1": {
        "details": "false"
      },
      "scenario-2_0": {
        "details": "true"
      },
      "scenario-2_1": {
        "details": "false"
      }
    }
  }
]

Update a scenario test

You can update attributes of a scenario test with the nextmv cloud scenario update command, such as its:

  • Name
  • Description

Please use the --help option on the nextmv cloud scenario update command to see all the available options for updating a scenario test.

You cannot update the ID of a scenario test.

nextmv cloud scenario update \
    --app-id uncanny-rodent \
    --scenario-test-id fluffy-scenario-test \
    --name "Updated Scenario Test Name" \
    --description "Updated description for the scenario test"

⏳ Updating scenario test...
✅ Scenario test fluffy-scenario-test updated successfully in application uncanny-rodent.
{
  "id": "fluffy-scenario-test",
  "name": "Updated Scenario Test Name",
  "created_at": "2026-07-25T16:03:01.817797Z",
  "updated_at": "2026-07-25T16:11:30.752257Z",
  "description": "Updated description for the scenario test"
}

Delete a scenario test

Warning

Deleting a scenario test is irreversible. All the runs associated with the scenario test will be permanently deleted.

Delete a scenario test using the nextmv cloud scenario delete command. The CLI will prompt you to confirm deletion but you can override it with the --yes option.

nextmv cloud scenario delete --app-id uncanny-rodent --scenario-test-id fluffy-scenario-test

Are you sure you want to delete scenario test fluffy-scenario-test from application uncanny-rodent? This action cannot be undone.
💡 Confirm Yes
✅ Scenario test fluffy-scenario-test deleted.