Queuing and prioritization

Queuing lets you submit runs beyond your allowed maximum concurrency limits. Queues are maintained for each execution class and ordered by priority then age. The next run started upon a completion will be the highest priority, oldest request for the execution class of the completed run.

A priority is between 1 (highest) and 9, with a priority 6 by default. Since queuing is specific to execution classes, run requests in execution classes that are not at limit will start without delay.

In some cases you may not want a run request to queue. An example could be a real time workload that must complete in a short time. In that case when you submit you specify no queuing behavior in the request, and you'll receive a busy failure if you are at capacity that and you can then manage immediately.

Using priorities and queuing

There are several ways to specify queuing behaviors.

When defined as part of an instance configuration

• creating an instance:

  • Nextmv CLI.

    • Use the -p, --priority flag to specify priority.
    • use the --no-queuing flag to disable queuing.

  • Cloud API.

    • Set the configuration.queuing.priority field to specify priority (1-9).
    • Set the configuration.queuing.disabled field to true to disable.

• updating an instance:

  • Nextmv CLI.

    • Use the -p, --priority flag to specify priority.
    • use the --no-queuing flag to disable queuing.

  • Cloud API.

    • Set the configuration.queuing.priority field to specify priority (1-9).
    • Set the configuration.queuing.disabled field to true to disable.

When starting a running with the CLI

As part of the run, when running remotely. Use one of the following interfaces.

• Nextmv CLI.
  • Use the -p, --priority flag to specify priority.
  • use the --no-queuing flag to disable queuing.

When starting a run with the API

• Cloud API.
  • Set the configuration.queuing.priority field to specify priority (1-9).
  • Set the configuration.queuing.disabled field to true to disable

Here is a full example for using the cloud API when starting a run.

When starting a run with the Python SDK

You can use the configuration argument. Here is an example:

import json
import os

from nextmv import cloud

client = cloud.Client(api_key=os.getenv("NEXTMV_API_KEY"))
app = cloud.Application(client=client, id="<YOUR_APP_ID>")

result = app.new_run_with_result(
    input=input,
    instance_id="<YOUR_INSTANCE_ID>",
    run_options={
        "solve.duration": "10s",
        "solve.iterations": "20",
    },
    configuration=cloud.RunConfiguration(
        queuing=cloud.RunQueuing(
            priority=3,
        ),
    ),
)

print(json.dumps(result.to_dict(), indent=2))