Cohere

Cohere provides Command models and APIs for generation, embeddings, and reranking. AISIX places these capabilities behind gateway-managed credentials, caller access, rate limits, and usage accounting.

Cohere publishes a native API and a compatibility API for OpenAI-shaped requests. This guide uses the compatibility API for chat completions and embeddings, then configures the native API separately for reranking.

Prerequisites

Before starting, prepare the following:

• One AISIX setup:
  • For AISIX Cloud, an environment with an attached gateway and a write-scoped admin token. For On-Premises, follow the AISIX Cloud Quickstart. To request Hybrid Cloud access, contact API7.
  • For the open-source AISIX gateway, prepare either a local AISIX installation or the Docker setup from the Open-Source AISIX Gateway Quickstart. Configure the gateway to load a declarative resources file.
• A Cohere API key from the Cohere dashboard.
• curl and jq.

Configure with AISIX Cloud

Export the AISIX Cloud connection details:

# AISIX_CP is the Admin API base URL; include /api and omit a trailing slash
# The local On-Premises quickstart uses http://localhost:8080/api
export AISIX_CP="YOUR_AISIX_CLOUD_ADMIN_API_URL"
export AISIX_TOKEN="YOUR_ADMIN_TOKEN"
export ENV_ID="YOUR_ENVIRONMENT_ID"

Create a provider key, model alias, and caller API key for the Cohere-backed chat-completions route.

AISIX connects to Cohere's compatibility API through the openai adapter. Chat completions and embeddings need no request translation. Cohere rerank uses the native API, and the same provider key serves it, as shown in Add a Rerank Model.

Create a Provider Key

Create the provider key that stores the Cohere credential and API root:

# Replace with your value
export COHERE_API_KEY="YOUR_PROVIDER_API_KEY"

PROVIDER_KEY_ID=$(curl -sS -X POST "$AISIX_CP/provider_keys" \
  -H "Authorization: Bearer $AISIX_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{
    "display_name": "cohere-prod",
    "provider": "cohere",
    "api_key": "'"${COHERE_API_KEY}"'",
    "api_base": "https://api.cohere.ai/compatibility/v1",
    "allowed_environments": ["'"${ENV_ID}"'"]
  }' | jq -r '.provider_key.id')

echo "$PROVIDER_KEY_ID"

❶ provider is cohere. The AISIX Cloud Admin API derives the adapter from the catalog provider; the adapter field is only accepted on BYO provider keys.

❷ api_key stores the Cohere API key. It follows the credential-handling behavior in Provider Keys.

❸ api_base points at Cohere's compatibility API. The path carries a /compatibility segment before the /v1 version segment, because the OpenAI-shaped routes are served on a separate prefix from Cohere's native API. The bare host root, such as https://api.cohere.com, works as well: see How AISIX Builds Cohere URLs. For the cohere catalog provider the field is optional — the AISIX Cloud Admin API fills in the same value when you omit it — but the examples set it explicitly so the surface each key targets stays visible in the configuration.

The command captures the returned provider key ID in PROVIDER_KEY_ID.

Create a Model

Cohere model IDs combine a family name with a release date: command-a-03-2025 is the March 2025 snapshot of the command-a family, and capability variants add a suffix before the date, as in command-a-reasoning-08-2025 or command-a-vision-07-2025. Because the date is part of the ID, a model alias created from a catalog ID pins one snapshot. Check the Cohere models list for current IDs.

Create the model alias callers will send in requests:

MODEL_ID=$(curl -sS -X POST "$AISIX_CP/environments/$ENV_ID/models" \
  -H "Authorization: Bearer $AISIX_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{
    "display_name": "cohere-command-a-prod",
    "model_name": "command-a-03-2025",
    "provider_key_id": "'"${PROVIDER_KEY_ID}"'"
  }' | jq -r '.model.id')

echo "$MODEL_ID"

❶ display_name is the alias callers send in model.

❷ model_name is the Cohere model ID, for example command-a-03-2025, command-a-plus-05-2026, or command-a-reasoning-08-2025.

❸ provider_key_id attaches the alias to the Cohere provider key.

Create a Caller API Key

Create the caller API key that can access the model alias. The plaintext key is server-generated and returned once in the response — store it securely:

AISIX_API_KEY=$(curl -sS -X POST "$AISIX_CP/environments/$ENV_ID/api_keys" \
  -H "Authorization: Bearer $AISIX_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{
    "display_name": "cohere-caller",
    "allowed_models": ["'"${MODEL_ID}"'"]
  }' | jq -r '.plaintext')

echo "$AISIX_API_KEY"

The allowed_models value must reference the model ID captured in the previous step. After the write, the configuration projects to attached gateways automatically.

Configure with the Open-Source AISIX Gateway

Export the upstream credential and choose the caller API key that applications will send to the gateway:

export COHERE_API_KEY="YOUR_PROVIDER_API_KEY"
export CALLER_API_KEY="YOUR_CALLER_API_KEY"

For a new gateway, use this complete resources file. For an existing gateway, merge the entries into its current file, preserving its other resources:

_format_version: "1"

provider_keys:
  - display_name: "cohere-prod"
    provider: "cohere"
    adapter: "openai"
    api_key: ${COHERE_API_KEY}
    api_base: "https://api.cohere.ai/compatibility/v1"

models:
  - display_name: "cohere-command-a-prod"
    provider: "cohere"
    model_name: "command-a-03-2025"
    provider_key: "cohere-prod"

api_keys:
  - display_name: "cohere-caller"
    key_env: CALLER_API_KEY
    allowed_models:
      - "cohere-command-a-prod"

If AISIX is installed locally, validate the file before loading it:

aisix validate --resources resources.yaml

After validation, start the gateway with the referenced environment variables in its process environment. Reload an existing gateway only if those variables are already available to the process; otherwise, restart it with the updated environment.

If you use Docker, adapt the validation and startup commands in the Open-Source AISIX Gateway Quickstart. Mount this resources.yaml file and pass every environment variable it references with -e in both commands. After the resources load, prepare the shared verification request below:

export AISIX_API_KEY="$CALLER_API_KEY"

Verify the Provider Connection

Export the AISIX gateway origin:

# AISIX_PROXY has no trailing slash or endpoint path
# The local quickstarts use http://127.0.0.1:3000
export AISIX_PROXY="YOUR_AISIX_GATEWAY_URL"

Send a chat-completions request through the AISIX proxy:

curl -sS -X POST "$AISIX_PROXY/v1/chat/completions" \
  -H "Authorization: Bearer ${AISIX_API_KEY}" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "cohere-command-a-prod",
    "messages": [
      {
        "role": "user",
        "content": "Say hello from Cohere."
      }
    ]
  }'

The gateway returns an OpenAI-compatible response that echoes the caller-facing alias cohere-command-a-prod. If the request fails, check the provider key api_key, api_base, and the Cohere model ID in model_name. A 404 from the upstream usually means api_base carries a path other than the forms listed in How AISIX Builds Cohere URLs.

How AISIX Builds Cohere URLs

A Cohere provider key reaches the right endpoint for chat completions, embeddings, and rerank with either form of api_base: the bare host root, such as https://api.cohere.com, or the compatibility base, such as https://api.cohere.ai/compatibility/v1. AISIX removes a trailing /compatibility/v1, /v1, or /v2 from api_base to get the host root, then builds each route from that root:

A base with any other path, such as a private forwarding proxy, keeps that path, and AISIX appends /compatibility/v1 or /v2/rerank to it as written.

Control Reasoning Effort

Cohere's reasoning-capable Command models, such as command-a-reasoning-08-2025 and command-a-plus-05-2026, expose thinking differently on each API surface. Because AISIX routes to the compatibility API, callers control it with the OpenAI-shaped reasoning_effort field:

{
  "reasoning_effort": "none"
}

Cohere's compatibility API accepts none and high on this field, which map to disabled and enabled thinking. The native API's token-budget control is not documented on this surface, so a request cannot cap the reasoning token count through the compatibility route. See Cohere's Reasoning documentation for the current field behavior.

AISIX preserves reasoning_content in responses and normalizes reasoning to that canonical field. The Cohere provider key carries no reasoning-field override. If your model streams reasoning under a different delta path, set response.reasoning_field on the provider key.

Add a Rerank Model

The /v1/rerank route accepts a model whose provider value is openai, cohere, or jina, so a Cohere-backed alias is admitted on this route. Cohere rerank is not part of the compatibility API; AISIX sends it to Cohere's native v2 rerank endpoint under the host root of the provider key's api_base, as described in How AISIX Builds Cohere URLs. The provider key you created for chat completions therefore serves rerank too.

In AISIX Cloud, create the rerank model alias on that provider key and a caller key scoped to it:

RERANK_MODEL_ID=$(curl -sS -X POST "$AISIX_CP/environments/$ENV_ID/models" \
  -H "Authorization: Bearer $AISIX_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{
    "display_name": "cohere-rerank-prod",
    "model_name": "rerank-v3.5",
    "provider_key_id": "'"${PROVIDER_KEY_ID}"'"
  }' | jq -r '.model.id')

RERANK_API_KEY=$(curl -sS -X POST "$AISIX_CP/environments/$ENV_ID/api_keys" \
  -H "Authorization: Bearer $AISIX_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{
    "display_name": "cohere-rerank-caller",
    "allowed_models": ["'"${RERANK_MODEL_ID}"'"]
  }' | jq -r '.plaintext')

For the open-source AISIX gateway, add cohere-rerank-prod to models on the existing cohere-prod provider key. Replace the existing cohere-caller entry with the updated entry below so it allows both model aliases. Preserve unrelated entries and collections:

models:
  - display_name: "cohere-rerank-prod"
    provider: "cohere"
    model_name: "rerank-v3.5"
    provider_key: "cohere-prod"

api_keys:
  - display_name: "cohere-caller"
    key_env: CALLER_API_KEY
    allowed_models:
      - "cohere-command-a-prod"
      - "cohere-rerank-prod"

Validate and reload or restart the declarative resources file as described above, then use the existing caller key for the rerank request:

export RERANK_API_KEY="$CALLER_API_KEY"

Rerank model IDs follow their own naming, separate from the Command families: rerank-v3.5, and the two Rerank 4.0 variants rerank-v4.0-fast and rerank-v4.0-pro. Because this route resolves to Cohere's v2 rerank endpoint, confirm on the rerank model overview that the ID you pick is served there before you point a production alias at it.

Send a rerank request through the proxy:

curl -sS -X POST "$AISIX_PROXY/v1/rerank" \
  -H "Authorization: Bearer ${RERANK_API_KEY}" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "cohere-rerank-prod",
    "query": "How do I rotate a provider credential?",
    "documents": [
      "Provider keys store the upstream credential.",
      "Caller API keys authorize model access.",
      "Rate limits apply per caller key."
    ],
    "top_n": 2
  }'

AISIX rewrites only the model field to rerank-v3.5 and forwards the body unchanged, so Cohere's top_n parameter reaches the upstream as written. The response keeps Cohere's rerank shape: a results array ordered by relevance_score, plus a meta object. Cohere's rerank models report usage as meta.billed_units.search_units, with no token count. AISIX does not price search units, so each rerank request is recorded in the usage log with zero tokens: it appears in Request Logs alongside chat traffic but adds no cost to budget totals. When a Cohere response does include meta.billed_units.input_tokens, AISIX records that count.

Endpoint Support for Cohere

The routes below behave as follows for a model alias backed by a Cohere provider key, with api_base in either form described in How AISIX Builds Cohere URLs.

Next Steps

You have now connected AISIX to Cohere, verified the model alias, and added a rerank route. Continue with these guides:

• Model Aliases: configure routing, retry behavior, or cost metadata for these aliases.
• Rerank: review the rerank request contract and its provider requirement.
• Routing and Failover: fail over between Cohere and a second provider.
• Provider Compatibility: review supported proxy endpoints and provider-specific boundaries.