Chat Completions

POST /v1/chat/completions

OpenAI-compatible chat completions with streaming support.

Request

{
  "model": "auto",
  "messages": [
    {"role": "system", "content": "You are helpful."},
    {"role": "user", "content": "Hello"}
  ],
  "temperature": 0.7,
  "max_tokens": 2048,
  "stream": false
}

max_completion_tokens is accepted as an alias for max_tokens (newer OpenAI SDKs send only the new name).

mycellm routing (optional)

{
  "model": "auto",
  "messages": [{"role": "user", "content": "Write code"}],
  "mycellm": {
    "min_tier": "capable",
    "required_tags": ["code"],
    "routing": "quality",
    "fallback": "downgrade"
  }
}

Field	Type	Description
min_tier	string	tiny, fast, capable, frontier
required_tags	string[]	code, reasoning, vision
routing	string	best (quality) or fastest (latency)
fallback	string	downgrade or reject

Vision (multimodal)

Send images alongside text using OpenAI-style content parts. A user message’s content becomes an array of text and image_url parts; images may be a data: URL (base64) or an https:// URL.

{
  "model": "auto",
  "messages": [
    {"role": "user", "content": [
      {"type": "text", "text": "What's in this image?"},
      {"type": "image_url", "image_url": {"url": "data:image/jpeg;base64,/9j/4AAQ..."}}
    ]}
  ]
}

Image requests are routed to a vision-capable node (e.g. a Qwen2.5-VL or Gemma-3 vision model); the gateway will not silently fall back to a text-only model. If no vision node is available you’ll get a 503 with a clear message rather than a hallucinated answer. Nodes that load a vision model advertise the vision tag automatically.

Response

{
  "id": "chatcmpl-abc123",
  "object": "chat.completion",
  "created": 1711234567,
  "model": "Qwen2.5-3B-Instruct-Q8_0",
  "choices": [{
    "index": 0,
    "message": {"role": "assistant", "content": "Hello!"},
    "finish_reason": "stop"
  }],
  "usage": {
    "prompt_tokens": 12,
    "completion_tokens": 3,
    "total_tokens": 15
  }
}

Streaming

Set "stream": true. Response is Server-Sent Events:

data: {"id":"chatcmpl-abc","object":"chat.completion.chunk","model":"Qwen2.5-3B","choices":[{"delta":{"content":"Hello"},"finish_reason":null}]}

data: {"id":"chatcmpl-abc","object":"chat.completion.chunk","model":"Qwen2.5-3B","choices":[{"delta":{},"finish_reason":"stop"}]}

data: [DONE]

With "stream_options": {"include_usage": true} (0.6.3+), a final chunk with empty choices carries exact backend-counted token usage before [DONE]:

data: {"id":"chatcmpl-abc","object":"chat.completion.chunk","model":"Qwen2.5-3B","choices":[],"usage":{"prompt_tokens":18,"completion_tokens":42,"total_tokens":60}}

The usage chunk is only emitted when the backend reported real counts — it is never fabricated.