Models API

GET /v1/models

List all available models across the network (local + QUIC peers + fleet).

{
  "object": "list",
  "data": [
    {"id": "Qwen2.5-3B-Instruct-Q8_0", "object": "model", "owned_by": "local", "context_length": 8192},
    {"id": "Mistral-Small-24B-Q4_K_M", "object": "model", "owned_by": "fleet:aurora"},
    {"id": "relay:llama3.2:3b", "object": "model", "owned_by": "relay:ipad"}
  ]
}

context_length is the effective window — if the memory preflight clamped the configured context to fit this node, the clamped value is reported, so agents can size prompts and auto-compaction deterministically.

Models prefixed with relay: are served by external devices connected via relay backends.

POST /v1/node/models/load

Local GGUF model

{
  "model_path": "/path/to/model.gguf",
  "name": "my-model",
  "ctx_len": 4096
}

MLX model options (Apple Silicon)

MLX loads accept additional per-model options (all optional, persisted in the saved config):

{
  "backend": "mlx",
  "model_path": "mlx-community/Qwen3-1.7B-4bit",
  "ctx_len": 32768,
  "max_kv_size": 8192,
  "draft_model": "mlx-community/Qwen3-0.6B-4bit",
  "num_draft_tokens": 3
}

• max_kv_size — bounded (rotating) KV cache: caps per-request KV memory at this many tokens; older entries are overwritten. The preflight then estimates KV at min(ctx_len, max_kv_size). Good for memory-tight nodes.
• draft_model / num_draft_tokens — opt-in speculative decoding on the single-request path. Requires a same-tokenizer draft; mutually exclusive with max_kv_size. Benchmark before enabling — on M1-class hardware it is typically slower than plain decoding.
• backend: "mlx-embeddings" loads an embedding model (MiniLM / BERT / XLM-R family) that serves /v1/embeddings with length-grouped batching.

API Provider model

{
  "name": "claude-sonnet",
  "backend": "openai",
  "api_base": "https://openrouter.ai/api/v1",
  "api_key": "secret:openrouter",
  "api_model": "anthropic/claude-sonnet-4",
  "max_concurrent": 32
}

Use secret:name to reference encrypted secrets instead of raw API keys.

max_concurrent controls how many simultaneous requests this model can handle (default: 32 for API/relay, 1 for local GGUF). See relay docs for details.

POST /v1/node/models/download

Installs a model. Takes one of three forms, distinguished by which field you send. Served by both Python and iOS nodes with the same request shape.

From Hugging Face

{"repo_id": "bartowski/Qwen2.5-3B-Instruct-GGUF", "filename": "Qwen2.5-3B-Instruct-Q4_K_M.gguf"}

The finished file is verified against the repo’s published content hash (raw SHA-256 for LFS weights, git blob SHA-1 for small files). Progress entries in /v1/node/models/downloads carry a verified field (sha256 / git-sha1 / unverified); a hash mismatch deletes the file and fails the download.

From an arbitrary URL (single file)

For a model the Hub doesn’t have — a private fine-tune, or anything an air-gapped fleet has to stage itself.

{
  "url": "https://models.example.org/my-finetune-Q4_K_M.gguf",
  "sha256": "9f86d081884c7d659a2feaa0c55ad015a3bf4f1b2b0b822cd15d6c15b0f00a08",
  "filename": "my-finetune-Q4_K_M.gguf"
}

filename is optional and defaults to the URL’s last path component.

From a manifest (MLX directory)

An MLX model is a directory — config, tokenizer and one or more safetensors shards that only mean anything together — so the install form is per-file. name is the directory it publishes as.

{
  "name": "my-finetune-mlx",
  "files": [
    {"path": "config.json",       "url": "https://…/config.json",       "sha256": "…"},
    {"path": "model.safetensors", "url": "https://…/model.safetensors", "sha256": "…", "size": 4128},
    {"path": "tokenizer.json",    "url": "https://…/tokenizer.json",    "sha256": "…"}
  ]
}

A manifest must include config.json and at least one .safetensors file. A directory missing either is refused before anything is fetched, rather than published as something the model picker offers and the engine cannot load.

Rules for URL and manifest installs

These are enforced, not advisory — an installer that accepts arbitrary URLs is a remote-code-execution surface pointed at every node in the fleet.

Rule	Behaviour
https only	http is refused
sha256 required	64 hex characters, per file; verified as the bytes land
Digest mismatch	Fails the install and deletes the staged bytes
path must be relative	.. and absolute paths refused
Staging	Files land in .staging/ and publish by rename, so an interrupted install leaves nothing half-usable

Hugging Face tokens are not attached to non-Hub requests — the auth header is built per download rather than per session, so a url install cannot leak your token to whatever host it points at.

Metered networks (iOS)

An iOS node refuses a download on cellular or in Low Data Mode with 409 expensive_network, reporting network and estimated_bytes. Send "allow_expensive": true to override per request, or grant standing permission in Settings → Downloads.

POST /v1/node/models/unload

{"model": "my-model"}

Model Tiers

Models are classified by parameter count:

Tier	Parameters	Examples
Tier 1	up to 8B	Qwen 7B, Llama 8B, Phi-4
Tier 2	up to 70B	Mistral Small 24B, Llama 70B
Tier 3	70B+	Qwen 72B, Llama 405B

Access is governed by credit balance — earned by contributing GPU time to the network:

Tier	Credit balance	Access
Free	Anonymous / < 10	Tier 1 models, 5K tokens/day
Contributor	10+ credits	Tier 1 + 2 models, 50K tokens/day
Power Seeder	50+ credits	All tiers, highest daily limits

Credits are earned by serving inference and spent by consuming it. The system is designed to be fair — contribute compute, get compute back.