/v1/chat/completions

Request

POST /v1/chat/completions
Content-Type: application/json

Body

{
  "model": "fastlane-auto",
  "messages": [
    {"role": "user", "content": "What is 2+2?"}
  ],
  "temperature": 0.7,
  "max_tokens": 1000,
  "stream": false
}

Parameters

Parameter	Type	Required	Description
model	string	Yes	Model ID or fastlane-auto
messages	array	Yes	Chat messages
temperature	float	No	Sampling temperature (0-2)
max_tokens	int	No	Max response tokens
stream	bool	No	Enable SSE streaming

Response

{
  "id": "chatcmpl-abc123",
  "object": "chat.completion",
  "created": 1700000000,
  "model": "gpt-4o-mini",
  "choices": [
    {
      "index": 0,
      "message": {
        "role": "assistant",
        "content": "4"
      },
      "finish_reason": "stop"
    }
  ],
  "usage": {
    "prompt_tokens": 10,
    "completion_tokens": 1,
    "total_tokens": 11
  }
}

Streaming

Set "stream": true to receive Server-Sent Events:

curl -N http://127.0.0.1:8787/v1/chat/completions \
  -H "Content-Type: application/json" \
  -d '{
    "model": "fastlane-auto",
    "messages": [{"role": "user", "content": "Hello"}],
    "stream": true
  }'

Routing with fastlane-auto

When you use model: "fastlane-auto", Fastlane:

1. Extracts the prompt text
2. Runs it through the ONNX ranking model
3. Picks the best provider based on your mode setting
4. Forwards the request to that provider

The response includes the actual model used in the model field.