# LLM Proxy Contract

> **Status: not yet implemented on the backend.** This document is the contract that `lib-llm-providers-ui` expects from arcadia. Implement `POST /api/v1/ai/llm/chat` server-side to make `mode: "proxy"` work in the client.

## Why a proxy?

The Settings UI ships in two transport modes:

- **`direct`** — the browser fetches the API key from arcadia's vault (`GET /api/v1/secrets/:name`), then calls OpenAI/Anthropic/DeepSeek/Qwen directly. Works today, but the key briefly lives in browser memory and the prompt contents go straight to the upstream provider with no opportunity for arcadia to log, meter, or rewrite them.
- **`proxy`** — the browser sends the chat request to arcadia, which reads the secret server-side and calls the upstream provider. Keys never leave arcadia. This is what production should use.

This contract only covers the proxy mode.

## Endpoint

```
POST /api/v1/ai/llm/chat
Authorization: Bearer <arcadia session token>
X-Tenant-ID:   <tenant id>
Content-Type:  application/json
```

The path is `/api/v1/ai/llm/chat` so it lives under the existing `/api/v1/ai/*` scope (next to `embeddings`, `tools`, `llm/usage`).

## Request body

The shape is OpenAI's chat-completion request, **plus** two arcadia-specific fields:

```json
{
  "provider":    "openai",
  "secret_name": "llm-openai-api-key",
  "model":       "gpt-4o-mini",
  "messages": [
    { "role": "system", "content": "You are a helpful assistant." },
    { "role": "user",   "content": "Hello!" }
  ],
  "stream":      true,
  "max_tokens":  1024,
  "temperature": 0.7,
  "tools": [
    {
      "type": "function",
      "function": {
        "name": "search_docs",
        "description": "...",
        "parameters": { "type": "object", "properties": {} }
      }
    }
  ],
  "tool_choice": "auto"
}
```

### Provider-specific fields

| Field         | Type                                            | Notes |
|---------------|-------------------------------------------------|-------|
| `provider`    | `"openai" \| "anthropic" \| "deepseek" \| "qwen" \| "lmstudio"` | Selects the upstream backend. |
| `secret_name` | `string` (optional for `lmstudio`)              | Name of the vault secret holding the upstream API key. The proxy resolves it via the same `Secrets.get/3` used for tenant-facing reads. |

The proxy must:
1. Authenticate the arcadia session.
2. Resolve `secret_name` for the current tenant (or fall back to platform-level). Refuse the call if the secret is disabled, expired, or IP-blocked. The existing `Arcadia.Secrets.get/3` already returns the right error codes.
3. Map the request to the upstream's native shape (Anthropic's `/v1/messages` differs from OpenAI's `/v1/chat/completions`).
4. Forward it with the resolved key as the upstream's expected auth header (`Authorization: Bearer <key>` for OpenAI/DeepSeek/Qwen, `x-api-key: <key>` + `anthropic-version: 2023-06-01` for Anthropic).
5. Stream the response back as **OpenAI-shape SSE** regardless of upstream. (See "Response — streaming" below.)
6. Record a usage row via the existing `POST /ai/llm/usage` after each completion.

## Response — non-streaming (`stream: false`)

OpenAI chat-completion shape, returned as a single JSON document:

```json
{
  "id": "chatcmpl-...",
  "object": "chat.completion",
  "created": 1714512000,
  "model": "gpt-4o-mini",
  "choices": [
    {
      "index": 0,
      "finish_reason": "stop",
      "message": {
        "role": "assistant",
        "content": "Hi there!",
        "tool_calls": null
      }
    }
  ],
  "usage": {
    "prompt_tokens": 12,
    "completion_tokens": 4,
    "total_tokens": 16
  }
}
```

For Anthropic upstream, translate `usage.input_tokens` / `output_tokens` → `prompt_tokens` / `completion_tokens` and combine `content` blocks into a single string (or surface `tool_use` blocks via `tool_calls`).

## Response — streaming (`stream: true`)

Server-Sent Events, one event per delta, terminated with `data: [DONE]`. Each `data:` line is JSON of OpenAI's chat-completion *delta* shape:

```
data: {"id":"chatcmpl-...","object":"chat.completion.chunk","created":1714512000,"model":"gpt-4o-mini","choices":[{"index":0,"delta":{"role":"assistant","content":""},"finish_reason":null}]}

data: {"id":"chatcmpl-...","object":"chat.completion.chunk","created":1714512000,"model":"gpt-4o-mini","choices":[{"index":0,"delta":{"content":"Hi"},"finish_reason":null}]}

data: {"id":"chatcmpl-...","object":"chat.completion.chunk","created":1714512000,"model":"gpt-4o-mini","choices":[{"index":0,"delta":{"content":" there"},"finish_reason":null}]}

data: {"id":"chatcmpl-...","object":"chat.completion.chunk","created":1714512000,"model":"gpt-4o-mini","choices":[{"index":0,"delta":{},"finish_reason":"stop"}]}

data: [DONE]
```

For Anthropic upstream, translate `content_block_delta` events of type `text_delta` into delta `content` strings, and `message_stop` into the `finish_reason: "stop"` event. Tool calls translate `content_block_start` of type `tool_use` (with id + name) and the streaming JSON arguments into OpenAI-shape `delta.tool_calls` entries.

The client uses the OpenAI parser in `@crema/llm-ui` (`OpenAICompatibleAdapter.stream()`), so any deviation from this shape will manifest as missing tokens or hung streams.

## Errors

Use the existing `ArcadiaWeb.FallbackController` envelope:

```json
{ "error": { "code": "secret_disabled", "message": "Secret is disabled" } }
```

Specific codes the client distinguishes:

| HTTP | code                    | When |
|------|-------------------------|------|
| 401  | `unauthorized`          | Missing / invalid arcadia session. |
| 403  | `secret_disabled`       | Vault returned `:disabled`. |
| 410  | `secret_expired`        | Vault returned `:expired`. |
| 410  | `secret_consumed`       | Read-once secret already consumed. |
| 403  | `ip_not_allowed`        | Caller IP blocked by the vault allowlist. |
| 404  | `unknown_provider`      | `provider` field not in the supported set. |
| 502  | `upstream_unavailable`  | Upstream returned 5xx or timed out. |
| 429  | `rate_limited`          | Either arcadia or upstream returned 429. Pass through `Retry-After` if present. |

## Auth

The proxy must verify the arcadia session bearer the same way the rest of `/api/v1/*` does. The vault read uses the **caller's tenant context**, so platform-admin sessions can use platform-level secrets and tenant sessions can use their own — no special privilege required beyond what `/api/v1/secrets/:name` already enforces.

## Usage tracking

After each completion (success or failure), write a row via the existing `POST /api/v1/ai/llm/usage` (or call the equivalent context module directly inside the proxy). Required fields on that endpoint already include model, prompt_tokens, completion_tokens, latency_ms — the proxy can fill them from the upstream response.

## Test fixture

A minimal Mix test in `apps/arcadia_core/test/arcadia_web/controllers/api/ai_controller_test.exs` should cover:

- 200 with stream off, OpenAI upstream stubbed via Bypass.
- 200 with stream on, Anthropic upstream stubbed; assert SSE chunks carry OpenAI-shape JSON.
- 403 when the named secret is disabled.
- 404 when `provider: "unknown"`.
- Usage row written on the success cases.