Build on a governed AI gateway

Introduction#

Sophy is an OpenAI-compatible gateway and operator console. Applications send input in familiar wire formats; Sophy authenticates the key, resolves its centrally managed configuration, calls the bound model with the project-owned credential for that model’s provider — the project’s Vercel AI Gateway credential, or its Cloudflare Workers AI credential for Cloudflare models — and records project-attributed usage.

Compatibility is intentionally scoped. Chat Completions and Responses cover text, streaming, multimodal input, and function-tool loops. Dedicated routes cover audio transcription, embeddings, image generation, short-lived file uploads, and the primary model bound to a key. Evaluation and extraction models are reached through native Sophy routes rather than an OpenAI client method. The sections below call out differences from the full OpenAI platform.

Supported API surfaces#

Authentication#

Send the Sophy key as a Bearer token. Keys look like mw_live_… and are issued inside a project in the Sophy console. The console and CLI use email codes for operator sign-in. A verified email can create a renameable My Project or accept an Admin invitation.

Authorization: Bearer mw_live_…

Missing Bearer authentication returns 401 missing_api_key. A malformed, unknown, revoked, expired, or inactive-project key returns 401 invalid_api_key. Keys are checked against the database on every request, so edits and revocation apply immediately.

Quickstart#

Change the base URL and API key, then call one of the supported methods. The placeholder model is required by some OpenAI SDK methods but Sophy uses the model bound to the key.

from openai import OpenAI

client = OpenAI(
    base_url="https://sophy.in/v1",
    api_key="mw_live_…",
)

resp = client.chat.completions.create(
    model="sophy",  # ignored — the key owns the model
    messages=[{"role": "user", "content": "Hello, Sophy!"}],
)
print(resp.choices[0].message.content)

import OpenAI from "openai";

const client = new OpenAI({
  baseURL: "https://sophy.in/v1",
  apiKey: "mw_live_…",
});

const resp = await client.chat.completions.create({
  model: "sophy",  // ignored — the key owns the model
  messages: [{ role: "user", content: "Hello, Sophy!" }],
});
console.log(resp.choices[0].message.content);

curl https://sophy.in/v1/chat/completions \
  -H "Authorization: Bearer mw_live_…" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "sophy",
    "messages": [{"role": "user", "content": "Hello, Sophy!"}]
  }'

Chat Completions#

Key-owned or ignored: model, temperature, top_p, max_tokens, max_completion_tokens, response_format, n, and user. Sophy applies the key’s configured model and generation parameters instead.

Buffered response

{
  "id": "chatcmpl-…",
  "object": "chat.completion",
  "created": 1735689600,
  "model": "anthropic/claude-sonnet-4.6",
  "choices": [{
    "index": 0,
    "message": { "role": "assistant", "content": "Hello! How can I help?" },
    "finish_reason": "stop"
  }],
  "usage": {
    "prompt_tokens": 9,
    "completion_tokens": 7,
    "total_tokens": 16,
    "prompt_tokens_details": { "cached_tokens": 0 }
  }
}

Streaming

A stream begins with an assistant-role chunk, emits text and/or tool-call deltas, finishes with a reason chunk, optionally emits usage, and ends with data: [DONE].

data: {"id":"chatcmpl-…","object":"chat.completion.chunk","created":1735689600,"model":"anthropic/claude-sonnet-4.6","choices":[{"index":0,"delta":{"role":"assistant"},"finish_reason":null}]}

data: {"id":"chatcmpl-…","object":"chat.completion.chunk","created":1735689600,"model":"anthropic/claude-sonnet-4.6","choices":[{"index":0,"delta":{"content":"Hello"},"finish_reason":null}]}

data: {"id":"chatcmpl-…","object":"chat.completion.chunk","created":1735689600,"model":"anthropic/claude-sonnet-4.6","choices":[{"index":0,"delta":{},"finish_reason":"stop"}]}

data: [DONE]

Responses API#

Use client.responses.create(...) for a Responses-compatible text, multimodal, streaming, or function-tool workflow. Sophy is stateless: send the full conversation and any prior function calls/results in input on every request.

Key-owned or ignored: model, temperature, top_p, max_output_tokens, and text.format.

from openai import OpenAI

client = OpenAI(base_url="https://sophy.in/v1", api_key="mw_live_…")

resp = client.responses.create(
    model="sophy",
    input="Write a haiku about gateways.",
)
print(resp.output_text)

curl https://sophy.in/v1/responses \
  -H "Authorization: Bearer mw_live_…" \
  -H "Content-Type: application/json" \
  -d '{"model":"sophy","input":"Write a haiku about gateways."}'

{
  "id": "resp_…",
  "object": "response",
  "created_at": 1735689600,
  "status": "completed",
  "model": "anthropic/claude-sonnet-4.6",
  "store": false,
  "output": [{
    "id": "msg_…",
    "type": "message",
    "status": "completed",
    "role": "assistant",
    "content": [{ "type": "output_text", "text": "Silent routes converge…", "annotations": [] }]
  }],
  "usage": {
    "input_tokens": 11,
    "input_tokens_details": { "cached_tokens": 0 },
    "output_tokens": 9,
    "output_tokens_details": { "reasoning_tokens": 0 },
    "total_tokens": 20
  }
}

Streaming uses events such as response.created, response.output_text.delta, response.function_call_arguments.delta, and response.completed, each with an increasing sequence_number. There is no [DONE] sentinel.

Multimodal input#

Language keys can send public image/file URLs or URLs returned by POST /v1/files, provided the bound model supports the content type. Sophy ownership-checks its own uploads; public external URLs pass through to the model provider.

uploaded_url = "https://…/uploads/<project>/<key>/invoice-….pdf"
uploaded_filename = "invoice.pdf"

resp = client.chat.completions.create(
    model="sophy",
    messages=[{
        "role": "user",
        "content": [
            {"type": "text", "text": "Summarize this document."},
            {
                "type": "file",
                "file": {
                    "file_url": uploaded_url,
                    "filename": uploaded_filename,
                },
            },
        ],
    }],
)

uploaded_url = "https://…/uploads/<project>/<key>/photo-….png"

resp = client.responses.create(
    model="sophy",
    input=[{
        "role": "user",
        "content": [
            {"type": "input_text", "text": "What is shown here?"},
            {"type": "input_image", "image_url": uploaded_url},
        ],
    }],
)

Tool / function calling#

Sophy passes function definitions to the bound language model and maps its requested calls back to the selected OpenAI wire format. Sophy never executes a tool. Your application runs it, then sends the result and full conversation state on the next call.

tools = [{
    "type": "function",
    "function": {
        "name": "get_weather",
        "description": "Get weather for a city",
        "parameters": {
            "type": "object",
            "properties": {"city": {"type": "string"}},
            "required": ["city"],
        },
    },
}]

first = client.chat.completions.create(
    model="sophy",
    tools=tools,
    messages=[{"role": "user", "content": "Weather in Paris?"}],
)
call = first.choices[0].message.tool_calls[0]

second = client.chat.completions.create(model="sophy", tools=tools, messages=[
    {"role": "user", "content": "Weather in Paris?"},
    first.choices[0].message,
    {"role": "tool", "tool_call_id": call.id, "content": "18°C, sunny"},
])

tools = [{
    "type": "function",
    "name": "get_weather",
    "description": "Get weather for a city",
    "parameters": {
        "type": "object",
        "properties": {"city": {"type": "string"}},
        "required": ["city"],
    },
}]

first = client.responses.create(
    model="sophy",
    tools=tools,
    input="Weather in Paris?",
)
call = next(item for item in first.output if item.type == "function_call")

second = client.responses.create(model="sophy", tools=tools, input=[
    {"role": "user", "content": "Weather in Paris?"},
    {
        "type": "function_call",
        "call_id": call.call_id,
        "name": call.name,
        "arguments": call.arguments,
    },
    {
        "type": "function_call_output",
        "call_id": call.call_id,
        "output": "18°C, sunny",
    },
])

Only function tools are supported; other Responses tool types are ignored. Chat uses the nested { type: "function", function: { ... } } shape; Responses uses the flat { type: "function", name, parameters } shape. Legacy Chat functions returns 400 functions_unsupported.

Structured output#

An operator can attach a JSON Schema to a language key. Sophy normalizes the schema across supported model providers. Buffered calls use constrained generation, tolerant JSON extraction and retry, then validate the completed object; streaming uses schema-constrained generation with post-hoc validation.

The schema lives on the key. Client-sent Chat response_format and Responses text.format do not replace it. Successful JSON is returned as the assistant text, so parse choices[0].message.content or response.output_text.

Audio transcription#

Bind the key’s primary model to a transcription model and send audio as multipart/form-data. Sophy returns raw text when no processing policy is set, or applies centrally managed instructions in a second language-model step and returns only that processed text.

Accepted audio formats are MP3/MPEG/MPGA, MP4/M4A, WAV, WebM, FLAC, and OGG. Sophy identifies the format from the file bytes instead of trusting its name or client-declared MIME type. Files outside that allowlist return 400 unsupported_audio_format; files over the limit return 413 file_too_large.

from openai import OpenAI

client = OpenAI(base_url="https://sophy.in/v1", api_key="mw_live_…")

with open("meeting.m4a", "rb") as audio:
    resp = client.audio.transcriptions.create(
        model="sophy",
        file=audio,
        language="en",
        response_format="json",
    )

print(resp.text)

curl https://sophy.in/v1/audio/transcriptions \
  -H "Authorization: Bearer mw_live_…" \
  -F "file=@meeting.m4a" \
  -F "model=sophy" \
  -F "language=en" \
  -F "response_format=json"

{
  "text": "Decisions: launch on Friday and send the checklist today.",
  "processed": true,
  "language": "en",
  "duration": 8.4
}

One uploaded clip consumes one RPM slot and counts as one client request. Sophy attributes transcription and processing to their actual models, while the processor component does not add a second client request. If content logging is enabled on the key, Sophy stores the filename, byte count, language hint, raw transcript, and final text—but never the audio bytes. Turn content logging off when unprocessed text must not be retained.

Embeddings#

Bind the key to an embedding model and use the standard OpenAI embeddings method. The key-owned model is returned in the response, and usage records input tokens and estimated cost.

Pre-tokenized integer arrays are not portable across a gateway-bound model and return 400 token_input_unsupported.

from openai import OpenAI

client = OpenAI(base_url="https://sophy.in/v1", api_key="mw_live_…")

resp = client.embeddings.create(
    model="sophy",
    input=["First document", "Second document"],
    encoding_format="float",
)
print(resp.data[0].embedding)

curl https://sophy.in/v1/embeddings \
  -H "Authorization: Bearer mw_live_…" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "sophy",
    "input": ["First document", "Second document"],
    "encoding_format": "float"
  }'

{
  "object": "list",
  "data": [
    { "object": "embedding", "index": 0, "embedding": [0.012, -0.044, 0.008] },
    { "object": "embedding", "index": 1, "embedding": [-0.031, 0.017, 0.052] }
  ],
  "model": "openai/text-embedding-3-small",
  "usage": { "prompt_tokens": 5, "total_tokens": 5 }
}

Evaluation models#

Evaluation models are not exposed through the OpenAI-compatible client methods, so call this native Sophy route directly with an HTTP client. Bind the key to an evaluation model — one from the Vercel AI Gateway catalog, such as TypeSafe Jev, or one of Cloudflare’s Clef models — send the state to judge together with a named set of typed questions, and Sophy returns one answer per question plus token usage.

Questions are judged against the same shared state, so keep each one self-contained. The call is a single buffered round trip; there is no streaming.

Question types and answers

Requests are validated before any paid call. Missing or oversized state returns 400 missing_state or 400 state_too_large; a bad question set returns 400 missing_questions, 400 invalid_questions, 400 questions_empty, 400 too_many_questions, 400 invalid_question, 400 missing_instructions, 400 unsupported_question_type, or 400 invalid_criteria. Out-of-namespace knobs return 400 unsupported_provider_options, and a key whose model is known to be something other than an evaluation model — including Clairic and any unsupported @cf/ model — returns 400 model_not_evaluation. Clef keys add the checks described below.

import httpx

resp = httpx.post(
    "https://sophy.in/v1/evaluate",
    headers={"Authorization": "Bearer mw_live_…"},
    json={
        "state": {
            "question": "How do I reset my password?",
            "answer": "Open Settings and use the reset link we email you.",
        },
        "questions": {
            "grounded": {
                "type": "boolean",
                "instructions": "Is the answer supported by the question?",
            },
            "tone": {
                "type": "choice",
                "instructions": "Classify the tone of the answer.",
                "criteria": {
                    "formal": "Professional and impersonal.",
                    "casual": "Friendly and conversational.",
                },
            },
            "helpfulness": {
                "type": "score",
                "instructions": "Rate how helpful the answer is.",
                "criteria": ["useless", "partial", "complete"],
            },
        },
    },
    timeout=120,
)
print(resp.json()["answers"])

curl https://sophy.in/v1/evaluate \
  -H "Authorization: Bearer mw_live_…" \
  -H "Content-Type: application/json" \
  -d '{
    "state": "Q: How do I reset my password? A: Open Settings and use the reset link.",
    "questions": {
      "grounded": {
        "type": "boolean",
        "instructions": "Is the answer supported by the question?"
      },
      "tone": {
        "type": "choice",
        "instructions": "Classify the tone of the answer.",
        "criteria": {
          "formal": "Professional and impersonal.",
          "casual": "Friendly and conversational."
        }
      },
      "helpfulness": {
        "type": "score",
        "instructions": "Rate how helpful the answer is.",
        "criteria": ["useless", "partial", "complete"]
      }
    }
  }'

{
  "model": "typesafe-ai/jev",
  "answers": {
    "grounded": { "type": "boolean", "probability": 0.93 },
    "tone": {
      "type": "choice",
      "choice": "casual",
      "probabilities": { "formal": 0.18, "casual": 0.82 }
    },
    "helpfulness": {
      "type": "score",
      "score": 1.71,
      "probabilities": { "0": 0.04, "1": 0.21, "2": 0.75 }
    }
  },
  "usage": { "inputTokens": 412, "outputTokens": 36, "totalTokens": 448 }
}

Images (TypeSafe Jev)

Keys bound to TypeSafe Jev 1.13.0 or later can send still images in assets. Give each image a key, then use that key as a value in state and refer to it by its state name in your questions. Images must be PNG or JPEG, base64-encoded with no data: prefix, 280–1,568 pixels on each side, and already rotated upright (EXIF orientation is not applied). The whole request must stay under 4.5 MB. TypeSafe still labels image input experimental, so validate results before you depend on them.

curl https://sophy.in/v1/evaluate \
  -H "Authorization: Bearer mw_live_…" \
  -H "Content-Type: application/json" \
  -d '{
    "state": { "selfie": "selfie-1" },
    "assets": {
      "selfie-1": { "type": "image", "format": "jpg", "data_base64": "/9j/4AAQ…" }
    },
    "questions": {
      "groomed": {
        "type": "score",
        "instructions": "Rate the grooming of the person in `selfie`.",
        "criteria": ["poor", "acceptable", "good"]
      }
    }
  }'

Bad image input is rejected before any paid call with 400 assets_not_supported, 400 invalid_assets, 400 too_many_assets, 400 invalid_asset_key, 400 invalid_asset, 400 unsupported_asset_format, 400 invalid_asset_data, 400 asset_format_mismatch, or 400 unreferenced_asset. An image the model rejects, for example one outside the size range, returns 400 upstream_invalid_request with TypeSafe’s message.

Cloudflare Clef models

Keys can also bind Cloudflare Workers AI’s Clef decision models. Sophy calls them with the project’s own Cloudflare Workers AI credential, never through Vercel AI Gateway, so a Vercel AI Gateway outage doesn’t stop them. Send the same request as for any evaluation model: Sophy picks Cloudflare’s model selector from the key, sends boolean questions as Cloudflare’s noul type, and maps answers and usage back to the shape above.

• @cf/cloudflare/clef — Clef, context window 65,536 tokens.
• @cf/cloudflare/clef-flash — Clef Flash, context window 24,576 tokens.
• @cf/cloudflare/clef-omni — Clef Omni, context window 65,536 tokens.

Sophy sends these models text and images only. Cloudflare’s video input, and Clef Omni’s audio input, are not supported yet.

Clef keys add Cloudflare’s limits to the checks above, still before any paid call. Question names must be 1–100 letters, digits, _, ., or -. Choice questions take 2–255 options and score questions 2–10 levels. providerOptions is not accepted. Sophy also estimates the request size the way Cloudflare does — the JSON it sends upstream, without images, at four characters per token — and returns 400 state_too_large when the estimate exceeds the model’s context window: about 98,000 characters of JSON on Clef Flash, while on Clef and Clef Omni the 200,000-character state limit usually applies first.

Dense text such as logs can use about two characters per token, so a request under the estimate can still exceed the model’s real limit. In testing, Clef and Clef Flash rejected such requests instead of shortening the state, and Sophy returns 400 upstream_invalid_request with Cloudflare’s message, such as “Request requires 100819 tokens; the maximum is 65529”. Clef Omni’s limit is also its context window, and over-long input is expected to be rejected the same way, but that hasn’t been verified. Keep the state well inside the context window.

curl https://sophy.in/v1/evaluate \
  -H "Authorization: Bearer mw_live_…" \
  -H "Content-Type: application/json" \
  -d '{
    "state": "Support ticket #4821 from a retail customer: our checkout page has returned HTTP 500 errors for every customer since 09:10 UTC. We are losing roughly $2,000 per minute in sales. Please escalate immediately.",
    "questions": {
      "urgent": {
        "type": "boolean",
        "instructions": "Is this support request urgent?",
        "criteria": {
          "true": "Needs action within the hour",
          "false": "Can wait in the normal queue"
        }
      },
      "team": {
        "type": "choice",
        "instructions": "Which team should handle this request?",
        "criteria": {
          "billing": "Payments, invoices, and refunds",
          "technical": "Outages, errors, and configuration",
          "sales": "Plans and upgrades"
        }
      },
      "severity": {
        "type": "score",
        "instructions": "How severe is the customer impact?",
        "criteria": ["No impact", "Minor", "Major", "Critical"]
      }
    }
  }'

{
  "model": "@cf/cloudflare/clef",
  "answers": {
    "urgent": { "type": "boolean", "probability": 0.8998 },
    "team": {
      "type": "choice",
      "choice": "technical",
      "probabilities": { "billing": 0.0605, "technical": 0.931, "sales": 0.0085 },
      "confidence": 0.8058
    },
    "severity": {
      "type": "score",
      "score": 2.9603,
      "probabilities": { "0": 0.0035, "1": 0.0035, "2": 0.0222, "3": 0.9708 },
      "confidence": 0.9239
    }
  },
  "usage": { "inputTokens": 381, "outputTokens": 0, "totalTokens": 381 }
}

Clef keys can send up to four still images in assets, in the same keyed shape as Jev: PNG, JPEG, or WebP, base64-encoded with no data: prefix, at most 4 MiB each and 8 MiB in total once decoded. Cloudflare also limits each image to 16 megapixels, which Sophy doesn’t check. The whole request must still stay under 4.5 MB, which binds first. Clef sees every image before the state, in the key order of assets; as in JavaScript, keys that are plain whole numbers, such as 2, come first in numeric order. An image key doesn’t need to appear in the state or a question, and Sophy doesn’t swap a key for its image, so refer to images by position, such as “the first image”.

curl https://sophy.in/v1/evaluate \
  -H "Authorization: Bearer mw_live_…" \
  -H "Content-Type: application/json" \
  -d '{
    "state": "Ticket #4821: checkout has returned HTTP 500 for every customer since 09:10 UTC. The first image is a screenshot from the customer.",
    "assets": {
      "screenshot": { "type": "image", "format": "png", "data_base64": "iVBORw0KGgo…" }
    },
    "questions": {
      "error_page": {
        "type": "boolean",
        "instructions": "Does the first image show an error page?"
      }
    }
  }'

Bad Clef input is rejected before any paid call with 400 invalid_question_id, 400 invalid_criteria, 400 unsupported_provider_options, 400 unsupported_media_field, 400 invalid_assets, 400 too_many_assets, 400 invalid_asset_key, 400 invalid_asset, 400 unsupported_asset_format, 400 invalid_asset_data, 400 asset_format_mismatch, 400 asset_too_large, 400 assets_too_large, or 400 state_too_large. A project without a healthy Cloudflare credential returns 503 project_gateway_unavailable, even when its Vercel AI Gateway credential is healthy. A Cloudflare account on the Workers Free plan that has used its daily free allocation returns 429 rate_limit_exceeded until 00:00 UTC, and Sophy doesn’t retry it.

One evaluate call consumes one RPM slot and is checked against the key’s monthly cost cap before the paid model call. When content logging is enabled on the key, Sophy stores the evaluated state, the questions, and the returned answers. For images it logs only each one’s format and size, never the image itself. Clef calls are priced from Cloudflare’s reported usage; see Limits and quota.

Extraction models (preview)#

Extraction models read text and images and return typed values — text, numbers, dates, times, durations, lists, estimates, and forecasts — instead of free text. Like evaluation models, they are not exposed through the OpenAI-compatible client methods, so call this native Sophy route with an HTTP client. Bind the key to Clairic, send the state to read together with named, typed fields, and Sophy returns one answer per field plus token usage.

Calls use the project’s own Cloudflare Workers AI credential, never Vercel AI Gateway. The call is a single buffered round trip; there is no streaming.

Field types

Each field spec has a type and may add any of these settings; a setting not listed here returns 400 invalid_field.

Requests are validated before any paid call. A key whose model is not an extraction model, a Clef key included, returns 400 model_not_extraction. A bad body returns 400 invalid_state, 400 missing_state, 400 state_too_large, 400 missing_fields, 400 invalid_fields, 400 fields_empty, 400 too_many_fields, 400 invalid_field_name, 400 unsupported_field_type, 400 invalid_field, 400 invalid_reference_date, or 400 invalid_options, and param names the problem, such as fields.total.decimals. Field specs may use only the settings above. Sophy checks each setting’s type and range, requires items for lists and min and max for estimates and forecasts, and leaves the rest to Clairic, whose rejections return 400 upstream_invalid_request with its message.

Sophy also estimates the request size the way Cloudflare does — the JSON it sends upstream, meaning the state and field specs without images, at four characters per token — and returns 400 state_too_large when the estimate exceeds Clairic’s 65,536-token context window, about 262,000 characters of JSON. JSON writes each newline, tab, quote, and backslash as two characters (other control characters as six), so a newline-dense series, tab- or quote-heavy CSV/TSV, or serialized JSON can reach that limit below 200,000 characters.

import httpx

resp = httpx.post(
    "https://sophy.in/v1/extract",
    headers={"Authorization": "Bearer mw_live_…"},
    json={
        "state": (
            "Invoice INV-7731 from Blue Harbor Logistics. Issued 14 Oct. Net 15. "
            "Freight: 3 pallets at $410.00 each; fuel surcharge $57.50. "
            "Contacts: Meera Iyer, Tom Okafor."
        ),
        "reference_date": "2026-10-20",
        "fields": {
            "invoice_number": {"type": "text", "instructions": "Invoice number"},
            "total": {
                "type": "number",
                "unit": "USD",
                "decimals": 2,
                "instructions": "Amount due including the surcharge",
            },
            "due_date": {"type": "date", "instructions": "Issue date plus the net payment days"},
            "contacts": {"type": "list", "items": "text", "instructions": "Contact person names"},
            "po_number": {"type": "text", "instructions": "Purchase order number"},
            "days_to_pay": {
                "type": "estimate",
                "min": 0,
                "max": 90,
                "step": 1,
                "instructions": "Days after issue until this invoice is paid",
            },
        },
    },
    timeout=120,
)
print(resp.json()["answers"])

curl https://sophy.in/v1/extract \
  -H "Authorization: Bearer mw_live_…" \
  -H "Content-Type: application/json" \
  -d '{
    "state": "Invoice INV-7731 from Blue Harbor Logistics. Issued 14 Oct. Net 15. Freight: 3 pallets at $410.00 each; fuel surcharge $57.50. Contacts: Meera Iyer, Tom Okafor.",
    "reference_date": "2026-10-20",
    "fields": {
      "invoice_number": { "type": "text", "instructions": "Invoice number" },
      "total": { "type": "number", "unit": "USD", "decimals": 2, "instructions": "Amount due including the surcharge" },
      "due_date": { "type": "date", "instructions": "Issue date plus the net payment days" },
      "contacts": { "type": "list", "items": "text", "instructions": "Contact person names" },
      "po_number": { "type": "text", "instructions": "Purchase order number" },
      "days_to_pay": { "type": "estimate", "min": 0, "max": 90, "step": 1, "instructions": "Days after issue until this invoice is paid" }
    }
  }'

{
  "model": "@cf/cloudflare/clairic",
  "answers": {
    "invoice_number": {
      "type": "text",
      "p_null": 0.0001,
      "span": [8, 16],
      "guess": "INV-7731",
      "confidence": 1,
      "value": "INV-7731"
    },
    "total": {
      "type": "number",
      "p_null": 0.0003,
      "guess": 1287.5,
      "confidence": 0.8683,
      "value": 1287.5
    },
    "due_date": {
      "type": "date",
      "p_null": 0.00001,
      "guess": "2026-10-29",
      "confidence": 0.9999,
      "value": "2026-10-29"
    },
    "contacts": {
      "type": "list",
      "value": ["Meera Iyer", "Tom Okafor"],
      "spans": [[137, 147], [149, 159]],
      "confidence": 0.9708
    },
    "po_number": {
      "type": "text",
      "p_null": 0.999999,
      "span": [8, 16],
      "guess": "INV-7731",
      "confidence": 0.9993,
      "value": null
    },
    "days_to_pay": {
      "type": "estimate",
      "value": 18,
      "mean": 17.58,
      "median": 16,
      "p10": 13,
      "p90": 25,
      "p_below": 0.000009,
      "p_above": 0.000009
    }
  },
  "truncated": false,
  "usage": { "inputTokens": 715, "outputTokens": 0, "totalTokens": 715 }
}

Reading answers

Images

Send up to four still images in assets, exactly as for Clef keys on evaluate: PNG, JPEG, or WebP, base64-encoded with no data: prefix, at most 4 MiB each and 8 MiB in total once decoded, and a whole request under 4.5 MB. Clairic sees them before the state, in key order, and no reference is needed. With images, state may be empty or omitted. A top-level images, audio, or videos field returns 400 unsupported_media_field, and bad images return the same asset codes as Clef keys.

curl https://sophy.in/v1/extract \
  -H "Authorization: Bearer mw_live_…" \
  -H "Content-Type: application/json" \
  -d '{
    "assets": {
      "receipt": { "type": "image", "format": "jpg", "data_base64": "/9j/4AAQ…" }
    },
    "fields": {
      "merchant": { "type": "text", "instructions": "Store name" },
      "paid": { "type": "number", "unit": "USD", "decimals": 2, "instructions": "Total paid" },
      "paid_on": { "type": "date", "instructions": "Purchase date" }
    }
  }'

One extract call consumes one RPM slot and is checked against the key’s monthly cost cap before the paid model call; its cost is estimated as described in Limits and quota. When content logging is enabled on the key, Sophy stores the state, the field specs, the reference date and options, and the returned answers. For images it logs only each one’s format and size, and it never logs state_text. A project without a healthy Cloudflare credential returns 503 project_gateway_unavailable, even when its Vercel AI Gateway credential is healthy. A Cloudflare account on the Workers Free plan that has used its daily free allocation returns 429 rate_limit_exceeded until 00:00 UTC, and Sophy doesn’t retry it.

Image generation#

Bind the key to a supported image model. Generated images are returned inline as base64 and are not stored by Sophy.

from openai import OpenAI
import base64

client = OpenAI(base_url="https://sophy.in/v1", api_key="mw_live_…")

resp = client.images.generate(
    model="sophy",
    prompt="A red panda coding at a desk, watercolor",
    n=1,
    size="1024x1024",
)
with open("out.png", "wb") as f:
    f.write(base64.b64decode(resp.data[0].b64_json))

curl https://sophy.in/v1/images/generations \
  -H "Authorization: Bearer mw_live_…" \
  -H "Content-Type: application/json" \
  -d '{"prompt":"A red panda coding at a desk, watercolor","size":"1024x1024"}'

{
  "created": 1735689600,
  "data": [{ "b64_json": "iVBORw0KGgoAAAANSUhEUgAA…" }]
}

Files#

Upload a file as multipart/form-data using the field name file. Success returns 201. Use the returned URL in a Chat file/image part or a Responses input_file/input_image part.

Accepted content types are PDF, PNG, JPEG, WebP, GIF, plain text, CSV, and JSON. The maximum size is 4 MiB (4,194,304 bytes). Uploads become eligible for batched cleanup after 24 hours, so they are request staging—not durable file storage or a precise expiry service.

curl https://sophy.in/v1/files \
  -H "Authorization: Bearer mw_live_…" \
  -F "file=@./invoice.pdf"

{
  "id": "https://…/uploads/<project>/<key>/invoice-….pdf",
  "url": "https://…/uploads/<project>/<key>/invoice-….pdf",
  "pathname": "uploads/<project>/<key>/invoice-….pdf",
  "filename": "invoice.pdf",
  "bytes": 48213,
  "contentType": "application/pdf"
}

Upload URLs are public but unguessable so model providers can fetch them; treat each URL as a bearer URL, not a secret store. When a Sophy URL is submitted back through the API, Sophy checks its issuing key, and cross-key reuse returns 403 file_access_denied.

Models#

Returns exactly the primary model currently bound to the authenticated key in OpenAI list shape. This route does not return the full console catalog or an optional transcript processor, and it does not select a model for a later request.

{
  "object": "list",
  "data": [
    {
      "id": "anthropic/claude-sonnet-4.6",
      "object": "model",
      "created": 0,
      "owned_by": "sophy"
    }
  ]
}

Key-owned policy#

The key is Sophy’s unit of configuration. Operators can change its policy without changing or redeploying the client. The next request reads the latest values.

Client values for model and generation parameters are silently ignored. Chat and Responses prompt behavior is controlled separately by Agent mode. Audio transcription rejects a client prompt; only the key-owned system prompt can request post-processing.

When a language or transcript-processing call has an effective system prompt, Sophy prepends a fixed platform security preamble before the key and any applicable Agent-mode client instructions. On Anthropic models, Chat and Responses requests mark that system prompt (with any tools) as an ephemeral prompt-cache breakpoint, so requests on the same key within the cache window re-read it instead of paying full input price; knowledgebase context is sent after the breakpoint so it doesn’t break the shared prefix. Multi-turn requests on keys without a knowledgebase also mark the latest message, caching the conversation so far. Prompts shorter than the model’s cacheable minimum are not cached. Cache reads appear in usage as prompt_tokens_details.cached_tokens (Chat) and input_tokens_details.cached_tokens (Responses).

Agent mode#

Standard keys are governed mode: Sophy drops client system and developer messages plus Responses instructions, then uses the key-owned prompt.

For a trusted server-side agent whose instructions must vary per request, an operator can enable Agent mode on the key. Sophy keeps the key prompt ahead of client instructions (after Sophy’s fixed security preamble), then appends:

• Chat: consecutive leading system/developer messages, up to the first non-system turn.
• Responses: instructions, followed by leading system/developer input items.

Later system-role items in the conversation are still dropped. Model, generation parameters, schema, knowledge, budgets, and rate limits remain key-owned.

Knowledgebases#

Project Admins can manage every project knowledgebase. Editors can create and manage knowledgebases they own. Both roles can build them from PDF, DOCX, Markdown, plain text, CSV, and JSON documents up to 4 MiB each. Sophy extracts, chunks, and embeds sources in the background.

Knowledgebases are shareable inside their project. An Admin can attach one to any project key, while an Editor can attach one only to a key they own. The same collection can ground multiple permitted keys.

On Chat or Responses requests with usable user text, Sophy embeds the latest user text (capped at its first 8,000 characters), retrieves the six closest chunks, and adds them to the effective prompt. Image- or file-only turns have no retrieval query. A transient retrieval timeout can continue without grounding. Missing, invalid, or billing-blocked project Gateway credentials fail closed before the model call; they never fall back to Sophy’s deployment credentials or another project.

Knowledgebase ingestion and query-embedding spend is tracked separately from client proxy traffic in the console.

Operator console#

One passwordless identity can belong to multiple projects with an independent Admin or Editor role in each. The project in the URL is the tenant boundary, and every read, mutation, key, knowledgebase, log, evaluation, and audit event stays inside it.

Operator sign-in with email codes#

The console and CLI use the same email identity. Sophy sends a six-digit one-time password (OTP) to your inbox. Each code expires after 10 minutes and permits at most five attempts. A successful attempt consumes the code.

Sign-in links no longer authenticate an account. Invitation links identify the invitation, but acceptance requires sign-in with the matching email account.

{ "email": "operator@example.com" }

{ "ok": true, "challengeId": "<challenge-id>" }

This response does not reveal whether an account exists. Email delivery and verification are subject to rate limits. When Sophy issues a new code, it invalidates the previous code for that email. The optional next field preserves a local invitation path through sign-in.

Code requests require 60 seconds between deliveries. Each email permits five codes per 15 minutes and 10 per hour. Each IP permits 10 requests per 15 minutes for each sign-in endpoint. Sign-in JSON bodies permit at most 4 KiB.

{ "challengeId": "<challenge-id>", "code": "012345", "client": "cli" }

For web, verification sets the browser session cookie and returns the next local path. For cli, it returns token, expiresAt, and user. A CLI session expires 30 days after its last use: each authenticated management request extends it to 30 days from that request, so a CLI in regular use stays signed in. Each request checks the current account, project membership, and project status.

Send the CLI token in the Authorization: Bearer header to revoke that session. The CLI removes its local session after a successful logout.

Sign-in endpoints return { "error": "code" } on errors. Malformed verification requests return 400 invalid_request. Invalid emails return 400 invalid_email. Invalid, used, or expired codes return 401 invalid_or_expired_code. Request limits return 429 too_many_attempts. An invalid CLI logout token returns 401 unauthorized.

Sophy CLI#

Manage Sophy from a terminal or a coding agent that can run terminal commands. The CLI uses the same project roles and resource ownership as the console. Admins manage project resources. Editors manage the keys and knowledgebases they own.

Install the public @sophyai/sophy-cli npm package. A source checkout is not required.

Use Node.js 22 or newer. Install the CLI from npm:

npm install -g @sophyai/sophy-cli
sophy --help

To run a command without a global installation, use npx:

npx --package @sophyai/sophy-cli sophy --help
npx --package @sophyai/sophy-cli sophy login --email operator@example.com

The following examples use the installed sophy command. With npx, prefix each command with npx --package @sophyai/sophy-cli.

Sign in with your email. Then choose a project from your project list:

sophy login --email operator@example.com
sophy whoami
sophy projects list
sophy projects use <project-id>
sophy keys list

Enter the six-digit email code at the hidden login prompt. The default server is https://sophy.in. The installed command and npx use the same local session. The --project flag selects a project for one command. Run sophy logout to revoke the session.

For development from a source checkout, run npm install -g ./cli from the repository root.

Update key instructions#

The systemPrompt field stores the instructions for a key. An update applies to subsequent requests with that key. Read the current settings before you change them:

sophy keys get <key-id> --project <project-id> --json

For a short instruction, use the --system-prompt flag:

sophy keys update <key-id> --project <project-id> \
  --system-prompt "Answer support questions using the attached knowledgebase."

For longer instructions, save a JSON file named key-instructions.json. Use \n for line breaks inside the JSON string:

{
  "systemPrompt": "You are the support assistant.\nUse the attached knowledgebase.\nIf the answer is unavailable, say so."
}

Apply the file. Then read the key again to confirm the saved instructions:

sophy keys update <key-id> --project <project-id> \
  --data @key-instructions.json --json
sophy keys get <key-id> --project <project-id> --json

Use Sophy with coding agents#

A coding agent can use the CLI through its terminal. Commands run with the account signed in to that environment. A remote agent needs its own login in its environment. Your browser session or another computer's CLI session does not sign in that agent.

Use --json for compact JSON output. Use --data @file.json for nested input or long prompts. Project roles and resource ownership still apply to every command. Admin-only changes remain restricted to project admins.

Copy these instructions into a task for your coding agent:

Use the Sophy CLI for this task.

1. Run sophy --help, then sophy keys update --help to read the supported fields.
2. Run sophy whoami --json. If sign-in is required, ask me to complete
   sophy login in this environment. Do not copy a session from another machine.
3. Run sophy projects list --json. Use the project ID and key ID I identify.
   Include --project <project-id> on every project command.
4. Before editing, run:
   sophy keys get <key-id> --project <project-id> --json
5. Put only the requested changes in changes.json. Use systemPrompt for instructions.
   Preserve omitted settings. If changing params, include all parameters to retain.
6. Apply the changes:
   sophy keys update <key-id> --project <project-id> --data @changes.json --json
7. Read the key again with keys get. Confirm the requested changes and preserved settings.
   Report the saved result. Treat any nonzero exit code as incomplete work.
8. Use --yes only for an action I explicitly authorized. Do not include secrets in reports.

The CLI returns exit code 1 for errors. Exit code 2 means a bulk action skipped items or a key update requires evaluation-stop confirmation. The JSON result identifies the incomplete work.

CLI commands and management API#

Discover commands, command groups, and input fields through the built-in help:

sophy --help
sophy keys --help
sophy keys update --help

The CLI accepts credential secrets only from a file or standard input, never as command-line arguments. Put {"accountId": "…", "apiToken": "…"} in a file and run sophy cloudflare connect --data @cloudflare.json, or pipe it with --data -. You can pass the account ID as --account-id instead and keep only the token in the JSON. The Vercel AI Gateway key uses sophy gateway connect the same way, with {"apiKey": "…"}. The cloudflare commands require @sophyai/sophy-cli 0.3.0 or later.

Sophy checks the Cloudflare account ID and token with Cloudflare before it saves them, and a failed check leaves the current credential in place. A failed connection returns one of these errors:

• 400 cloudflare_invalid_account_id: Enter the 32-character Cloudflare account ID.
• 400 cloudflare_invalid: The Cloudflare account ID or API token was rejected. Create the token from Cloudflare's "Workers AI" template.
• 503 cloudflare_unreachable: Cloudflare could not be reached to validate the credential. Try again.
• 409 cloudflare_in_use: This Cloudflare API token is already in use by another project.

Creating a key for a Cloudflare model, or changing a key to one, returns 409 cloudflare_not_ready (“Connect a healthy project Cloudflare credential first.”) until that credential is ready. An unsupported @cf/ model returns 400 invalid_request with the message “Unsupported Cloudflare Workers AI model”.

Key creation and rotation show the new secret once. Sophy cannot return an existing secret. Rotation and revocation require confirmation.

To read one key's recent traffic, supply its ID. The command returns the key's 100 most recent logs from every log source, newest first. Add --source or a smaller --limit to narrow the list:

sophy logs list --key-id <key-id> --project <project-id> --json
sophy logs get <log-id> --project <project-id> --json

A log list contains metadata. logs get returns one log with any retained content. Admins can read logs for any key in the project, and editors for the keys they own. Any other key returns 404 not_found. The management API takes the key as keyId in the logs.list input.

The management API requires a CLI session token in the Authorization: Bearer header. Browser cookies and application API keys cannot authenticate this route. CLI tokens cannot authenticate the AI routes under /v1.

{ "operation": "keys.list", "projectId": "<project-id>", "input": {} }

{ "ok": true, "data": [], "sessionExpiresAt": "2026-11-21T09:30:00.000Z" }

{ "error": { "code": "forbidden", "message": "Your current project role or resource ownership does not allow this action." } }

sessionExpiresAt is the session's current expiry after this request's renewal. The server records renewals at most once a day, so it can trail the request time plus 30 days by up to one day. Logout and revoked or expired sessions are never renewed.

Management responses use Cache-Control: no-store. Access decisions happen on the server. A local project choice or a stale CLI session never grants additional project rights.

JSON requests permit at most 1 MiB. Knowledgebase uploads use multipart fields operation=knowledgebases.upload, projectId, kbId, and file. Files permit at most 4 MiB, with 64 KiB for the multipart envelope. Each account permits 120 management requests per minute.

Invalid fields return 400 invalid_request. Missing project selection returns 400 project_required. Unknown operations return 400 unknown_operation. Invalid sessions return 401 unauthorized, denied access returns 403 forbidden, and unavailable resources return 404 not_found.

Oversized requests return 413 payload_too_large. Unsupported content types return 415 unsupported_media_type. Request limits return 429 rate_limited with Retry-After: 60. Unexpected failures return 500 internal_error without database details.

Errors and retries#

Before a response stream starts, Sophy-generated failures use the OpenAI error envelope below with Cache-Control: no-store. Field validation errors may use code: null and identify the field in param.

{
  "error": {
    "message": "Invalid API key.",
    "type": "authentication_error",
    "param": null,
    "code": "invalid_api_key"
  }
}

Upstream client-input rejections (400/413/422) become 400 upstream_invalid_request with a safe, capped message. Upstream rate limits preserve 429 and forward Retry-After when present. A valid Sophy key receives 503 project_gateway_unavailable when the project credential its model needs — Vercel AI Gateway, or Cloudflare Workers AI for Cloudflare models — is missing, disconnected, invalid, billing-blocked, not yet verified, or undecryptable. The message is provider-neutral and asks a project admin to connect or check the project’s AI provider credentials. Sophy makes no fallback upstream call: a Cloudflare model never falls back to Vercel AI Gateway, or the reverse. When Cloudflare rejects a stored token during a request, Sophy marks that credential invalid and returns the same 503 until an Admin reconnects it. Transient network and server failures are hidden behind 502 upstream_error.

Rate limits and monthly quota#

Chat, Responses, audio transcription, embeddings, image generation, model evaluation, and extraction enforce the key’s optional requests-per-minute limit and monthly cost cap before the paid model call. /files and /models do not consume those request counters.

• RPM uses a fixed 60-second window. Exceeding it returns 429 rate_limit_exceeded with Retry-After in seconds.
• Monthly quota uses the current UTC calendar month and counts client-attributable spend, including transcript processing and the estimated cost of Cloudflare Workers AI calls. Reaching it returns 402 quota_exceeded.
• Champion-versus-challenger evaluation and knowledgebase spend is recorded under separate sources for visibility and does not consume a key’s proxy-traffic cap.

Console cost values are estimates for operational decisions, not a billing-grade ledger. Vercel AI Gateway calls record the cost the gateway reports. Cloudflare Workers AI calls are estimated from the neurons Cloudflare reports for each call, at its published $0.011 per 1,000 neurons, or from the model’s listed input-token price when no neuron count comes back; output tokens are not billed. The estimate ignores Cloudflare’s free allocation of 10,000 neurons per account per day, so Cloudflare can bill less than Sophy records. A response that Cloudflare completed and billed but Sophy couldn’t use — for example, one missing a requested field, which returns 502 upstream_error — is still metered from the neurons Cloudflare reported for it. Requests that Cloudflare rejected record no cost.