Direct answer
Route coding-agent requests by an explicit policy that combines task requirements, approved models, protocol capability, data boundary, budget, and current route health. Keep model aliases and fallback rules versioned, isolate credentials, preserve streaming and tool semantics, and verify the effective route with a minimal end-to-end task before production use.
Key takeaways
- Route on declared workflow capabilities, not model-name similarity.
- Version aliases, policy, fallback order, protocol transforms, and credential ownership.
- Bound failover and prevent silent switches across data or capability boundaries.
- Verify non-streaming, streaming, tool use, errors, and usage attribution separately.
Key facts
| Fact | Verified value | Evidence |
|---|---|---|
| Codex provider configuration | Codex documents configurable model providers and wire-protocol settings. | View source |
| Claude gateway connection | Claude Code documents environment and gateway configuration for enterprise routing. | View source |
| Protocol requirements | Claude Code separately documents protocol behaviors an LLM gateway must preserve. | View source |
| Tool boundary | MCP specifies tool interactions separately from the model transport and should be verified as its own path. | View source |
What this guide helps you solve
Help platform teams design gateway routing without confusing text completion with full coding-agent compatibility.
Verification scope and limits
- Evidence basis
- Official documentationSpecification
- Verification scope
- The design covers provider configuration, gateway protocol behavior, policy routing, failover, credentials, and workflow verification.
- Limits and invalidation conditions
- - No gateway or provider is endorsed.
- - Compatibility depends on deployed transformations and client versions.
- - No latency, availability, or price benchmark is asserted.
Define a route contract before writing policy
For each workflow, state required request and response protocol, streaming events, tool semantics, context needs, authentication owner, data region, allowed models, budget class, and failure behavior. A route is eligible only when it satisfies the complete contract.
Keep user-facing aliases separate from upstream model identifiers. An alias should resolve through a versioned policy and produce an observable effective route without exposing credentials.
Verify compatibility in layers
A successful text response proves only the narrow request that was sent. Test authentication, model discovery where applicable, non-streaming output, streaming termination, error mapping, cancellation, tool calls, usage reporting, and long-running behavior independently.
Document every transformation the gateway applies. Dropping unknown fields, buffering streams, rewriting model names, or normalizing errors can change client behavior even when HTTP status is successful.
Keep client and upstream credentials separate
The client should authenticate to the gateway with a scoped credential; the gateway should hold provider credentials in a secret store and apply tenant, project, model, and rate policy server-side. Never return upstream tokens or log authorization headers.
- Short-lived or rotatable client credentials
- Provider secrets unavailable to the coding session
- Per-route authorization and quotas
- Sanitized request IDs for support
- Independent revocation for client and upstream access
Make failover explicit and bounded
Fallback is allowed only to a route that satisfies the same declared capability, policy, and data boundary. Limit attempts and total time, avoid replaying state-changing tool work, and expose when a fallback occurred. Some failures should stop instead of changing model or provider silently.
Version policy and reverify every material change
Record the client version, gateway release, route-policy version, alias map, protocol transform, and verification date. Re-run the capability suite after changing any of them. Roll out gradually and keep a fast disable path for a route that corrupts streams, errors, tools, or accounting.
Official sources and verification scope
This guide is based on public official documentation. Commands and configuration may change between client versions, so verify the linked sources before use.
Read the verification methodology