Quickstart: coding agents and IDEs

Coding tools that talk to an OpenAI-compatible endpoint are raced the same way as any client: one base-URL change, nothing to install in the editor, no extension. The sidecar runs on your laptop (or one per developer host), the tool points at it, and about 30% of the completions the tool requests (α = 0.3, the default) are raced across the legs you configured; the rest go to one leg.

1. Run the sidecar next to the editor

$ pip install --index-url https://airmet.ai/simple/ airmet   # inside a virtual environment; or npm, the bare binary, Docker
$ airmet up --providers https://api.provider/v1 \
    --model-name <the model id the tool will send> --api-key-env PROVIDER_KEY
airmet up · Go data plane · 2 legs on 1 provider · twinleg α=0.3 · /v1 on 127.0.0.1:8080 · run dir airmet-results/up-…

Two legs of the same model — one provider, which the sidecar races against itself when you give it one URL as above, or two providers with a key each in a hedger.yaml — is the coding-agent shape: long, cache-heavy conversations where the first token is what you feel. With no policy named the sidecar twinlegs at α = 0.3: about 30% of requests race both legs, and the slower leg is not cancelled for being slower: it runs to its own finish unless the runaway guard stops it for looping or running far past the winner, so both legs' prompt caches stay warm through the conversation — a leg cancelled mid-turn would re-read the prompt cold on its next race. --hedge-n 1 serves one leg alone. The raced share bills both legs (the cost note is on the OpenAI-compatible quickstart); the hedge object on every response says what fired: how many legs it launched and which one won.

Per-leg keys, models and provider pinning need a hedger.yaml — see the OpenAI-compatible quickstart.

2. Point the tool at http://127.0.0.1:8080/v1

The tool's own API key goes no further than the sidecar: the sidecar relays the client's body verbatim and sends each leg only the bearer you configured for it (--api-key-env, or api_key_env per leg in hedger.yaml). Prompts and keys never leave your machine except to the providers you named.

3. Check what it did

$ airmet status
airmet status · http://127.0.0.1:8080
ready:       yes (2/2 replicas healthy)
…
your wins (vs unhedged baseline, from local records):
  win rate:    <share>% of <n> hedged requests (<k> backup-first-token wins)
  …
  p99 TTFT:    -<ms> ms  (est. unhedged <ms> ms → hedged <ms> ms)
…

The output above is the shape airmet status prints, with your own values in place of the <…>: they come from your own records under airmet-results/, and the estimate caveats travel with them. airmet traffic lists everything the sidecar sends off the machine (per-request telemetry records, on by default, opt out with --no-telemetry).

What does not change

• Streaming and tool calls pass through untouched; the tool sees one response, from whichever leg was first. The client's own stream_options is not forwarded to the legs: the sidecar honours it on the response it sends back.
• Prompt caching keys on the request prefix per provider, so a self-race on one provider stays warm on both legs; a cross-provider race warms both caches.
• A conversation never has two requests on one leg at once: a turn that arrives while the leg kept from the previous turn is still finishing runs on one leg. Tools that send no task id are told apart by their prompt cache key, else by the conversation's opening messages.
• Name a policy that cancels the loser (--policy deferred-p95) and a leg on a provider that bills a cancelled request in full is never raced (on_no_refund=decline); airmet status shows each leg's verdict.