Provider pinning

One key, two genuinely different providers. Major model marketplaces let a request pin which underlying provider serves it. Airmet exposes that per leg: one marketplace key, each leg pinned to a different provider, so you get cross-provider independence without a second account. This is the first of the three setups on the docs index, and the shape our pilot runs.

hedger.yaml

# hedger.yaml — one marketplace key, two pinned providers
replicas:
  - id: leg0
    engine: openai
    base_url: https://marketplace.example/api/v1
    model: llama-3.3-70b
    api_key_env: MARKETPLACE_API_KEY
    provider_routing:              # pin leg 0 to one underlying provider
      only: [provider-slug-a]
      order: [provider-slug-a]
      allow_fallbacks: false
  - id: leg1
    engine: openai
    base_url: https://marketplace.example/api/v1
    model: llama-3.3-70b
    api_key_env: MARKETPLACE_API_KEY
    provider_routing:              # pin leg 1 to a different one
      only: [provider-slug-b]
      order: [provider-slug-b]
      allow_fallbacks: false
hedge:
  policy: twinleg                  # the product: keeps the slower leg running, so both stay warm
  alpha: 0.3                       # sync rate — the fraction of turns that get the second leg
  mode: throughput                 # or ttft

policy lives inside the hedge: block. A bare top-level policy: key is not part of the schema: the Python runtime rejects it and the Go runtime ignores it, leaving the pool on the default a file that names no policy gets — twinleg at α = 0.3, as plain airmet up --providers runs. The airmet up banner names the policy each run races.

The per-leg provider_routing object is forwarded verbatim as the request body's provider field on that leg — Airmet does not interpret or rewrite it, so any object your marketplace's provider-routing accepts works unchanged. The shape above (only + order + allow_fallbacks: false) is the fully-pinned form our benchmarks use; consult your marketplace's provider directory for the exact slugs. provider_routing applies to engine: openai legs only — the token-native engines (vLLM/SGLang) have no provider concept.

Tuning when the second leg fires

A deferred policy launches the second leg only once the first is already slow, and hedge.deferred is where you say how slow. The dial is quantile (the boot log calls it alpha; it is a quantile, not twinleg's α): the backup fires at the primary's own alpha-quantile of first-token latency, so alpha sets roughly what fraction of requests race at all — 0.95 means about the slowest 5%, 0.90 about 10%.

hedge:
  policy: deferred-adaptive
  deferred:
    quantile: 0.90                 # alpha — fire into the slowest ~10%
    tail_quantile: 0.99            # beta — the primary tail the backup must beat
    window_s: 2400                 # widen both of these on a low-traffic pool,
    min_samples: 30                #   or the estimator never warms
    floor_ms: 20                   # never fire sooner than this
    max_refusal_rate: 0.10         # stop racing into a leg that is 429-ing
    min_quota_headroom: 0.10       # keep this share of a leg's rpm_limit free
    min_win_prob: 0.0              # 0 = off. Above 0, decline a second leg
                                   #   that cannot overtake — see below

Every key is optional. The defaults are the values above except three, which the example widens for a low-traffic pool: quantile 0.95, window_s 60 and min_samples 100. deferred-adaptive reads all of them; the simpler deferred-p95 reads quantile, window_s, min_samples and floor_ms only, and has none of the refusal or quota guards.

Values are checked at startup, not clamped: quantile: 1.5, or an alpha at or above tail_quantile, fails the boot with the key named rather than quietly serving at some other operating point. So does a misspelt key. The dials in force are printed once at boot (deferred dials: alpha=… beta=… …), so a run's operating point is in its own log.

min_win_prob: declining a second leg that cannot win

quantile decides when a backup fires; it says nothing about whether the backup can still get there. A backup launched at t_fire starts from cold and has to redo the whole prompt, so on a prompt-heavy workload it often cannot produce a first token inside the time the primary has left — the fire costs a prefill and returns nothing. On the two workloads we replayed (2026-09-07), many fires had less runway than the backup leg's own best-ever first token: they could not have won at any speed that leg had ever shown.

Set min_win_prob above 0 and a second leg is declined when its estimated chance of actually overtaking is below it, recorded as outpaced and counted in declined_outpaced. Default 0 keeps the old behaviour exactly.

Start at 0.2–0.3, and treat it as a floor for hopeless fires rather than a profitability threshold. The estimate is deliberately conservative — a primary that loses a race is cancelled and never reports a time, so the distribution it is read from under-states how often the primary really is slow — which means the number reads low. It is dependable at the bottom of its range, where it is identifying fires that had no chance; pushed to 0.7 it declined second legs that would have won on every workload we replayed it against. It also needs samples: below min_samples on either leg it abstains and the ordinary rule decides.

Raising it will not fix a workload where racing simply does not pay. If the backup's service time approaches the primary's total — a long prompt against a provider with a tight body — there may be no quantile that pays, and a policy that declines is telling you that, not failing.

The right alpha is workload-specific — a heavy-tailed provider pays back a higher fire rate, a tight one does not — so sweep it against your own deadline metric rather than assuming the default.

Why pin at all?

An unpinned marketplace key may route both legs to the same underlying provider — often the same one having the slow hour. Pinning restores the thing the race depends on: two draws from independent queues. In our observatory runs of 2026-08-10, two cross-provider pairs containing one provider's free-tier front completed 100% of turns (231/231 and 320/320) while that front refused about 40% of them (rate limits under our own pacing); independence also lowers the chance that both legs slow down together at long context.

Notes

• Pinning constrains the marketplace's freedom to route around a provider that is down. That is the point — the race is your availability mechanism, not the marketplace's failover.
• Leave provider_routing off a leg to let the marketplace route it freely; you can pin one leg and float the other.
• allow_fallbacks: false keeps a pinned leg honest — without it the marketplace may silently substitute another provider and both legs can land on the same queue.
• Model names may differ per provider on some marketplaces; the model field is set per leg if yours requires it.