A cascade of agents: the terminal picks whichever one is alive

TL;DR

A bash launcher called agent that picks the best working AI coding CLI before it starts. It degrades along two axes: first the model (Opus → Sonnet, staying inside Anthropic), then the transport, and only then the vendor. Every probe makes a real API call, because model lists lie. A watchdog checks the lower rungs every two hours and alerts on Telegram when one dies.

$ agent --status
  OK   claude / claude-opus-5         api.anthropic.com reachable, claude-opus-5 answered
  OK   claude / claude-sonnet-5       api.anthropic.com reachable, claude-sonnet-5 answered
  OK   claude-proxy / claude-opus-5   claude-opus-5 answered
  OK   claude-proxy / claude-sonnet-5 claude-sonnet-5 answered
  OK   agy                            antigravity answered on Gemini 3.6 Flash (Low)
  OK   gemini                         localhost:8317 answered for gemini-3.1-pro-low

The Problem

When Claude is unavailable I want my terminal to keep working. The naive version of this is "if Claude is down, run Gemini" — and that is what I thought I had. But the interesting failures are not outages. The common one is that Opus caps out while Sonnet answers fine. Jumping to another vendor there is absurd: the model next door is healthy.

So a real fallback has to degrade by model before it degrades by provider. And it has to know which rung is actually alive — which turns out to be the hard part, and the part I got wrong twice.

The five-month lie

Before writing any of this I checked what I already had. The proxy's auth directory told the whole story in two timestamps:

claude-newiqa@gmail.com.json — refreshed today, 06:41 · gemini-newiqa@gmail.com-….json — last touched March 27

The Claude token refreshed daily. The Gemini one had not moved in five months. My health-check script only ever looked at claude-*.json, so nothing ever noticed. The fallback had been decorative since spring — a green light over an empty room. That single observation is what shaped everything below: a fallback nobody actually calls is not a fallback.

Why a gateway does not solve this

The obvious move is a local LLM gateway — Bifrost, LiteLLM — with provider failover in its config. I already run one (CLIProxyAPI), so adding a second would have bought nothing but a second daemon to babysit.

More importantly, a gateway fails at the actual job. Failover at the API layer means swapping the model underneath one client. Feed Claude Code a Gemini response through a translation layer and it falls apart on tool-use and streaming. The fallback has to switch the CLI, not the model behind it. That is a launcher's job, not a proxy's.

Six rungs, two axes

Provider and model fail independently, so the ladder interleaves them. Each rung buys you something specific, and the cost of climbing down is explicit:

In code a rung is a triple — backend, the model to probe, the model to run with. The empty run-model on the top rung matters more than it looks: passing no --model preserves the profile's opus[1m] and its 1M context. Naming the model explicitly would silently cut that down to plain Opus on every launch.

# Each rung: backend | model to probe | model to run with
# An empty run-model means "let the CLI use its configured default", which keeps
# the profile's opus[1m] (and its 1M context) instead of downgrading it to plain
# opus just to name it explicitly.
RUNGS=(
  "claude|$AGENT_OPUS_MODEL|"
  "claude|$AGENT_SONNET_MODEL|sonnet"
  "claude-proxy|$AGENT_OPUS_MODEL|$AGENT_OPUS_MODEL"
  "claude-proxy|$AGENT_SONNET_MODEL|$AGENT_SONNET_MODEL"
  "agy||"
  "gemini||"
)

When the top rung is unavailable, the drop is announced rather than silent — a degraded session should never be mistaken for a normal one:

$ agent -p "explain this bug"
[agent] Claude Code (claude-opus-5) unavailable — claude-opus-5 unavailable: {"error":...}
[agent] using Claude Code (claude-sonnet-5)

# ...the session continues on Sonnet, inside Anthropic, with no vendor switch.

Model lists lie

The first instinct is to probe cheaply: ask the gateway for /v1/models and check the model is there. This is worthless. My proxy happily advertised gemini-* the entire time every call to those models returned auth_unavailable. The Antigravity CLI does the same — agy models answers from a catalog, not from a working session.

The same applies to model health. No listing endpoint will ever tell you Opus is rate-limited right now. Only a real request shows a 429 or a 529. So every probe makes one:

# Real /v1/messages call for ONE model. This is what distinguishes "Opus is
# capped" from "Anthropic is down" — no model list can tell you that.
probe_model() {
  local m="$1" body
  body=$(curl -sS --max-time "$PROBE_TIMEOUT" \
    "$AGENT_PROXY_URL/v1/messages" \
    -H "Authorization: Bearer $AGENT_PROXY_KEY" \
    -H 'content-type: application/json' \
    -d "{\"model\":\"$m\",\"max_tokens\":1,\"messages\":[{\"role\":\"user\",\"content\":\"hi\"}]}")
  case "$body" in
    *'"type":"message"'*) REASON="$m answered"; return 0 ;;
  esac
  REASON="$m unavailable: $(printf '%s' "$body" | tr -d '\n' | cut -c1-140)"
  return 1
}

Per-model availability for the official rungs is borrowed from the proxy, which fronts the same Anthropic account — a model capped there is capped in the official CLI too. The whole preflight measured at 7 tokens and about 1.3 seconds: a free unauthenticated 401 liveness check plus one 1-token call, with the Opus result cached so two rungs cost one probe.

Two safety nets, not one

A preflight probe cannot catch everything. If the quota runs out in the gap between the probe and the launch, the probe was right and still useless. So there is a second net: exit code plus elapsed time.

start=$SECONDS
run_rung "$backend" "$rmodel" "$@"
rc=$?
elapsed=$(( SECONDS - start ))

# Clean exit, or the user interrupted it — either way, done.
if [ $rc -eq 0 ] || [ $rc -eq 130 ]; then
  exit $rc
fi

# Survived long enough to have been genuinely used: a real error, not a rung
# that never started. Do not silently rerun the work somewhere else.
if [ $elapsed -ge $FASTFAIL_SECONDS ]; then
  exit $rc
fi

warn "$(label "$backend" "$pmodel") exited $rc after ${elapsed}s — treating as unavailable"

The 25-second threshold encodes a judgement. A rung that dies almost immediately never really started — bad auth, capped model — so moving on is safe. A rung that ran for minutes and then failed was genuinely working, and silently re-running that work on another vendor would be worse than surfacing the error.

The watchdog, and the trap inside it

The launcher only runs when I run it, so a rung can rot between sessions. A check wired into my existing health droid (launchd, every two hours, Telegram alerts) calls the same probe code. My first version had exactly the bug this whole article is about: it returned success on the first green rung.

# Healthy — every non-Anthropic rung answers
$ agent --check-fallback
all provider-independent fallbacks OK: agy — antigravity answered...; gemini — ...
rc=0

# One rung rotted while the other still works. This is the case that used to
# pass silently, and the whole reason the check exists.
$ GEMINI_API_KEY=broken agent --check-fallback
DEGRADED: still covered by agy — antigravity answered on Gemini 3.6 Flash (Low)
but a rung died: gemini — gemini call failed: {"error":"Invalid API key"}
rc=1

That is wrong, because the two non-Anthropic rungs hold separate tokens — the Antigravity CLI has its own, while the Gemini rung uses the proxy's. One can rot while the other looks perfect, and stopping at the first success would hide it exactly the way the original five-month failure hid. So the check probes every rung and treats DEGRADED as a failure too: early warning while there is still a spare.

Three mistakes worth copying down

Every one of these produced a green signal over broken machinery — the same failure mode, three times in one afternoon:

• The probe tested a model I never use. I probed gemini-3.1-pro-preview while the CLI was configured for gemini-3.1-pro-low. They resolve to different providers, so I diagnosed an "expired token" that did not exist — the real answer was a provider that was never logged in. A probe must exercise the exact path the real client takes.
• A silently empty variable. Parsing the proxy's YAML key with awk and \x27 for the quote character produced an empty string on BSD awk — no error, no warning. The check still passed, because the rung it happened to test needed no key. Passing the quote via awk -v q="'" fixed it.
• The probe passed while the real thing failed. A curl probe against the Gemini endpoint returned 200, but the actual gemini -p refused to start: headless mode blocks on untrusted directories. The rung was broken precisely in the situation it exists for, and only running the real CLI revealed it.

Using it

Typing agent gives you the best available rung with the normal Claude Code experience. Explicit selection skips probing entirely — if you ask for Gemini you get Gemini, not an argument about why Claude is better:

agent                          # auto-pick the best live rung
agent --use gemini             # force a backend
agent --use claude:sonnet      # force a backend AND a model
agent --use claude-proxy:opus  # opus/sonnet/haiku map to full ids per backend
agent --list                   # backend names
agent --status                 # probe every rung and report
agent --check-fallback         # exit 1 if a non-Anthropic rung is missing

agent -p "..."                 # flags pass straight through to the chosen CLI

Conclusions

The engineering here is unremarkable: a bash script, some curl calls, an ordered array. What took the effort was distrusting my own checks. Three times I built something that reported success while the underlying thing was broken, and each time the cheap-and-fast probe was the culprit.

Resilience is not the list of backends you configured. It is whether anything verifies that the list is still true. Mine had six rungs on paper and one real one for five months, and only a check that made an actual call could tell the difference.

Sources

• Claude Code Documentation — headless mode, ANTHROPIC_BASE_URL, model flags
• CLIProxyAPI — the local multi-provider gateway behind the proxy rungs
• Gemini CLI — the last rung, and its trusted-folders behaviour
• Apple Developer — launchd jobs, used by the watchdog