Smart Balancers
One endpoint. A live routing board behind it.
Route by configured intent and priority, with ordered failover between healthy destinations.
Live board
Example balancer: support-balancer
request
customer prompt
endpoint
Example endpoint: support-balancer
efficient route
active
Example: qwen-cheap
summaries, drafts, simple replies
reasoning route
active
Example: gpt-oss-120b
multi-step support decisions
fallback route
active
backup group
same endpoint keeps answering
Control plane
Move routing decisions into the control surface.
Treat Smart Balancers like a routing control surface: keep one caller contract in place, tune route priorities behind it, and decide when an intent-based route or ordered failover should handle the request.
Request enters
01Your app calls one stable endpoint.
The SDK keeps using the example balancer, support-balancer, instead of selecting raw model endpoints in application code.
Route selected
02The board chooses the right lane.
Route by configured intent and priority. Intent-based routing adds a router-model call, which can add latency and cost; ordered mode provides deterministic failover between healthy destinations.
Fallback stays ready
03The backup path is already wired.
When the primary route fails, Smart Balancers keep the same caller contract and move traffic behind it.
Fallback routing
Fail over without teaching every app a second route.
When the primary path turns unavailable, Smart Balancers keep the caller on the example balancer support-balancer and move the route in the control layer.
Example: support-balancer
client unchanged
primary route
priority 1
health state
unavailable
fallback route
active
01 Primary
Requests enter the example balancer support-balancer and try the primary group first.
02 Failure
The primary route is marked unavailable inside the control layer.
03 Fallback
The same endpoint keeps answering through the fallback group.
Prompt-based routing
Spend strong models where the prompt earns it.
Put configured-intent routing behind one endpoint. Intent-based routing adds a router-model call, which can add latency and cost; ordered mode provides deterministic failover between healthy destinations.
board endpoint
Example: support-balancer
Both decisions happen behind the same endpoint, so clients keep calling the same model name while the board chooses the route.
Example routing decision
Summarise this support reply.
efficient route
Example routing decision
Resolve a multi-step refund dispute.
reasoning route
What Smart Balancers include.
Routing, fallback, and cost control without changing the client contract.
Target groups
Group multiple models behind one balancer so the endpoint stays stable while routing logic evolves.
Fallback routing
Pair a primary group with a fallback group and use ordered failover between healthy destinations.
Prompt-based routing
Intent-based routing adds a router-model call, which can add latency and cost when selecting a destination from configured intents.
Route priorities
Ordered mode provides deterministic failover by trying healthy destinations in priority order.
OpenAI-compatible endpoint
Keep existing SDK flows and point them at one balancer endpoint instead of one raw model endpoint.
Capacity pricing inheritance
Compute-backed routes retain capacity pricing; Public Model routes remain token-priced.
Existing SDK flow
One balancer endpoint. Existing SDK flow.
Your caller talks to one endpoint. Smart Balancers decide which group answers behind it, in what order, and when fallback takes over.
Compute-backed routes retain capacity pricing; Public Model routes remain token-priced.
from openai import OpenAI
client = OpenAI(
base_url="https://api.qdiv0.com/v1",
api_key="your-api-key",
)
response = client.chat.completions.create(
model="support-balancer",
messages=[{"role": "user", "content": prompt}],
)efficient_group
Example configured intent route for a lower-cost destination.
reasoning_group
Example configured intent route for a stronger destination.
fallback_group
Keep a backup route behind the same endpoint.
Balance cost, quality, and continuity behind one endpoint.
Put routing logic in the control layer, not in every client that calls it.