OmniRoute
An OpenAI-compatible AI gateway whose opt-in jev combo strategy asks TypeSafe's Jev which of the configured models should answer each request.
An OpenAI-compatible AI gateway whose opt-in jev combo strategy asks TypeSafe's Jev which of the configured models should answer each request.
The state is the incoming request plus the operator's ranked list of configured models and their descriptions, capped at 8,000 characters. One choice question keyed route asks which of those models is the most appropriate for this request, with a routing instruction, up to 255 options and the jev-latest model. The returned choice, the per-option probabilities and the confidence reach the combo dispatcher, which reorders the providers it will try before any chat completion is sent to them.
Which of these configured models is the most appropriate for this request?
The state carries the request text and the operator's ranked list of configured providers, with each entry's description, truncated to 8,000 characters. The option list is the operator's own configuration; the four ids below stand in for it.
The gateway receives an OpenAI-compatible chat request on its single endpoint and resolves the combo the request selects.
The jev strategy runs only when COMBO_JEV_STRATEGY_ENABLED is registered as on; otherwise the operator's order is kept and no credential is read.
The request text and the configured models become one choice question with a routing instruction, a 255-option ceiling and an 8-second timeout.
A usable answer sets the provider order the dispatcher tries first; a missing key, timeout, HTTP error or invalid body falls open to the operator's order and is logged.
The chosen provider receives the chat completion, with OmniRoute's token compression and failover rules applied across the remaining providers in that order.
Model selection becomes a single typed call with a hard 8-second ceiling instead of a reasoning step: the gateway keeps the operator's order whenever the flag is off, the key is missing, the answer is malformed or the call is slow, so a paid routing hint can only improve the order and never fail the request.