LeetLLM
MediumServingPython 3

Model Fallback Router

Route requests to the cheapest eligible healthy model with capability and quota checks.

30m2 sample tests3 hidden tests

Choose the best healthy model candidate for a request.

Requirements

  • Define choose_model(request, candidates).
  • request has capabilities, tokens, and optional max_latency_ms.
  • Each candidate has id, healthy, capabilities, remaining_tokens, latency_ms, and cost.
  • A candidate is eligible only if:
    • it is healthy,
    • it has every required capability,
    • it has enough remaining tokens,
    • it satisfies max_latency_ms when present.
  • Choose the eligible candidate with lowest cost.
  • Ties use lower latency_ms, then id alphabetically.
  • Return the chosen model ID, or None.

Example

python
1request = {"capabilities": ["tool_use"], "tokens": 100} 2models = [{"id": "small", "healthy": True, "capabilities": ["tool_use"], "remaining_tokens": 500, "latency_ms": 80, "cost": 1}] 3assert choose_model(request, models) == "small"

Constraints

  • Do not mutate candidates.
  • Use deterministic tie-breaking.
  • Missing max_latency_ms means no latency ceiling.

Editor
Results
Run sample tests or submit all tests.