Tagged “routing”
2 reports on this topic. All publications →
-
Right-Sizing the Model to the Question: Cutting AI Chatbot Energy Without Losing Quality
AI chatbots answer every question with the largest model available — the same frontier system whether the user asks about contract law or for a recipe. Inference energy scales with model size and answer length, so a model roughly ten times smaller spends…
Read the report → -
Too Many Tools Break Mid-Size Models: A Two-Stage Method for Reliable Tool Use
Model cards imply broad tool support. Operating a Mistral-based assistant with more than twenty tools, we found reliable behaviour only up to roughly four to ten tools per call, with reproducible failures past that: the model answers from memory in prose…
Read the report →