Stop hardcoding model names: routing across open-source LLMs

When I started comparing open models seriously, the list kept growing: Kimi K3, GLM-5.2, DeepSeek V4 Pro, Qwen 3. Every few weeks something new landed and I caught myself asking which one I should switch to. Eventually I realised that was the wrong question. The better one was why my code cared at all.

If a model name is written into forty places, switching is a project. If it is written into one place, switching is an edit. So I moved to a design where models are interchangeable providers behind a router. The calling code says what kind of task it has, such as summarising, pulling fields out of text, or drafting, and the router decides which model gets it. Different task types can go to different models, and I can try a new model on one task type without touching the others.

Loom, the automation platform I am building, follows the same idea. The LLM layer is provider-agnostic, and each provider can sit behind a circuit breaker. The pattern is simple: when a provider keeps failing, stop sending it requests for a while and use the next choice, instead of stacking up retries against something that is down.

I have also tested running GLM-5.2 and Kimi K2-family models on an 8×B200 setup, mostly to understand what self-hosting really involves in terms of memory and batching before making any promises to myself about it.

If you are building on top of these models, my advice is short. Pick a boring interface early, keep each prompt next to the task it belongs to, and let the routing table be the only place in your codebase that knows a model name.