My Four-Tier Model Routing Policy for Self-Hosted Agents

I used to let too many task types share one default model lane. The stack got weird fast. Cheap jobs started costing too much, background work crept into foreground time, and the tasks that actually needed stronger judgement showed up late.

That is the problem this model routing policy solves in my OpenClaw setup. I run a mixed local-and-cloud stack on a CPU-only VPS, so routing is not a nice optimisation. It is the difference between a system that feels usable and one that feels like punishment.

Last updated: 2026-04-05 · Tested against the live OpenClaw routing notes and the Ollama-based local model layer in my stack

Contents

1. What problem does a four-tier model routing policy solve?
2. How do the four tiers work in practice?
3. When should a task move up a tier?
4. What broke before I started routing work like this?
5. How would I implement this without rebuilding my whole stack?
6. Frequently asked questions
7. Why this policy matters more than the exact models

What problem does a four-tier model routing policy solve?

The short version is this: not every task deserves the same intelligence budget, latency budget, or failure blast radius.

In my OpenClaw stack, the cheapest local jobs are things like cron, heartbeat checks, and summaries. At the other end, the main chat agent and a few user-facing workflows do need stronger judgement, better tool use, and a cleaner final answer. Treating those two ends as the same job class is how you burn money and patience at the same time.

This is even more obvious on a CPU-only VPS. Local inference is useful, but it is not free in operational terms. Small jobs are fine. Bigger reasoning jobs and coder-heavy tasks are noticeably slower, so the policy exists to stop slow work from wandering into the wrong lane.

The routing policy exists to respect that reality. If a cheap job can stay cheap, I keep it there. If a coding task needs code-aware output, it goes to the coder lane. If a decision is high-stakes enough that a bad answer costs more than the tokens, it escalates to cloud immediately.

How do the four tiers work in practice?

I use four buckets: L0, L1, L2, and C1. The names are boring on purpose. I want the routing rule to be operational, not poetic.

What's model routing? Model routing is the policy layer that decides which model gets a task before the prompt ever runs. It is less about prompts and more about protecting the system from bad defaults.

L0 is for jobs that should almost disappear into the background. If they fail occasionally, the system can retry. That makes them ideal for the cheapest local lane.

L1 is for operational reasoning that still benefits from local execution. This is the middle of the stack. It is where I want decent judgement without paying cloud rates for every housekeeping task.

L2 is where code-aware work lives. I learned the hard way that "bigger local model" and "coder model" are not the same thing. A general local model can be fine at prose reasoning and still be annoying at diffs, file-level edits, or structured technical output. Giving coding work its own lane is part of what keeps the rest of the stack readable.

C1 is the expensive lane. That is exactly why I protect it. Main chat, polished responses, and the work that can create user-facing damage if it goes sideways all live here. If you want broader context, stronger tool use, or a noticeably better final answer, this is the lane that earns the spend.

If you want the broader OpenClaw runtime context behind these choices, What Is OpenClaw? covers the stack shape and why each agent gets its own execution lane.

When should a task move up a tier?

The routing policy is only useful if escalation rules are clear. "We will just know when it needs a better model" is how everything drifts to premium by default.

I use a simple escalation test:

1. Can the cheaper tier do this reliably? If yes, keep it there.
2. Does failure create visible user damage or operational drag? If yes, move it up.
3. Is the task code-shaped rather than prose-shaped? If yes, push it toward the coder tier.
4. Does the task need judgement more than throughput? If yes, escalate to cloud sooner.

The easiest way to think about it is by failure cost.

That is the real routing logic. Not "big task vs small task." It is "what happens if the model gets this wrong?"

For example, a summary that lands a bit rough is fine. A publish-ready explanation that misstates your architecture is not. A slightly slow cron check is annoying. A main chat lane that stalls because background work is hogging the same inference path is a design problem.

What broke before I started routing work like this?

The first mistake was letting too many jobs share the same default lane. That sounds harmless until background work starts competing with interactive work.

In OpenClaw, one of the critical ideas is that each agentId gets its own execution lane. That matters because cron jobs on the main agent block the chat experience. I learned that one painfully enough that it turned into a hard architecture rule: the cron agent runs separately, and background work should never quietly pile onto the main conversation lane.

The second mistake was pretending local inference cost is only about money. It is also about CPU time, queue pressure, and tolerance for delay. I now keep the local layer intentionally constrained because that is how I stop the whole box from feeling soggy.

The third mistake was not separating coding work early enough. Code tasks have their own failure texture. Weak code edits create cleanup work, and cleanup work tends to hit the expensive lane later anyway. You do not save money if the cheap route just manufactures rework.

There is a broader VPS story behind that operational discipline too. How I Manage 30 Docker Services Without Losing My Mind gets into the server-side version of the same lesson: once a stack grows, clean boundaries matter more than clever shortcuts.

How would I implement this without rebuilding my whole stack?

You do not need a fancy orchestration framework first. You need a routing table, hard job boundaries, and the discipline to stop cheating.

1. Name your tiers. Use plain labels like L0, L1, L2, and C1. The naming is not the important part. The consistency is.
2. Assign one default model per tier. Do not give every agent a free-for-all list. Make the default obvious.
3. Map task classes to tiers. Summaries, cron, code edits, publish-quality output, and user-facing chat should all have a first-choice lane.
4. Write escalation rules before you need them. Decide what counts as "move up" now, not in the middle of an incident.
5. Protect the premium tier. If cloud is the best model in the stack, treat it like a scarce resource instead of a lazy default.
6. Review the misses monthly. Every bad route teaches you whether the problem was the model, the prompt, or the task classification.

If your system already has tool calling or callable boundaries, the Model Context Protocol is useful context for thinking about where model selection sits in the wider workflow. The routing rule should happen before the expensive work starts, not after the wrong lane has already burned time.

You should also keep the local layer grounded in the actual models you can host and tolerate. The Ollama model library is useful for availability and packaging, but the better question is "what still feels acceptable on my hardware under load?"

Frequently asked questions

Do I need four tiers, or is that overkill?

No, you do not need four tiers on day one. Two tiers is enough to start: one cheap local lane and one premium lane. The reason I use four is that coding work and medium-complexity operational work behave differently in practice, and I wanted that difference to be explicit.

When should I keep a task local instead of sending it to cloud?

Keep it local when the task is repeatable, low-risk, and easy to retry. Move it to cloud when a wrong answer costs more than the tokens or when the task needs stronger judgement, broader context, or a better final finish.

Why not just run the best cloud model for everything?

Because cost is only one constraint. Latency, system shape, failure boundaries, and operational discipline matter too. Routing forces you to be honest about what the work actually needs.

What is the biggest mistake people make with model routing?

They classify by vibes instead of by failure cost. "Feels important" is a weak rule. "Hurts the user if wrong" is a much better one.

Why this policy matters more than the exact models

The exact models will change. That part is guaranteed. What tends to hold up is the policy layer.

The useful decision is not "which model is best?" It is "which class of work deserves which class of model?" Once that line is clear, swapping models becomes much less dramatic. You can improve a tier without redesigning the whole stack.

That is why I think routing policy matters more than model brand loyalty. A clean routing system survives model churn. A messy one turns every new release into another excuse to break your own rules.

If you want the adjacent publishing-side version of this same idea, How Claude Published Directly to Labs via MCP shows what happens when you treat system actions as callable operations instead of manual handoffs.

If you are building a self-hosted agent stack, start by drawing the task classes before you touch the prompts. The routing rule will save you more pain than one more clever model swap ever will.