Models, costs, operations.

10.1 A model is not just a model: choosing by task

In the first chapters you learned that a coding agent consists of a language model, tools and a loop. The language model is the most interchangeable and at the same time the most expensive component. Providers almost always offer several model classes: large, slow, expensive models with a lot of judgment; mid-sized models as workhorses; small, fast, cheap models for simple, well-defined tasks. Names, versions and prices change on a monthly basis. The principle behind them stays stable.

The principle is: Not every task needs the strongest model. In everyday software work with agents, three task types can be distinguished that make very different demands.

Planning

When planning, the agent breaks a goal into work packages, weighs architecture options, and identifies dependencies and risks. Mistakes in this phase are expensive because they carry through all subsequent steps: a wrong cut produces ten wrong build runs. At the same time, the output of a planning run is comparatively short — a few Markdown files instead of hundreds of lines of code. A strong model pays off here: the extra cost per token applies to few tokens and avoids expensive downstream errors.

Building

When building, the agent implements a well-defined plan: creating files, writing code, adding tests, running the build, fixing errors. This work is extensive but mostly more mechanical. A solid mid-sized model usually handles it reliably if the task is precise — which is exactly what the spec and plan from chapters 4 and 5 ensure. This is where most of the token volume arises; the model class therefore has a strong effect on total costs.

Reviewing

When reviewing, a reviewer reads a diff and reports findings. Reviews are frequent, often run in parallel and work with limited context. A small, fast model is often sufficient here, especially since a good review process does not rely on an LLM alone anyway but adds rule-based checks (tests, static analysis, coverage checks) alongside it. Where a review is very demanding — a security assessment, for example — a stronger model can make sense.

Model choice is a configuration question, not a code question

Because models become outdated quickly, no model name should be hard-wired into tools, scripts or prompts. If you configure model choice centrally, you can switch in one place when a new model arrives — and know afterwards which run was produced with which model. This traceability is not just tidiness: when a result stands out, “which model was that?” is one of the first questions.

10.2 Tokens: the currency of agents

Language models compute in tokens. A token is a piece of text — roughly a word fragment; in German a word often takes more tokens than in English, and source code with many special characters is also token-heavy. With API usage, billing is typically split into three kinds:

Open full-size graphicOpen full-size graphic

Why agent runs consume so much input

Remember the agent loop from chapter 2: the model thinks, calls a tool, gets the result, thinks further. In every round, the entire context so far is processed again — the instructions, the project context, all previous tool calls and their results. A run with forty tool calls therefore processes the beginning of the context roughly forty times. If the agent reads in a large log file or a long test report along the way, that text travels along into all following rounds.

This leads to an important insight: The cost of an agent run does not grow linearly with the size of the task, but rather with the number of rounds times the context size. An unclear task that forces the agent to search and experiment is therefore doubly expensive: more rounds and more context.

Prompt caching

Many providers offer prompt caching: an unchanging beginning of the context (prefix) is stored the first time and reused more cheaply in subsequent requests. This fits agents well, because instructions, tool definitions and project context stay identical from round to round. The details differ by provider (cache lifetime, minimum size, surcharge for writing, discount for reading); in general:

• Writing to the cache usually costs somewhat more than normal input, reading costs considerably less.
• Caching only works on a stable prefix. If you keep putting changing content at the beginning of the context (timestamps, random orderings), you prevent cache hits.
• Agent CLIs usually use caching automatically. You influence it mainly indirectly: through stable project files and instructions.

Subscription instead of a token bill

Besides per-token API billing, providers of coding agents also offer subscriptions in which runs count against usage limits rather than a bill. The terms change frequently and are bound to the respective provider’s terms of use — in particular, who a personal subscription is intended for and for what purpose. For the factory, the current state is described in the user manual and on the subscription setup page.

10.3 Cost levers: what really saves

Anyone who wants to reduce agent costs often thinks of a cheaper model first. That is one lever, but rarely the biggest. The most effective levers act earlier.

Lever 1: Precise tasks

A precise goal with clear acceptance criteria drastically reduces the number of rounds. The agent has to search less, guess less and rework less. This is where the work from chapter 4 (specs and EARS) and chapter 5 (planning) pays off directly in tokens: a build run against a well-cut backlog item with bound acceptance criteria is shorter than a run with the goal “finish the order management”.

Lever 2: Keep context small and stable

• Small: Only put into the context what the task needs. Huge log outputs, generated files or dependency directories are expensive ballast.
• Stable: Project files, instructions and guardrails should not change with every run. This helps caching and makes results more comparable.
• Curated: A well-maintained project memory with the most important decisions is better than a growing pile of unfiltered notes.

Lever 3: The right model class

As described in 10.1: a strong model for planning, the workhorse for building, a small model for reviews. What matters is measuring the choice, not guessing. If a cheaper build model needs twice as many correction loops, it is not cheaper in the end.

Lever 4: No blind repetitions

A failed run that is restarted unchanged costs the same again — with a similar chance of success. A repetition is only worthwhile if new information is added: compiler output, test failures, a reviewer finding, a merge conflict, a changed instruction. This is exactly the principle the factory implements in its correction loop (chapter 6): it feeds the build or gate feedback into the repeated run and limits the number of attempts.

Lever 5: Small steps

A large run that fails on a trifle after an hour burns a lot. Several small runs, each with a verifiable result, fail more cheaply and can be corrected in a more targeted way. It is the same logic as with small pull requests — except that money is also at stake here.

10.4 Tenants, roles and the four-eyes principle

As soon as more than one person works with agents, two questions arise: Who may do what? and Who sees what? The first is answered by a role model (RBAC, role-based access control), the second by tenant separation.

Roles: graduated permissions

Good role models for agent platforms separate at least three things: reading, executing and approving. Executing means letting agents write code — this costs money and changes repositories. Approving means taking responsibility for a result. If you do not separate these permissions, you lose the four-eyes principle: the same person commissions the agent and waves its result through.

Tenants: hard visibility boundaries

A tenant is an isolated unit — a department, a customer or a subsidiary, for example. A tenant’s data must not be visible to others, not even via guessed IDs. This is more than a convenience feature: agent runs contain source code, requirements, logs and costs, i.e. often confidential information.

Single sign-on

In companies, identity usually comes from a central identity provider (OIDC). An important security principle here: the integration must never accidentally extend access. New, automatically created users should start with minimal permissions; the actual assignment remains a deliberate administrative act.

10.5 Deployment forms and operating modes

Agent platforms are operated in very different ways: from an individual on their own laptop to an isolated government installation without internet. The deployment form decides three questions:

• How does the model come into play? Personal subscription, API key, cloud gateway or local model.
• How many people share the instance? Roles, tenants and budgets depend on this.
• How much network is allowed? From “everything” to “nothing” (air gap).

One point deserves special attention: Personal subscriptions are personal. A subscription intended for an individual’s use of a coding agent is not access that a team should share via a common platform. As soon as several people use an instance, the runs belong on API keys or a gateway — with clean billing and clear terms of use.

Configuration without restarts

A configuration hierarchy is helpful in everyday work: general defaults globally, deviations per user or project. That way not every project has to define everything anew, and special cases remain visible where they belong.

10.6 Summary

• Choose model classes by task: strong for planning, workhorse for building, small for reviews — and measure the choice.
• Agent costs grow with rounds times context size. Precise tasks and a small, stable context are the biggest levers.
• Prompt caching rewards stable prefixes; cache writes cost somewhat more, reads considerably less.
• Repeat only with new information.
• The factory resolves the model by override → team → role default (Claude only) → CLI choice and attests the source.
• Euro values in Claude subscription mode are estimates at the API rate, not a bill.
• Budgets first observe-only, then as a hard cap per tenant; the seat view evaluates but does not limit.
• Roles separate executing and approving; ADMIN is not a data super admin; SSO users start as VIEWER without a tenant.
• Personal subscriptions belong on a single workstation; teams work in TEAM_API_POOL.

Chapter questions

1. Why is a more expensive model often worthwhile for planning runs compared to build runs?

   Model answer

   Planning mistakes multiply, because all subsequent build runs build on the plan. At the same time, a plan run produces comparatively little output. The extra cost per token therefore applies to few tokens and prevents expensive downstream errors. Build runs, by contrast, produce most of the volume for rather mechanical work, where a mid-sized model is usually sufficient.

2. Explain why the cost of an agent run does not grow linearly with the size of the task.

   Model answer

   In every round of the agent loop, the entire context so far is processed again: instructions, project context, all tool calls and their results. Costs therefore grow roughly with the number of rounds times the context size. Unclear tasks increase both, and large outputs that are read in inflate every subsequent round.

3. A build run with the claude adapter ran with Sonnet, even though you entered Opus on the developer agent. What is the most likely cause, and where do you check?

   Model answer

   A routing override run.routing.build-model takes precedence over the team’s preferred model. Alternatively, a different team was selected. In the why-trace (/runs/{id}/trace), RUN_MODELL_AUFGELOEST with quelle=override, team, rollen-default or adapter-default shows where the model came from.

4. Why does a run with the Aider adapter not get a Claude role default?

   Model answer

   Aider (like Bedrock/Vertex/Azure, Local-LLM and Kimi) passes --model unchanged to its own CLI or backend. A Claude model name would be wrong or meaningless there. For these adapters, the default model per adapter or the CLI’s choice applies.

5. In the cost view, a Claude run shows a euro amount with the note “estimate (API rate), not billed”. What does that mean, and does the amount count against the budget?

   Model answer

   The run ran in subscription mode and counts against the subscription’s usage limits; no API bill arises. The factory still values the tokens at the API rate to make the generated load visible. The amount counts in consumption and therefore against the budget and hard cap.

6. A run with several hundred thousand tokens shows €0. What are the two possible explanations?

   Model answer

   Either it is a flat-rate model (e.g. k3, kimi-for-coding), for which €0 is deliberately recorded, or the model has no entry in the price table in application.yml and is conservatively valued at €0. In the second case, the table should be extended.

7. An administrator complains that she cannot see another tenant’s projects. Is that a bug? What needs to be done?

   Model answer

   Not a bug: in the factory, ADMIN is not a super admin for data. Projects, runs, logs and costs remain tenant-scoped. Only account management and project assignment are isolation-free. The solution is the tenant assignment (assign the account or project to the right tenant) or a deliberate organizational arrangement.

8. Why do users newly created via SSO receive the role VIEWER without a tenant, and which general principle lies behind this?

   Model answer

   An SSO integration must never accidentally extend access. Minimal permissions at first login and a deliberate administrative assignment of role and tenant implement the principle of least privilege and fail-safe defaults.

9. Transfer: You are asked to cut your team’s agent costs by a third. Name three measures in the order in which you would examine them, and justify the order.

   Model answer

   Example: (1) Make tasks more precise and cut them smaller (specs with acceptance criteria, backlog items instead of a mega-goal), because that reduces rounds and context at the same time. (2) Stop repetitions without new information and evaluate correction loops, because blind retries are pure waste. (3) Only then adjust the model classes per task and measure the effect in /analytics, because a cheaper model with more corrections can end up more expensive.

10. Which three things do you need to back up, and why does the attestation key belong to them?

    Model answer

    The PostgreSQL database, the workspace directory with the Git repositories, and the attestation key together with the secrets master key. Without the attestation key, the signed audit chain can no longer be verified later; without the master key, stored secrets can no longer be read. After the restore, the chain verification under /attestierung shows whether the backup was complete.