Article

Why a model writes its tool call as text

1 November 2026 · Todenta · 6 min read · Compiled by AI, edited by a human

One day someone typed a question about revenue into an accounting application's chat field. The answer was a SQL query. The whole query — table names, conditions, everything — right in front of the customer.

The easy explanation is that the model is bad. That was not the cause.

What actually happened

The system gives the model tools: functions it can call to fetch data. The model does not see the database. It sees a list of tools and their descriptions, and if it needs numbers, it is supposed to call a tool.

A tool call travels through the API as a structured field — separate from text. The response has two fields:

{
  "choices": [{
    "message": {
      "content": null,               ← text answer
      "tool_calls": [{               ← structured call
        "function": { "name": "fetch_numbers", "arguments": "{...}" }
      }]
    },
    "finish_reason": "tool_calls"
  }]
}

When the model uses the tool_calls field, the application sees the call and can execute it. The user sees nothing — only the final answer.

But the model can also do something else: write the same thing as text in the content field.

{
  "choices": [{
    "message": {
      "content": "[TOOL_CALL]{\"tool\":\"fetch_numbers\",\"args\":{...}}[/TOOL_CALL]",
      "tool_calls": null             ← no call
    },
    "finish_reason": "stop"
  }]
}

Then the application sees only text. No function to call. And if the application displays the content field as-is — as a chat window does — the user sees the raw text.

Measured in this system:

  • Short question to the model directly → finish_reason: stop, no tool_calls key at all
  • Long system prompt + tools included → [TOOL_CALL] as text in the content field
  • The API log recorded the call as successful — there was no error anywhere

The last point is what makes this hard. The system did not fail. HTTP 200, valid JSON, a valid response. The only strange thing was its content — and nothing validates the correctness of content.

Why the model does it

The exact shape of a tool call is an API contract, not a model feature. The model is trained to produce both structured calls and text, and which one comes out depends on what is in the prompt and which API version is used.

Three common causes, in this case probably all three:

1. The prompt describes the tool as text

If the system prompt says "you have available a function fetch_numbers which you call to get the numbers", the model may interpret that it should write the call. A structured call requires the tools be passed in the tools parameter, not as text in the prompt.

2. A long prompt drowns out the structure

When the prompt contains a thousand lines of rules, the tools definition becomes a relatively small part of the context. The longer the prompt, the more likely the model imitates the textual form it sees rather than the structure.

3. Tool choice is not mandatory

By default the model may choose: call a tool or answer directly. If it judges it cannot, it may "show effort" by writing the call as text. This is especially common when the model does not know the table structure — it invents table names, and an invented table reveals that the guess failed.

In this case the model invented tables that do not exist: invoices, company_id, amount. The real ones were entries, sales_invoices, purchase_invoices, chart_of_accounts.

Three ways to fix it — and why two of them are bad

Option A: force a tool call

Most APIs support tool_choice: "required", which forces the model to call some tool. It works — but breaks normal conversation. When the user says "hi", the model cannot answer, because it must call a tool.

Usable only when all traffic is tool-driven. Not in a chat window.

Option B: parse the text back into structure

Find the [TOOL_CALL] block in the response and interpret it. This is tempting because it "fixes" the problem visibly. But it is also a contract you have to live with: the model's textual format is not documented, not versioned and not guaranteed. It can change in the next model update.

Option C: clean the response before displaying it

Accept that the model sometimes leaks, and clean it away before the user sees it. Do not try to fix the model's behaviour — fix the result.

This is what we did, because it is the only option that does not depend on the model's internal logic. It is not beautiful, but it is deterministic: the same input always produces the same output.

What cleaning means in practice

The format gate is not one rule but four, and each corresponds to a different kind of leak:

LeakWhat is removed
[TOOL_CALL]...[/TOOL_CALL]The whole block, start to end
A code fence containing SQL or JSONThe whole fence with its contents
At line start, [SQL] or a bare SELECTThat line
At line start, "Searching…", "Checking…", "I will use the tool…"That line

The fourth line is the one I did not understand at first. Once the blocks are removed, what often remains is a promise of what the model was supposed to do:

"Searching for revenue in the accounting of Company 2."

This is neither an answer nor an error. It is a description of intent left behind when the actual call was removed. To the user it looks like the system is about to do something but never gets there.

The part that went wrong first

The first version of the cleaning used a rule: if more than half the response was removed and fewer than 40 characters remained, reject the whole response. A reasonable-looking threshold.

It broke a correct answer:

Result: 1,121,280 EUR       ← 19 characters, but entirely correct

A short answer is not a bad answer. The fix was simple but not obvious: a number alone is enough content. If the remaining text contains a number, it is data. Pure meta-text contains no number.

This is the detail no documentation tells you, and that you learn only by seeing a wrong rejection.

What to take away

  • HTTP 200 does not mean success. A model's response can be formally correct and substantively wrong.
  • A tool call is a contract, not a given. If it does not travel structured, it travels as text.
  • A long prompt increases leak risk. The more rules, the more text the model has to imitate.
  • Thresholds are dangerous. Every "if under X characters" rule is wrong in some case.
  • A log that records only success cannot see this. The critical information was the response's content, not its status.

If you are building the same: put the gate in the response path, not in the prompt. A prompt is a wish, a gate is a check. And record what the gate removed — that number says more than any alert.

Todenta does this as part of its AI governance layer. Read what else it does →

Todenta · Lennoware Oy · Home · One page · Privacy policy