Context and Cost Control

Chat Completions is stateless. GLM 5 does not remember an earlier API call unless your application includes that history again in messages.

That makes context management an application responsibility—and the most important cost control for long-running chats and agents.

How repeated history is billed

Consider a conversation whose input grows on every turn:

The model sees and processes every token included in each request. Earlier messages can therefore be billed repeatedly.

Recommended strategy

Keep a token budget

Choose a maximum input budget for your application before sending a request. When history approaches that budget:

1. Preserve the system message.
2. Preserve the latest user request.
3. Preserve complete assistant tool-call and tool-result pairs.
4. Keep the most recent useful turns.
5. Summarize or remove older turns.

A message-count limit alone is unreliable because one message can contain far more tokens than another.

Summarize old turns

Replace older history with a compact system-level summary:

[
  {
    "role": "system",
    "content": "You are a technical assistant. Conversation summary: the user is deploying a Next.js API on Cloudflare, requires zero-downtime rollout, and has not yet configured rollback alerts."
  },
  {
    "role": "user",
    "content": "Now write the final deployment checklist."
  }
]

Summaries can omit details, so retain recent messages and important identifiers verbatim.

Start a new conversation

If earlier context is no longer relevant, send only the current system and user messages. This is both cheaper and less likely to distract the model.

Control output reservation

Before a request reaches the model, GLM 5 reserves credits using:

• The estimated input.
• The requested maximum output when one is supplied.
• An internal 8,192-token billing estimate when no output cap is supplied.

The internal estimate is not sent to the model and does not truncate the response. If neither max_completion_tokens nor max_tokens is supplied, the selected provider chooses its own default. For short answers, set a smaller explicit value:

{
  "model": "glm-5.2",
  "messages": [
    {
      "role": "user",
      "content": "Return a five-item checklist."
    }
  ],
  "max_completion_tokens": 600
}

The reservation is reconciled against actual usage after completion. Unused reserved credits are returned, but a large maximum can still cause a low-balance request to fail its preflight reservation.

Provider-enforced context limits

GLM 5 does not maintain or enforce a separate token context window. Requests are forwarded to the selected upstream model, whose provider decides the current context and output limits. This avoids stale local metadata rejecting a request that the provider can handle.

If the upstream provider returns context_length_exceeded, shorten or summarize the history, reduce tool definitions, or set a smaller output maximum. Provider limits can change independently and may differ between routing providers for the same model ID.

Monitor usage

A non-streaming response includes:

{
  "usage": {
    "prompt_tokens": 42215,
    "completion_tokens": 186,
    "total_tokens": 42401
  }
}

Useful safeguards include:

• Warn at 50%, 75%, and 90% of the application's context budget.
• Log input and output tokens per request.
• Set per-user or per-workflow spend limits.
• Alert on repeated growth across consecutive requests.
• Stop automated loops after quota or validation errors.

Context caching: helpful, but not a guardrail

Some model providers can recognize an identical prompt prefix from an earlier request and process those repeated tokens at a lower cost. This is commonly called prompt caching or context caching. It works best when the system prompt, tool definitions, and older messages remain byte-for-byte identical.

Caching can reduce upstream latency and token price, but it does not:

• Remove old messages from the request.
• Stop the context window from growing.
• Guarantee a cache hit after content changes or cache expiry.
• Make repeated tokens free.

For comparison, Z.AI context caching reports cached input separately and applies a lower cached-token rate. A similar implementation in GLM 5 would need to record cached tokens, expose them in usage.prompt_tokens_details.cached_tokens, and pass the discount through to credit settlement.

Request-size limit

The JSON body limit is 4 MB. This transport and memory protection remains in place, but it is not a token context limit. A request below 4 MB is forwarded and the selected provider decides whether its model can process the token count.