Skip to navigation

LLM Observability plugin

The LLM Observability plugin reads the token usage, response model, and finish reason from each LLM answer that carries them. An OpenAI chat stream carries usage only when the client sets stream_options.include_usage, and answers over 10 MiB aren’t read. For each LLM request, it writes one llm.complete log line on the data plane and, when metering is on and the data plane has a usage writer, one usage record. Other plugins, such as LLM Rate Limit, use the token counts it records.

Where it runs

ProtocolsLLM routes. On MCP and A2A requests it writes a log line with no provider, model, or tokens, and no usage record.
Scope kindsglobal only
PhaseRequest, each stream chunk, and after the response

Your organization can have one instance, and it must be named llm-observability. Attaching it to a route, connection, or consumer is rejected.

Configuration

Each field overrides the data plane’s default for requests that the instance applies to.

FieldTypeDefaultDescription
log_requestbooleanfalseAdds the request body to the llm.complete log line.
log_responsebooleanfalseAdds the response body to the llm.complete log line.
log_streambooleanfalseWrites one llm.stream_chunk log line per stream event.
max_body_bytesinteger4096 (data plane setting)The maximum size of a logged body.
capture_message_contentbooleanfalseKeeps streamed bytes and assembles the streamed reply. Non-streamed answers are always decoded.
max_payload_bytesinteger16384The maximum stream bytes kept when capture_message_content is on.

Logged bodies can contain prompts and personal data. Add the PII Redaction plugin to scrub them, or leave body logging off.

Example

{
"name": "llm-observability",
"plugin_id": "llm-observability",
"scope": { "kind": "global" },
"config": { "log_request": false, "log_response": false }
}