Observability

TelemetryHooks instruments every agent run with OpenTelemetry spans. Send them to any OTLP backend (Jaeger, Honeycomb, Grafana Tempo, Datadog) or to the console by passing a TelemetryHooks instance to AgentConfig.

Quick start

from cyclops import Agent, AgentConfig, TelemetryHooks

agent = Agent(AgentConfig(model="groq/llama-3.1-8b-instant", hooks=TelemetryHooks.console()))
agent.run("Summarise the Pythagorean theorem.")

TelemetryHooks.console() builds its own TracerProvider with a ConsoleSpanExporter. It does not touch the global tracer provider. Spans are exported in batches; call hooks.flush() when you need them out immediately (the provider also flushes at interpreter exit).

Your own tracer provider

Pass the provider your application already configures. Without one, spans go to the global provider.

from opentelemetry.sdk.trace import TracerProvider

provider = TracerProvider()
# provider.add_span_processor(...)
hooks = TelemetryHooks(tracer_provider=provider)

Flushing and shutdown stay with your application.

Concurrent runs

One TelemetryHooks instance can serve many runs at once, from threads or asyncio tasks. Span state is kept in a ContextVar, so each run gets its own root span and parallel tool calls in one run each get their own span. A run started while an OpenTelemetry span is current (for example inside a request handler, or an agent called from another agent’s tool) becomes a child of that span. This holds for run() and stream() too: the sync methods carry the caller’s context onto the loop that runs them.

A run that raises or is cancelled ends its span with status ERROR, together with any LLM or tool span it cut short. A stream the consumer closes early ends them without an error status. An LLM error the agent recovers from (a model that rejects native tool calls, retried in naive mode) fails only that chat span.

Span hierarchy

Each agent.run() call produces a root span with nested children:

invoke_agent
├── chat
├── execute_tool <name>
├── chat
└── execute_tool <name>

Span names and gen_ai.* attributes follow the OpenTelemetry GenAI semantic conventions in part: the hooks do not receive the request model or the provider, so gen_ai.request.model and gen_ai.provider.name are not set yet. While a tool runs, its span is the current span, so spans from the tool’s own code nest under it.

Span attributes

Span	Attribute	Description
all	gen_ai.operation.name	invoke_agent, chat or execute_tool
invoke_agent	cyclops.input.length	Character length of the input message
invoke_agent	cyclops.output.length	Character length of the final response
chat	cyclops.llm.message_count	Number of messages sent to the LLM
chat	gen_ai.response.model	Model identifier returned by the API
chat	gen_ai.response.id	Response id returned by the API
chat	gen_ai.response.finish_reasons	Finish reason of each choice
chat	gen_ai.usage.input_tokens	Prompt token count
chat	gen_ai.usage.output_tokens	Completion token count
chat	gen_ai.usage.cache_read.input_tokens	Prompt tokens read from the provider’s cache, when reported
chat	gen_ai.usage.cache_write.input_tokens	Prompt tokens written to the provider’s cache, when reported
chat	gen_ai.usage.reasoning.output_tokens	Completion tokens spent reasoning, when reported
execute_tool <name>	gen_ai.tool.name	Tool name
execute_tool <name>	gen_ai.tool.call.id	Tool call id (native tool calls only)
execute_tool <name>	cyclops.tool.args_count	Number of arguments passed
execute_tool <name>	cyclops.tool.result.length	Character length of the tool result
any failed span	error.type	Exception class name; the span status is ERROR and the exception is recorded as an event

Latency is the span duration.

Send to Jaeger (OTLP)

from cyclops import Agent, AgentConfig, TelemetryHooks

hooks = TelemetryHooks.otlp("http://localhost:4317")
agent = Agent(AgentConfig(model="groq/llama-3.1-8b-instant", hooks=hooks))
agent.run("Summarise the Pythagorean theorem.")
hooks.flush()

Install the OTLP exporter first:

uv add opentelemetry-exporter-otlp-proto-grpc

Start Jaeger locally:

docker run -d --name jaeger \
  -p 16686:16686 \
  -p 4317:4317 \
  jaegertracing/all-in-one:latest

Then open http://localhost:16686 and search for service cyclops.

Full example

examples/otel_example.py

"""OpenTelemetry observability example.

Demonstrates wiring TelemetryHooks to an agent and emitting spans to the console.
Use TelemetryHooks.otlp("http://localhost:4317") to send to Jaeger, Honeycomb,
Grafana Tempo, Datadog, or any OTLP-compatible backend.

Span hierarchy per agent.run(), following the OpenTelemetry GenAI conventions:
    invoke_agent
    ├── chat                   (gen_ai.response.model, gen_ai.usage.*)
    ├── execute_tool <name>    (gen_ai.tool.name)
    └── ...
"""

from cyclops import Agent, AgentConfig, TelemetryHooks
from cyclops.toolkit import tool

MODEL = "ollama/qwen3:4b"

# Alternatives:
#   MODEL = "groq/llama-3.1-8b-instant"              # GROQ_API_KEY
#   MODEL = "anthropic/claude-haiku-4-5-20251001"    # ANTHROPIC_API_KEY
#   MODEL = "gpt-4o-mini"                            # OPENAI_API_KEY


# ---------------------------------------------------------------------------
# 2. Tools
# ---------------------------------------------------------------------------


@tool
def get_weather(location: str) -> str:
    """Get the current weather for a location."""
    return f"Sunny, 22C in {location}"


@tool
def calculate(expression: str) -> str:
    """Evaluate a simple arithmetic expression."""
    try:
        return str(eval(expression))  # noqa: S307 (demo only)
    except Exception as exc:
        return f"Error: {exc}"


# ---------------------------------------------------------------------------
# 3. Run agent with TelemetryHooks
# ---------------------------------------------------------------------------


def main() -> None:
    # TelemetryHooks.console() builds its own TracerProvider with a ConsoleSpanExporter.
    # Pass TelemetryHooks(tracer_provider=...) to use the provider your app configures.
    # Swap for TelemetryHooks.otlp("http://localhost:4317") to send to Jaeger/Tempo/etc.
    hooks = TelemetryHooks.console()

    agent = Agent(
        AgentConfig(model=MODEL, hooks=hooks),
        tools=[get_weather, calculate],
    )

    print("Running agent...\n")
    agent.run("What is the weather in Tokyo, and what is 42 multiplied by 17?")
    hooks.flush()
    print("\nDone. Spans printed above.")


if __name__ == "__main__":
    main()