Traces

A trace represents a single end-to-end request through your AI agent. It contains one or more spans — individual operations captured by auto-instrumentation or custom decorators.

---

Trace Structure

Trace: support-bot / req-123
  ├── coalex_context (ROOT)
  │   ├── knowledge_base (RETRIEVER)
  │   │     query: "What is the deductible?"
  │   │     documents: 3
  │   ├── ChatOpenAI (LLM)
  │   │     model: gpt-4o
  │   │     tokens_in: 512, tokens_out: 87
  │   │     latency: 1.2s
  │   └── validate_output (GUARDRAIL)
  │         passed: true
  └── evaluate (internal)
        risk_score: 0.12
        status: auto_approved

Each trace has:

• Trace ID — Unique identifier for the entire request
• Agent ID — The agent that produced the trace (from coalex_context)
• Request ID — Application-level identifier (from coalex_context)
• Spans — Ordered list of operations with parent-child relationships

---

Status

A trace has the worst status of its spans: error when any span ended in error, else ok when any span ended ok, else unset. The parent of a span does not matter, so a trace whose root closed ok before a child failed is error, and a trace whose root was never exported (an agent hosted behind another service) takes its status from the spans it has. An approval wait ended by a cancellation and a guardrail that blocks by raising are error.

A request has the worst status of its traces by the same order. error_count is the number of spans in error, and root_span_name is the earliest coalex.invocation span whose parent is not stored, or the oldest span whose parent is not stored when there is none.

Status is recomputed whenever a span of the trace arrives. A stored trace keeps the status it was given until then; older traces are not recomputed.

---

Span Types

Span Kind	Source	Description
LLM	Auto-instrumentation	LLM completion calls (OpenAI, Anthropic, etc.)
CHAIN	Auto-instrumentation	LangChain/LlamaIndex chain or pipeline steps
RETRIEVER	Auto or @retrieval_span	Document retrieval operations
EMBEDDING	Auto or @embedding_span	Embedding generation
RERANKER	@reranker_span	Document reranking
TOOL	Auto or @tool_span	External tool or API calls
GUARDRAIL	@guardrail_span	Input/output validation checks

---

Span Attributes

All spans include standard OpenTelemetry attributes plus OpenInference-specific attributes:

Attribute	Description
openinference.span.kind	Span type (LLM, CHAIN, RETRIEVER, etc.)
input.value	Input to the operation (prompt, query, etc.)
output.value	Output from the operation (response, documents, etc.)
llm.model_name	LLM model identifier
llm.token_count.prompt	Input token count
llm.token_count.completion	Output token count
llm.token_count.total	Total token count

---

Viewing Traces

Dashboard

Open the admin dashboard and navigate to Traces. You can filter by:

• Agent ID
• Time range
• Status (ok, error, unset)
• Model name
• Minimum latency

API

curl https://traces.coalex.ai/v1/traces \
  -H "Authorization: Bearer $COALEX_API_KEY"

DuckLake (Direct Query)

SELECT trace_id, agent_id, span_count, duration_ms
FROM silver_traces
WHERE agent_id = 'support-bot'
ORDER BY ingested_at DESC
LIMIT 10;

---

Enrichments

The Coalex Transformer automatically enriches traces with:

Enrichment	Description
Token cost	Estimated USD cost per LLM call based on model pricing
PII detection	Flags spans containing personally identifiable information
Sustainability	Energy consumption (kWh), carbon footprint (kgCO2eq)
Latency percentiles	P50, P95, P99 latency per agent