Traces

A trace represents a single end-to-end request through your AI agent. It contains one or more spans — individual operations captured by auto-instrumentation or custom decorators.

---

Trace Structure

Trace: support-bot / req-123
  ├── coalex_context (ROOT)
  │   ├── knowledge_base (RETRIEVER)
  │   │     query: "What is the deductible?"
  │   │     documents: 3
  │   ├── ChatOpenAI (LLM)
  │   │     model: gpt-4o
  │   │     tokens_in: 512, tokens_out: 87
  │   │     latency: 1.2s
  │   └── validate_output (GUARDRAIL)
  │         passed: true
  └── evaluate (internal)
        risk_score: 0.12
        status: auto_approved

Each trace has:

• Trace ID — Unique identifier for the entire request
• Agent ID — The agent that produced the trace (from coalex_context)
• Request ID — Application-level identifier (from coalex_context)
• Spans — Ordered list of operations with parent-child relationships

---

Span Types

Span Kind	Source	Description
LLM	Auto-instrumentation	LLM completion calls (OpenAI, Anthropic, etc.)
CHAIN	Auto-instrumentation	LangChain/LlamaIndex chain or pipeline steps
RETRIEVER	Auto or @retrieval_span	Document retrieval operations
EMBEDDING	Auto or @embedding_span	Embedding generation
RERANKER	@reranker_span	Document reranking
TOOL	Auto or @tool_span	External tool or API calls
GUARDRAIL	@guardrail_span	Input/output validation checks

---

Span Attributes

All spans include standard OpenTelemetry attributes plus OpenInference-specific attributes:

Attribute	Description
openinference.span.kind	Span type (LLM, CHAIN, RETRIEVER, etc.)
input.value	Input to the operation (prompt, query, etc.)
output.value	Output from the operation (response, documents, etc.)
llm.model_name	LLM model identifier
llm.token_count.prompt	Input token count
llm.token_count.completion	Output token count
llm.token_count.total	Total token count

---

Viewing Traces

Dashboard

Open the admin dashboard and navigate to Traces. You can filter by:

• Agent ID
• Time range
• Status (success, error)
• Model name
• Minimum latency

API

curl https://your-org.coalex.ai/v1/traces \
  -H "Authorization: Bearer $COALEX_API_KEY"

DuckLake (Direct Query)

SELECT trace_id, agent_id, span_count, duration_ms
FROM silver_traces
WHERE agent_id = 'support-bot'
ORDER BY ingested_at DESC
LIMIT 10;

---

Enrichments

The Coalex Transformer automatically enriches traces with:

Enrichment	Description
Token cost	Estimated USD cost per LLM call based on model pricing
PII detection	Flags spans containing personally identifiable information
Sustainability	Energy consumption (kWh), carbon footprint (kgCO2eq)
Latency percentiles	P50, P95, P99 latency per agent