Skip to content

Instrumentation

Coalex captures AI agent telemetry through two mechanisms: auto-instrumentation for supported LLM frameworks and extension decorators for custom pipeline steps.

Auto-Instrumentation

Call auto_instrument() once after register() to automatically patch all installed LLM libraries:

results = coalex.auto_instrument()
# {"openai": "success", "langchain": "success", "llamaindex": "not_installed", ...}

const results = autoInstrument();
// { openai: "success", langchain: "success", llamaindex: "not_installed", ... }

Supported Frameworks

Framework Python TypeScript
OpenAI openai openai
Anthropic anthropic @anthropic-ai/sdk
Google Generative AI google-generativeai @google/generative-ai
LangChain langchain @langchain/*
LlamaIndex llama-index llamaindex
CrewAI crewai —
Haystack haystack-ai —
Guardrails AI guardrails-ai —
Instructor instructor —
Mistral AI mistralai —
Amazon Bedrock boto3 —
Groq groq —
LiteLLM litellm —
DSPy dspy —
Vertex AI vertexai —

What Gets Captured

For each LLM call, auto-instrumentation captures:

  • Model name — e.g., gpt-4o, claude-3-5-sonnet-20241022
  • Input messages — full prompt content
  • Output messages — full response content
  • Token counts — input, output, and total tokens
  • Latency — span duration in milliseconds
  • Chain structure — parent-child relationships for LangChain/LlamaIndex pipelines

Extension Decorators

For custom pipeline steps that aren't covered by auto-instrumentation, use extension decorators:

Decorator Span Kind Use Case
@retrieval_span RETRIEVER Custom vector search, database queries
@embedding_span EMBEDDING Custom embedding generation
@reranker_span RERANKER Custom document reranking
@tool_span TOOL External API calls, calculations
@guardrail_span GUARDRAIL Input/output validation

Example:

from coalex.ext.retrieval import retrieval_span, Document

@retrieval_span(name="knowledge_base", query_arg="query")
def search(query: str) -> list[Document]:
    results = my_vector_db.search(query, top_k=5)
    return [Document(content=r.text, id=r.id, score=r.score) for r in results]

See Extensions for the full reference.

Instrumentation Architecture

graph TD
    A[auto_instrument] --> B[OpenInference Instrumentors]
    B --> C[Monkey-patch LLM libraries]
    C --> D[Emit OpenTelemetry Spans]
    D --> E[OTLP Exporter]
    E --> F[Coalex Collector]

    G[Extension Decorators] --> D

All spans — whether from auto-instrumentation or custom decorators — flow through the same OpenTelemetry pipeline to the Coalex Collector.