Token, cost, and latency attributes

Span kinds and names give you shape. Numbers give you decisions. Tokens and cost tell you which tool drives the bill. Duration tells you which span blocks the user. These three attributes belong on every tool span.

observability.py (pricing table)
python
MODEL_PRICING = {
    "gpt-4o":         {"input": 2.50,  "output": 10.00},
    "gpt-4o-mini":    {"input": 0.15,  "output": 0.60},
    "gpt-4-turbo":    {"input": 10.00, "output": 30.00},
    "text-embedding-3-small": {"input": 0.02, "output": 0.00},
}

def estimate_tokens(text: str) -> int:
    return 0 if not text else len(text) // 4

def calculate_cost(model: str, input_tokens: int, output_tokens: int) -> float:
    pricing = MODEL_PRICING.get(model, {"input": 0.15, "output": 0.60})
    input_cost  = (input_tokens  / 1_000_000) * pricing["input"]
    output_cost = (output_tokens / 1_000_000) * pricing["output"]
    return round(input_cost + output_cost, 6)

A tiny pricing table is enough. Rough token estimates turn into real cost numbers that sort nicely in Phoenix aggregates.

observability.py (span attributes)
python
from opentelemetry import trace

def set_llm_usage(span, model: str, input_text: str, output_text: str) -> None:
    input_tokens  = estimate_tokens(input_text)
    output_tokens = estimate_tokens(output_text)
    span.set_attribute("llm.token_count.prompt",     input_tokens)
    span.set_attribute("llm.token_count.completion", output_tokens)
    span.set_attribute("llm.token_count.total",      input_tokens + output_tokens)
    span.set_attribute("cost.usd",                   calculate_cost(model, input_tokens, output_tokens))

Use the OpenInference attribute names so Phoenix renders them in the usage panel. cost.usd is a custom attribute you can aggregate yourself.

Matching exercise: Which attribute answers which question

Loading practice…

Quiz: Quiz

Loading practice…