Live LabsFree Live LabEngineersObservability1 hour

Build an AI Agent Dashboard with Langfuse

Once agents run on their own, the question is what they did while nobody was watching. Most teams find out from a user's complaint or the month's invoice. Live, Param traces every agent run in Langfuse, builds a dashboard of cost, latency and failures per agent and per customer, and alerts on Telegram when one goes over budget.

Why this matters

Once agents run on their own, the question becomes what they did while nobody was watching. Most teams find out from a user complaint or the month's invoice. Tracing every run and putting cost, latency and failures on one dashboard per agent and per customer turns those surprises into alerts you see first.

What happens in the hour

  1. The problem4 min

    Why most teams learn what their agents did from a complaint or the invoice.

  2. Built live40 min

    Each model call, tool call and approval in one trace; a dashboard per agent and per customer; a budget per agent that pings Telegram.

  3. Questions16 min

    The three numbers to watch on your own agents from day one.

What you will be able to do

Trace every step
Each model call, tool call and approval in one trace, linked to its user.
Track cost per agent
See which agent, and which customer, costs what.
Alert before the budget
A limit per agent that pings you on Telegram.

The ideas behind it

  • A trace per run

    Every model call and tool call in one run, in order, with its input, output, time and cost.

  • Cost per agent and per customer

    Spend grouped the way your business needs it, so you can see which agent or customer drives the bill.

  • Failure rates

    Errors, timeouts and retries per agent, so a broken tool shows up as a line going up.

  • Budget alerts

    A message on Telegram when an agent passes its budget, before the invoice does it for you.

Where it fits

The parts of a production agent system, in the order the buildcamp builds them. The lit tiles are the ones this live lab builds; the Agentic AI Buildcamp for Engineers builds all of them.

Week 1Architect
  • Roles and modelsOne job per agent, a model chosen for that job, and a token budget.
  • OrchestrationA queue agents pick work from, with hand-offs a person can follow.
  • Context and memoryRetrieval with sources, memory across sessions, prompts laid out for the cache.
  • Tool callingTyped tools that act in GitHub, Linear and Google.
Week 2Build
  • MCP serversEvery tool behind an MCP gateway, scoped to the role that needs it.
  • Durable executionRuns that resume after a crash and never repeat a write.
  • Human in the loopA person approves anything that cannot be undone.
Week 3Secure and deploy
  • Identity and permissionsEach agent signs in as itself and acts on behalf of a user.
  • LLM gatewayRouting, caching and a budget on every model call.
  • Traces, evals and costEvery run traced, scored and charged to the agent that made it.
  • Multi-tenancyEach team or customer kept apart, in data and in the bill.
Week 4Govern and extend
  • Policy as codeRules checked on every action, not written in a document.
  • Signed skillsNew roles built from reviewed, signed skills.

Before you come

For
Engineers running agents in production with no view of what they do, and the leads who answer for agent cost and reliability.
You need
None. Code is shown, not required.
Track
Agentic AI for engineers who build and run it.

Questions

Is Langfuse free?

Langfuse is open source and can be self-hosted, and it also has a hosted cloud with a free tier.

Does tracing slow agents down?

Traces are sent in the background, so the effect on response time is small.

Can we use another tool?

Yes. The dashboard ideas carry over to any tracing tool that supports OpenTelemetry.

Is this for engineers or managers?

Engineers build it; engineering managers get the view they need for cost and reliability reviews.

Is the Live Lab really free?

Yes. Live Labs are free on Maven. You sign up with your email and get the join link and the recording.

What if I cannot make it live?

Sign up anyway. Everyone who signs up gets the recording, so you can watch the build later and reply with questions.

Do I need to code along?

No. Most people watch the build and ask questions. Every step is shown, so you can repeat it on your own afterwards.

Go deeper

WorkshopDeploy Agentic AI Systems for EngineersA multi-agent system deployed with an identity per agent, permissions, guardrails, an LLM gateway, traces and evals, plus a checklist for your own agents.$297

Keep exploring