Langfuse

LLM observability platform for tracing and debugging AI agents.

---

Overview

Langfuse provides end-to-end observability for LLM applications.

Property	Value
Image	langfuse/langfuse:3
Container	langfuse-web
URL	langfuse.ravenhelm.dev
Port	3000 (internal)
Config	~/ravenhelm/services/langfuse/

---

Architecture

┌─────────────────────────────────────────────────────────────────┐
│                          LANGFUSE                                │
├─────────────────────────────────────────────────────────────────┤
│                                                                  │
│   ┌─────────────────┐              ┌─────────────────┐          │
│   │  langfuse-web   │◀────────────▶│ langfuse-worker │          │
│   │   (UI + API)    │              │  (Background)   │          │
│   └────────┬────────┘              └────────┬────────┘          │
│            │                                │                    │
│            ▼                                ▼                    │
│   ┌─────────────────┐              ┌─────────────────┐          │
│   │   PostgreSQL    │              │   ClickHouse    │          │
│   │  (Metadata)     │              │  (Analytics)    │          │
│   └─────────────────┘              └─────────────────┘          │
│                                            │                     │
│                                            ▼                     │
│                                    ┌─────────────────┐          │
│                                    │      MinIO      │          │
│                                    │ (Object Store)  │          │
│                                    └─────────────────┘          │
└─────────────────────────────────────────────────────────────────┘

---

Components

Container	Image	Port	Purpose
langfuse-web	langfuse/langfuse:3	3000	Web UI and REST API
langfuse-worker	langfuse/langfuse:3	-	Background job processing
langfuse-clickhouse	clickhouse/clickhouse-server:24	8123	Analytics OLAP database
langfuse-minio	minio/minio:latest	9000/9001	S3-compatible object storage

ClickHouse

High-performance analytics database for trace storage and querying.

• Purpose: Stores trace data, spans, generations for fast analytics
• Port: 8123 (HTTP), 9000 (native)
• Data: /Users/ravenhelm/ravenhelm/data/langfuse/clickhouse/

MinIO

S3-compatible object storage for large payloads.

• Purpose: Stores large inputs/outputs, media files, prompt templates
• Ports: 9000 (API), 9001 (Console)
• Data: /Users/ravenhelm/ravenhelm/data/langfuse/minio/

---

Features

Feature	Description
Traces	End-to-end request traces
Spans	Individual LLM calls
Scores	Quality metrics
Prompts	Prompt management
Datasets	Evaluation datasets

---

Norns Integration

Environment Variables

Norns requires these environment variables in docker-compose.yml:

# Langfuse Observability
- LANGFUSE_PUBLIC_KEY=${LANGFUSE_PUBLIC_KEY}
- LANGFUSE_SECRET_KEY=${LANGFUSE_SECRET_KEY}
- LANGFUSE_HOST=https://langfuse.ravenhelm.dev

Important: Use the external URL (https://langfuse.ravenhelm.dev) for LANGFUSE_HOST, not the internal Docker URL. This ensures "Open in Langfuse" links work correctly from the admin UI.

Secrets

Add to /Users/ravenhelm/ravenhelm/secrets/.env:

# Langfuse Observability
LANGFUSE_SECRET_KEY=sk-lf-...
LANGFUSE_PUBLIC_KEY=pk-lf-...

Get these from the Langfuse UI: Settings > API Keys.

LangChain Callback Handler

Norns uses the LangChain callback handler for automatic tracing:

from langfuse.langchain import CallbackHandler as LangfuseCallbackHandler

def get_langfuse_handler():
    """Get a Langfuse callback handler for tracing."""
    try:
        return LangfuseCallbackHandler()
    except Exception as e:
        logger.warning(f"Failed to initialize Langfuse: {e}")
        return None

# Use in LangChain invocations
langfuse_handler = get_langfuse_handler()
if langfuse_handler:
    config = {
        "callbacks": [langfuse_handler],
        "metadata": {
            "langfuse_session_id": session_id,
            "langfuse_user_id": slack_user_id,
        }
    }
response = chain.invoke({"messages": messages}, config=config)

Flushing Traces

Critical: Langfuse v3 buffers traces and requires explicit flushing. Without this, traces may not appear in the dashboard.

from langfuse import Langfuse

# After processing a message, flush traces
try:
    langfuse = Langfuse()
    langfuse.flush()
except Exception as e:
    logger.debug(f"Failed to flush Langfuse traces: {e}")

This is already implemented in graph.py:process_message().

---

Quick Commands

# View logs
docker logs -f langfuse-web

# Restart all components
docker restart langfuse-web langfuse-worker langfuse-clickhouse langfuse-minio

# Check health
curl https://langfuse.ravenhelm.dev/api/public/health

# Check ClickHouse
docker exec langfuse-clickhouse clickhouse-client --query "SELECT count() FROM traces"

# Check MinIO
docker exec langfuse-minio mc admin info local

# Test trace creation from Norns container
docker exec norns-agent python3 -c "
from langfuse import Langfuse
langfuse = Langfuse()
with langfuse.start_as_current_span(name='test-span') as span:
    print('Span created')
langfuse.flush()
print('Test trace sent')
"

---

Integration (Generic)

Add to Python services:

from langfuse import Langfuse

langfuse = Langfuse(
    public_key=os.getenv("LANGFUSE_PUBLIC_KEY"),
    secret_key=os.getenv("LANGFUSE_SECRET_KEY"),
    host=os.getenv("LANGFUSE_HOST")
)

# Create trace
with langfuse.start_as_current_span(name="my-operation") as span:
    # Do work
    result = process_something()

# Always flush at the end
langfuse.flush()

---

Authentication

Uses Zitadel SSO via custom OIDC provider.

---

Troubleshooting

Issue: Traces Not Appearing

Symptoms: No traces visible in Langfuse dashboard after sending messages to Norns.

Diagnosis:

# 1. Check API keys are set in container
docker exec norns-agent env | grep LANGFUSE

# Expected output:
# LANGFUSE_HOST=https://langfuse.ravenhelm.dev
# LANGFUSE_PUBLIC_KEY=pk-lf-...
# LANGFUSE_SECRET_KEY=sk-lf-...

# 2. Check connectivity from container
docker exec norns-agent python3 -c "
import requests, os
r = requests.get(f'{os.getenv(\"LANGFUSE_HOST\")}/api/public/health')
print(r.status_code, r.text)
"

# 3. Check for authentication errors in logs
docker logs norns-agent 2>&1 | grep -i langfuse

# 4. Verify flush is being called
docker exec norns-agent grep -A3 'Flush Langfuse' /app/graph.py

Common Causes:

Cause	Solution
Missing env vars	Add LANGFUSE_* to docker-compose.yml
Wrong LANGFUSE_HOST	Use https://langfuse.ravenhelm.dev (external URL)
No flush() call	Add langfuse.flush() after processing
Auth error	Regenerate API keys in Langfuse UI

Issue: "Open in Langfuse" Links Broken

Symptoms: Links point to http://langfuse-web:3000/ instead of external URL.

Solution: Change LANGFUSE_HOST from internal Docker URL to external URL:

# Wrong (internal)
- LANGFUSE_HOST=http://langfuse-web:3000

# Correct (external)
- LANGFUSE_HOST=https://langfuse.ravenhelm.dev

Issue: Worker Not Processing

Symptoms: Traces appear but metrics/scores not computed.

Diagnosis:

docker logs langfuse-worker | tail -50

Solutions:

1. Restart worker: docker restart langfuse-worker
2. Check ClickHouse connectivity
3. Verify Redis connection (if configured)

---

See Also

• View LLM Traces - Debugging AI agents
• LangGraph - Agent workflows
• Norns Agent - Main agent
• Norns Knowledge Domain - Knowledge tools