Troubleshooting Guide

Solutions for common Headroom issues.

---

Proxy Server Issues

"Proxy won't start"

Symptom: headroom proxy fails or hangs.

Solutions:

# 1. Check if port is already in use
lsof -i :8787
# If something is using the port, either kill it or use a different port

# 2. Try a different port
headroom proxy --port 8788

# 3. Check for missing dependencies
pip install "headroom[proxy]"

# 4. Run with debug logging
headroom proxy --log-level debug

"Connection refused" when calling proxy

Symptom: curl: (7) Failed to connect to localhost port 8787

Solutions:

# 1. Verify proxy is running
curl http://localhost:8787/health

# 2. Check if proxy started on a different port
ps aux | grep headroom

# 3. Check firewall settings (macOS)
sudo pfctl -s rules | grep 8787

"Proxy returns errors for some requests"

Symptom: Some requests work, others fail with 502/503.

Solutions:

# 1. Check proxy logs for the actual error
headroom proxy --log-level debug

# 2. Verify API key is set
echo $OPENAI_API_KEY  # or ANTHROPIC_API_KEY

# 3. Test the underlying API directly
curl https://api.openai.com/v1/models -H "Authorization: Bearer $OPENAI_API_KEY"

---

SDK Issues

"No token savings"

Symptom: stats['session']['tokens_saved_total'] is 0.

Diagnosis:

# 1. Check mode
stats = client.get_stats()
print(f"Mode: {stats['config']['mode']}")  # Should be "optimize"

# 2. Check transforms are enabled
print(f"SmartCrusher: {stats['transforms']['smart_crusher_enabled']}")

# 3. Check if content meets threshold
# SmartCrusher only compresses tool outputs > 200 tokens by default

Solutions:

# 1. Ensure mode is "optimize"
client = HeadroomClient(
    original_client=OpenAI(),
    provider=OpenAIProvider(),
    default_mode="optimize",  # NOT "audit"
)

# 2. Or override per-request
response = client.chat.completions.create(
    model="gpt-4o",
    messages=messages,
    headroom_mode="optimize",
)

# 3. Lower the compression threshold
config = HeadroomConfig()
config.smart_crusher.min_tokens_to_crush = 100  # Default is 200

Why It Might Be 0: - Mode is "audit" (observation only) - Messages don't contain tool outputs - Tool outputs are below the token threshold - Data isn't compressible (high uniqueness)

"Compression too aggressive"

Symptom: LLM responses are missing information that was in tool outputs.

Solutions:

# 1. Keep more items
config = HeadroomConfig()
config.smart_crusher.max_items_after_crush = 50  # Default: 15

# 2. Skip compression for specific tools
response = client.chat.completions.create(
    model="gpt-4o",
    messages=messages,
    headroom_tool_profiles={
        "important_tool": {"skip_compression": True},
    },
)

# 3. Disable SmartCrusher entirely
config.smart_crusher.enabled = False

"High latency"

Symptom: Requests take longer than expected.

Diagnosis:

import time
import logging

logging.basicConfig(level=logging.DEBUG)

start = time.time()
response = client.chat.completions.create(...)
print(f"Total time: {time.time() - start:.2f}s")

# Check logs for:
# - "SmartCrusher" timing
# - "EmbeddingScorer" timing (slow if using embeddings)

Solutions:

# 1. Use BM25 instead of embeddings (faster)
config = HeadroomConfig()
config.smart_crusher.relevance.tier = "bm25"  # Default may use embeddings

# 2. Increase threshold to skip small payloads
config.smart_crusher.min_tokens_to_crush = 500

# 3. Disable transforms you don't need
config.cache_aligner.enabled = False
config.rolling_window.enabled = False

"ValidationError on setup"

Symptom: validate_setup() returns errors.

Common Issues:

result = client.validate_setup()
print(result)

# Provider error:
# {"provider": {"ok": False, "error": "No API key"}}
# → Set OPENAI_API_KEY or pass api_key to OpenAI()

# Storage error:
# {"storage": {"ok": False, "error": "unable to open database"}}
# → Check path permissions, use :memory: for testing

# Config error:
# {"config": {"ok": False, "error": "Invalid mode"}}
# → Use "audit" or "optimize" only

Solutions:

# 1. For testing, use in-memory storage
client = HeadroomClient(
    original_client=OpenAI(),
    provider=OpenAIProvider(),
    store_url="sqlite:///:memory:",  # No file created
)

# 2. For temp directory storage
import tempfile
import os
db_path = os.path.join(tempfile.gettempdir(), "headroom.db")
client = HeadroomClient(
    original_client=OpenAI(),
    provider=OpenAIProvider(),
    store_url=f"sqlite:///{db_path}",
)

---

Import/Installation Issues

"ModuleNotFoundError: No module named 'headroom'"

# 1. Check it's installed in the right environment
pip show headroom

# 2. If using virtual environment, ensure it's activated
source venv/bin/activate  # or equivalent

# 3. Reinstall
pip install --upgrade headroom

"ImportError: cannot import name 'X' from 'headroom'"

# Check available imports
import headroom
print(dir(headroom))

# Common imports:
from headroom import (
    HeadroomClient,
    OpenAIProvider,
    AnthropicProvider,
    HeadroomConfig,
    # Exceptions
    HeadroomError,
    ConfigurationError,
    ProviderError,
)

"Missing optional dependency"

# For proxy server
pip install "headroom[proxy]"

# For embedding-based relevance scoring
pip install "headroom[relevance]"

# For everything
pip install "headroom[all]"

---

Provider-Specific Issues

OpenAI: "Invalid API key"

from openai import OpenAI
import os

# Ensure key is set
api_key = os.environ.get("OPENAI_API_KEY")
if not api_key:
    raise ValueError("OPENAI_API_KEY not set")

client = HeadroomClient(
    original_client=OpenAI(api_key=api_key),
    provider=OpenAIProvider(),
)

Anthropic: "Authentication error"

from anthropic import Anthropic
import os

api_key = os.environ.get("ANTHROPIC_API_KEY")
client = HeadroomClient(
    original_client=Anthropic(api_key=api_key),
    provider=AnthropicProvider(),
)

"Unknown model" warnings

# For custom/fine-tuned models, specify context limit
client = HeadroomClient(
    original_client=OpenAI(),
    provider=OpenAIProvider(),
    model_context_limits={
        "ft:gpt-4o-2024-08-06:my-org::abc123": 128000,
        "my-custom-model": 32000,
    },
)

---

Debugging Techniques

Enable Full Logging

import logging

# See everything
logging.basicConfig(
    level=logging.DEBUG,
    format="%(asctime)s %(name)s %(levelname)s %(message)s",
)

# Or just Headroom logs
logging.getLogger("headroom").setLevel(logging.DEBUG)

Inspect Transform Results

# Use simulate to see what would happen
plan = client.chat.completions.simulate(
    model="gpt-4o",
    messages=messages,
)

print(f"Tokens: {plan.tokens_before} -> {plan.tokens_after}")
print(f"Transforms: {plan.transforms}")
print(f"Waste signals: {plan.waste_signals}")

# See the actual optimized messages
import json
print(json.dumps(plan.messages_optimized, indent=2))

Check Storage Contents

from datetime import datetime, timedelta

# Get recent metrics
metrics = client.get_metrics(
    start_time=datetime.utcnow() - timedelta(hours=1),
    limit=10,
)

for m in metrics:
    print(f"{m.timestamp}: {m.tokens_input_before} -> {m.tokens_input_after}")
    print(f"  Transforms: {m.transforms_applied}")
    if m.error:
        print(f"  ERROR: {m.error}")

Manual Transform Testing

from headroom import SmartCrusher, Tokenizer
from headroom.config import SmartCrusherConfig
import json

# Test compression directly
config = SmartCrusherConfig()
crusher = SmartCrusher(config)
tokenizer = Tokenizer()

messages = [
    {"role": "tool", "content": json.dumps({"items": list(range(100))}), "tool_call_id": "1"}
]

result = crusher.apply(messages, tokenizer)
print(f"Tokens: {result.tokens_before} -> {result.tokens_after}")
print(f"Compressed content: {result.messages[0]['content'][:200]}...")

---

Error Reference

Exception	Meaning	Solution
ConfigurationError	Invalid config values	Check config parameters
ProviderError	Provider issue (unknown model, etc.)	Set model_context_limits
StorageError	Database issue	Check path/permissions
CompressionError	Compression failed	Rare - check data format
TokenizationError	Token counting failed	Check model name
ValidationError	Setup validation failed	Run validate_setup()

Handling Errors

from headroom import (
    HeadroomClient,
    HeadroomError,
    ConfigurationError,
    StorageError,
)

try:
    client = HeadroomClient(...)
    response = client.chat.completions.create(...)
except ConfigurationError as e:
    print(f"Config issue: {e}")
    print(f"Details: {e.details}")
except StorageError as e:
    print(f"Storage issue: {e}")
    # Headroom continues to work, just without metrics persistence
except HeadroomError as e:
    print(f"Headroom error: {e}")

---

Getting Help

1. Enable debug logging and check the output
2. Use simulate() to see what transforms would apply
3. Check validate_setup() for configuration issues
4. File an issue at https://github.com/headroom-sdk/headroom/issues

When filing an issue, include: - Headroom version (pip show headroom) - Python version - Provider (OpenAI/Anthropic) - Debug log output - Minimal reproduction code