❍ Forensic Observability for AI Agents

Your agent finished.
But did it actually work?

ARGUS catches silent failures and traces root causes before you deploy — so broken pipelines never reach production.

$pip install argus-agents

Runs

549 runs · 363 clean · 186 failed
RUNSTATUSSTEPSDURATIONTOKENS
parse_diff → detect_risk → review_code → ch...
20260613-061921-080f8dparse_diff
Silent Failure
5/5
25.09s
4d ago
router → weather → general
20260617
Clean
3/3
2.23s
7h ago
clean → meta
20260617
Clean
2/2
3.43s
7h ago
process
20260617
Clean
1/1
1.68s
7h ago
respond → summarize
20260617
Clean
2/2
2.84s
7h ago
answer
20260617
Clean
1/1
1.33s
7h ago
step_1 → step_2 → step_3 → step_4 → step_5...
20260617
Clean
7/7
11.51s
7h ago
scan → report
20260617
Clean
2/2
2.96s
7h ago
compute
20260617
Clean
1/1
1.33s
7h ago

The internet has been complaining about this for years.
We're just building the fix.

our agent hallucinated product features for 3 days straight. responses looked perfectly formatted so nobody flagged it. we only found out when a customer asked about a feature that doesn't exist.

Rakesh Venkat
Rakesh Venkat
X · 2.1k likes

the planner agent was outputting YAML but the executor expected JSON. pipeline showed green. outputs were garbage for a week. logs said 200 OK the entire time.

marco.builds
marco.builds
r/MachineLearning · 890 upvotes

a tool API changed its response format quietly. our agent started retrying in a loop. $15k in extra LLM costs before anyone caught it. no alert, no error, just a slowly growing bill.

Ashwin Rao
Ashwin Rao
X · 4.7k likes

42% of teams had production incidents from hallucinations within 3 months of launch. if you're shipping agents without semantic validation, you're just waiting for your turn.

Sam Nakamura
Sam Nakamura
Reels · 312k views

our sales agent started offering unauthorized 50% discounts. worked perfectly in demos. connected to real customer data? just started making things up. cost us 3 enterprise deals.

Chris Lee
Chris Lee
X · 11.3k likes

multi-agent telephone game is real. step 2 misinterprets step 1, step 3 builds on the wrong interpretation, final output is logically corrupted but looks completely fine to reviewers.

jenny.data
jenny.data
Discord · pinned

our agent hallucinated product features for 3 days straight. responses looked perfectly formatted so nobody flagged it. we only found out when a customer asked about a feature that doesn't exist.

Rakesh Venkat
Rakesh Venkat
X · 2.1k likes

the planner agent was outputting YAML but the executor expected JSON. pipeline showed green. outputs were garbage for a week. logs said 200 OK the entire time.

marco.builds
marco.builds
r/MachineLearning · 890 upvotes

a tool API changed its response format quietly. our agent started retrying in a loop. $15k in extra LLM costs before anyone caught it. no alert, no error, just a slowly growing bill.

Ashwin Rao
Ashwin Rao
X · 4.7k likes

42% of teams had production incidents from hallucinations within 3 months of launch. if you're shipping agents without semantic validation, you're just waiting for your turn.

Sam Nakamura
Sam Nakamura
Reels · 312k views

our sales agent started offering unauthorized 50% discounts. worked perfectly in demos. connected to real customer data? just started making things up. cost us 3 enterprise deals.

Chris Lee
Chris Lee
X · 11.3k likes

multi-agent telephone game is real. step 2 misinterprets step 1, step 3 builds on the wrong interpretation, final output is logically corrupted but looks completely fine to reviewers.

jenny.data
jenny.data
Discord · pinned

an agent 'completed' a maintenance task by deleting 1,200 records and fabricating 4,000 fake ones to cover it. every dashboard showed green. we found out 3 weeks later from a customer report.

Kate Murphy
Kate Murphy
X · 18.2k likes

merge queue bug silently reverted commits across 600+ repos. the UI showed green checkmarks the entire time. devs had no idea their code was reverted until someone dug into the actual diffs.

yusuf.ai
yusuf.ai
Discord · #incidents

only 14% of enterprises with AI agent pilots have reached production scale. the gap isn't model quality — it's that nobody can tell when their agents silently degrade. observability is the actual bottleneck.

Amy Zhang
Amy Zhang
r/dataengineering · 2.4k upvotes

our agent drifted over 2 months. slowly gave worse outputs. nobody noticed because there were no errors — just gradually degrading quality. customers churned before we connected the dots.

Jordan Rivera
Jordan Rivera
X · 6.8k likes

POV: you analyze 12M agent logs and discover 78% of issues were silent regressions and hallucinations — not timeouts or errors. traditional APM catches literally none of this.

Marcus Johnson
Marcus Johnson
Reels · 189k views

our agent posted 47 near-identical messages to a public slack channel when an API call failed. no circuit breaker, no alert, no observability. just 47 messages and a very confused sales team.

Kara Okonkwo
Kara Okonkwo
r/devops · 1.8k upvotes

an agent 'completed' a maintenance task by deleting 1,200 records and fabricating 4,000 fake ones to cover it. every dashboard showed green. we found out 3 weeks later from a customer report.

Kate Murphy
Kate Murphy
X · 18.2k likes

merge queue bug silently reverted commits across 600+ repos. the UI showed green checkmarks the entire time. devs had no idea their code was reverted until someone dug into the actual diffs.

yusuf.ai
yusuf.ai
Discord · #incidents

only 14% of enterprises with AI agent pilots have reached production scale. the gap isn't model quality — it's that nobody can tell when their agents silently degrade. observability is the actual bottleneck.

Amy Zhang
Amy Zhang
r/dataengineering · 2.4k upvotes

our agent drifted over 2 months. slowly gave worse outputs. nobody noticed because there were no errors — just gradually degrading quality. customers churned before we connected the dots.

Jordan Rivera
Jordan Rivera
X · 6.8k likes

POV: you analyze 12M agent logs and discover 78% of issues were silent regressions and hallucinations — not timeouts or errors. traditional APM catches literally none of this.

Marcus Johnson
Marcus Johnson
Reels · 189k views

our agent posted 47 near-identical messages to a public slack channel when an API call failed. no circuit breaker, no alert, no observability. just 47 messages and a very confused sales team.

Kara Okonkwo
Kara Okonkwo
r/devops · 1.8k upvotes
▸ Watch ARGUS catch a failure

Every node, traced.
Every silence, surfaced.

ARGUS watches every step of your agent pipeline during development and testing,catching silent failures before they ever reach production.

extract128ms

Data extracted successfully

enrich218ms

Entities enriched and structured

summarize1.34s

Silent failure:
placeholder returned

Root Cause

ARGUS detected this before it degraded downstream

validate87ms

Degradation detected in downstream step

respond--

Output at risk

▸ Zero-friction setup

Let your AI set up Argus

Copy this prompt, paste it into Claude or ChatGPT, and it will audit your pipeline and wire up ARGUS automatically.

1Copy the prompt
2Paste into your AI
3Pipeline monitored
argus-setup.prompt141 lines
I want to add ARGUS monitoring to my LangGraph pipeline. Before writing any code, audit my codebase and then integrate it properly.
 
STEP 1 — AUDIT MY PIPELINE
 
Find the file where my StateGraph is defined and check these things:
 
1. STATE TYPE: Find my state class. ARGUS works best when state is a TypedDict (or Pydantic model / dataclass). If my state is just a plain dict, convert it to a TypedDict with proper field annotations. Example:
 
# BAD — plain dict, ARGUS can't check field contracts
app = graph.compile()
result = app.invoke({"query": "...", "results": []})
 
# GOOD — TypedDict lets ARGUS verify fields between nodes
class AgentState(TypedDict):
query: str
results: list[str]
summary: str
 
2. NODE RETURN TYPES: Check every node function. Each should:
- Accept the state type as its first parameter (type-annotated)
- Return a dict with only the fields it's responsible for
- NOT return the entire state — just the fields it modifies
 
# BAD — no type hint, returns everything
def search(state):
return {**state, "results": [...]}
 
# GOOD — typed, returns only what it produces
def search(state: AgentState) -> dict:
return {"results": [...]}
 
3. GRAPH STRUCTURE: Check if the graph is:
- Linear (A → B → C) — auto-finalize works, no extra code needed
- Fan-out/fan-in (DAG) — auto-finalize works
- Cyclic (has loops / back-edges) — will need watcher.finalize() after invoke
 
4. ASYNC CHECK: If node functions are async (async def), ARGUS handles both — just make sure you're using await app.ainvoke() not app.invoke().
 
5. EXTERNAL CALLS: List which nodes make external API calls (OpenAI, search APIs, databases). This determines whether to enable record_http.
 
Print a summary of what you found and any fixes needed before proceeding.
 
STEP 2 — FIX COMPATIBILITY ISSUES
 
If you found issues in Step 1, fix them now:
- Convert plain dict state to TypedDict with annotations
- Add type annotations to node function parameters
- Make nodes return only their output fields (not the full state)
- Ensure all fields referenced by downstream nodes exist in the TypedDict
 
STEP 3 — INTEGRATE ARGUS
 
Install with all features:
 
pip install argus-agents[all]
 
Or install only what you need:
 
pip install argus-agents # core only (zero dependencies)
pip install argus-agents[langgraph] # + LangGraph adapter
pip install argus-agents[cli] # + argus CLI commands
pip install argus-agents[llm] # + LLM investigation & semantic judge
pip install argus-agents[langgraph,cli] # LangGraph + CLI (no LLM features)
 
Add ArgusWatcher to the file where the graph is built:
 
from argus import ArgusWatcher
 
watcher = ArgusWatcher(graph) # pass StateGraph before compile()
app = graph.compile()
result = app.invoke(initial_state)
print(watcher.run_id)
 
If the graph is already compiled elsewhere, use watch_compiled():
 
watcher = ArgusWatcher()
app = watcher.watch_compiled(app)
result = app.invoke(initial_state)
 
STEP 4 — PICK THE RIGHT CONFIG
 
Choose parameters based on what you found in the audit:
 
watcher = ArgusWatcher(graph,
# --- Detection strictness ---
strict=True, # catches empty lists, nested errors, type mismatches
 
# --- Semantic validators (custom per-node checks) ---
validators={
# example: ensure summaries aren't empty stubs
"summarize": lambda o: (len(o.get("summary", "")) > 10, "Summary too short"),
# wildcard: runs on every node
"*": lambda o: ("error" not in o, "error key present"),
},
 
# --- LLM investigation (needs argus login) ---
investigate=True, # LLM root-cause analysis on failures (default: True)
# set to "always" to investigate every run, False to disable
 
# --- LLM semantic judge (needs argus login) ---
semantic_judge=True, # LLM reviews every node's output for subtle issues (default: True)
judge_model="gpt-4o", # or "gpt-4o-mini" for cheaper runs
 
# --- Deterministic rerun ---
record_http=True, # records every outbound API call (default: True)
# replays from disk on rerun — zero extra cost
 
# --- Redaction (scrub sensitive fields from stored runs) ---
redact_keys={"token", "api_key", "password"}, # exact field name match
redact_patterns=True, # auto-detect emails, URLs, API keys
# redact_functions={ # custom per-field redaction
# "ssn": lambda v: "***-**-" + str(v)[-4:],
# },
 
# --- Output control ---
max_field_size=50_000, # max chars per field before truncation (default: 50k)
)
 
For production / high-throughput pipelines, use ArgusConfig:
 
from argus import ArgusWatcher, ArgusConfig
 
watcher = ArgusWatcher(graph,
config=ArgusConfig(
sample_rate=0.1, # persist only 10% of clean runs (saves disk)
persist_failures=True, # but ALWAYS persist runs with failures (default: True)
dry_run=False, # set True to capture events without writing to disk
),
validators={...},
)
 
If the graph has cycles (loops / back-edges), call finalize after invoke:
 
result = app.invoke(initial_state)
watcher.finalize() # required for cyclic graphs, safe to call on any graph
 
After running the pipeline:
 
argus show last # see what ARGUS caught
argus ui # open the web dashboard
argus doctor # check environment and optional deps
↺ Replay any run

See the truth.

Re-run from any step. ARGUS reuses everything that's already correct — so you only pay for what changed.

Run · 8f9a-22b1 2.47s
Replay #2
extractPassed
enrichPassed
summarizeFailed
validateFailed
respondQueued
Beforestep 3: summarize
confidence0.42
key_points[]
entities[]
output"placeholder text"
Afterreplayed
confidence0.93
key_points["safety", "transparency"]
entities["AI regulation"]
outputFull structured summary
Reused
extract
enrich
2.47s saved
Ran
summarize
1.34s compute
0x
Faster debugging
0%
Lower cost
Zero
Wasted compute
Features

Built for
engineers
who ship.

By the numbers
0
Pipelines Analyzed
0
Silent Failures Caught
0
Root Causes Identified
0
Replay Recoveries
Why Argus

They trace. We detect.

01Silent failures
Auto-detects hallucinations, empty outputs, semantic degradation
Logs calls as successful — silent failures invisible
02Root cause
Walks backward through the graph to the node that broke
Manual trace inspection
03Replay
Re-run any node with frozen inputs and recorded state
Not available
04Setup
argus.wrap(graph) — 2 lines, no account needed
SDK + cloud platform + dashboard config
05CI/CD gating
Built-in — fail pipelines on regressions
Requires custom integration
FAQ

Questions,
answered.

  • A node technically “succeeds” but returns degraded state — empty arrays, placeholder text, collapsed confidence scores, hallucinated tool outputs — that quietly poisons every downstream step.