AI agents rarely fail in one clean, obvious way. A run may finish with a plausible answer after calling the wrong tool, recovering from a hidden error, spending far more tokens than expected, or skipping a step that the product depends on. The final text is visible. The path that produced it often is not.

I maintain AgentInspect, an open-source TypeScript toolkit for inspecting agent executions locally. This article explains why I designed it as a local evidence debugger—not as another hosted agent dashboard—and where that choice helps or deliberately does not help. The commands and behavior shown here were checked against the immutable [email protected] release.

The missing artifact between “it failed” and “here is why”

When an agent misbehaves, engineers usually begin with one of three artifacts:

1. application logs,
2. the final model response, or
3. a screenshot from a monitoring product.

Each is useful, but none is automatically a complete explanation. Logs are often spread across services and ordered by time rather than causality. A final response hides the execution path. A dashboard may show the run clearly, yet the evidence can be difficult to attach to a pull request, reproduce in CI, or review without an account.

The gap I wanted to address was narrower:

Given one agent run, can an engineer capture enough structured evidence locally to inspect its path, check explicit expectations, compare it with another run, and package a reviewable artifact?

That question led to a different product boundary.

A debugger produces evidence; it does not replace operations

Hosted observability products are designed for durable ingestion, fleet-wide monitoring, retention, alerting, and team operations. Those are valuable capabilities. AgentInspect is not trying to recreate them on a laptop.

That contrast is deliberate. OpenTelemetry's concepts documentation describes the broader signal, instrumentation, context-propagation, and collection model. For a production-oriented example, this HackerNoon walkthrough of multi-agent observability with OpenTelemetry and SigNoz starts from centralized operational questions. My focus here is the smaller pre-release evidence loop that can live beside the code.

Its core workflow is deliberately smaller:

agent execution
      |
      v
local structured trace (JSONL)
      |
      +--> readable execution tree
      +--> deterministic checks
      +--> run-to-run diff
      +--> redacted evidence bundle

The output is an artifact that can travel with engineering work. A developer can inspect it before opening a pull request. CI can reject a known-bad trajectory. A reviewer can verify a bundle without reproducing the entire run. The source trace uses JSON Lines, a line-oriented structured format that remains friendly to ordinary text-processing tools.

That makes AgentInspect complementary to APM, hosted agent observability, and evaluation platforms. It does not provide hosted retention, production alerting, prompt management, automatic remediation, or compliance certification. Drawing that boundary matters: a local evidence tool is useful only when users know what it does not promise.

The smallest useful capture loop

The root package includes wrappers for a run, ordinary steps, tool calls, and LLM calls. A minimal synthetic example looks like this:

import { inspectRun, step } from "agent-inspect";

const answer = await inspectRun(
  "quickstart-support-agent",
  async () => {
    const plan = await step("plan", async () => ({
      topic: "refund eligibility",
    }));

    const docs = await step.tool("search-docs", async () => ({
      policy: "Returns are accepted within 30 days.",
    }));

    return step.llm("draft-answer", async () => ({
      text: `${plan.topic}: ${docs.policy}`,
      model: "synthetic-demo",
    }));
  },
  { traceDir: "./.agent-inspect" },
);

The example does not claim automatic discovery of every internal operation. It records the boundaries that the application instruments. Framework adapters and standards-based inputs can reduce integration work, but manual instrumentation remains explicit.

That explicitness is a design advantage in evidence-oriented debugging. The engineer decides which operations are meaningful enough to name and review. The trace can then be read without access to a remote service:

npx agent-inspect view quickstart-support-agent \
  --dir .agent-inspect \
  --summary

The local file is not the goal by itself. It is the source from which several review views are derived.

One run, four engineering questions

1. What actually happened?

The execution-tree view reconstructs parent-child relationships and exposes errors, fallbacks, retries, and parallel siblings. That is different from merely sorting log lines by timestamp.

2. Did the run satisfy our explicit contract?

Deterministic checks can verify properties such as required or forbidden tools, call limits, order constraints, run status, duration, token ceilings, and observation failures.

npx agent-inspect check quickstart-support-agent \
  --dir .agent-inspect \
  --preset trajectory \
  --required-tool search-docs \
  --fail-on-observation failed

These are not semantic LLM evaluations. They answer explicit structural questions with reproducible rules. Whether the answer is persuasive, safe, or factually correct may require other evaluators.

3. What changed between two runs?

A run diff can show the first behavioral divergence, added or removed steps, output changes, errors, and timing differences. This is useful because two code revisions with a small textual diff can produce very different agent trajectories—and identical code can produce different trajectories when models or external tools change.

Local traces can contain sensitive inputs, outputs, and metadata. AgentInspect includes best-effort redaction and safety assessment, followed by an evidence-bundle workflow:

npx agent-inspect verify-safe quickstart-support-agent \
  --dir .agent-inspect

npx agent-inspect bundle quickstart-support-agent \
  --dir .agent-inspect \
  --profile share \
  --out ./evidence

npx agent-inspect bundle verify ./evidence

The final command verifies the recorded file hashes and manifest integrity. It does not create a digital signature, establish authorship, or certify regulatory compliance. A passing integrity check means the files still match the bundle manifest—not that the underlying agent was correct.

Why local-first was an architectural decision

“Local-first” is sometimes treated as a privacy slogan. Here it is also about composability.

No account is required for the core loop

The basic workflow can run with a local directory and a CLI. That lowers friction for a developer who wants to inspect a single failing test or attach evidence to a change.

The evidence can participate in existing workflows

JSONL traces and generated reports can be archived as CI artifacts, reviewed in a pull request, or passed to another system. The evidence is not trapped behind one UI.

Capture and export are separate decisions

Recording locally does not make a trace safe to share. The separation between source trace, redacted artifact, safety assessment, and bundle makes that risk visible instead of treating export as a harmless final click.

Determinism belongs close to the change

Structural checks are most useful when they run beside code and fixtures. A contract such as “retrieval must occur before generation” can fail in CI without waiting for someone to notice a dashboard anomaly.

The tradeoffs I accepted

Every boundary removes capabilities as well as complexity.

AgentInspect does not continuously watch a production fleet. It does not guarantee that adapters observe operations that were never instrumented. Its deterministic checks cannot decide whether prose is correct. Safety scanning is best effort, and a human still owns the decision to share an artifact. Timing comparisons can be noisy. Evidence integrity is not evidence truth.

Those limitations are not footnotes. They define how the tool should be used:

• use execution evidence to understand behavior;
• use deterministic contracts for stable structural invariants;
• use semantic and domain evaluators for answer quality;
• use hosted operations tools when you need fleet monitoring and retention;
• review redacted artifacts before sharing them.

A better unit of review for agent changes

Traditional software review focuses on code because code strongly determines behavior. Agent systems add models, prompts, tools, retrieval, external state, and nondeterminism to that relationship. The code diff remains necessary, but it is no longer sufficient.

The additional review unit I want is compact and concrete:

change
  + source diff
  + representative run
  + explicit contract result
  + behavioral diff
  + share-checked evidence, when needed

This does not make an agent deterministic. It makes the engineering discussion less speculative. Instead of “the new prompt seems better,” a reviewer can ask: Which tools ran? Where did the path diverge? Which invariant failed? What evidence is safe to share?

That is why I built a local evidence debugger rather than another dashboard. The goal is not more telemetry. It is a shorter, reviewable path from an agent’s behavior to an engineering decision.

Try the boundary, not just the happy path

If you evaluate AgentInspect, start with a synthetic run that includes a failure and fallback rather than a polished demo. Inspect the tree, add one deterministic check, compare it with a corrected run, and build a share-profile bundle. That exercise reveals both the value and the limits of the approach quickly.

The project and the tagged release used for this article are available on GitHub. I welcome issues that identify unclear evidence, misleading defaults, or gaps between the documented boundary and real developer workflows.