Agent observability · TypeScript
Time Machine SDK
A TypeScript SDK and dashboard for debugging AI agents. It records every step of a run, lets you fork from any step and replay only what comes after it, and runs eval suites that can block a pull request.
- 11Claude Code hook events captured
- 7step types in the trace model
- 30+models in the built-in pricing table
- 0runtime dependencies in the SDK
Rerunning the whole chain
When an agent fails at step 9, the usual move is to rerun all of it with more print statements, paying again for every LLM call that already worked. The logs say step 9 failed. They don't say which earlier step sent it down the wrong path.
I wanted something like git for agent runs, where I can look at any step, diff two runs and branch from the point where things went wrong. Time Machine is my attempt at that. It is a Next.js dashboard and API on Supabase Postgres, plus a TypeScript SDK with no runtime dependencies.
What gets recorded
You start an execution with the SDK and record steps against it. A step is one of seven types: llm_call, tool_use, decision, retrieval, human_input, transform or custom. Each one stores its full input and output, token counts, cost, latency, tool calls and an optional state snapshot. Cost comes from a built-in pricing table for 30+ models, so the dashboard can show spend per step and per run.
The SDK batches steps and flushes every 500ms or every 10 steps, retries failed requests with exponential backoff, and fails open, so an SDK error never crashes the host app. With LangChain, a callback handler records LLM, tool, agent and retriever events for you.
Fork and replay
A fork copies every step before the fork point into a new execution unchanged and resets the fork step to pending with your edits merged into its input. Replay walks forward from there. LLM calls run again through OpenAI, Anthropic or OpenRouter, while human input, decisions and transforms reuse their recorded values. After the fork point the parent's cached outputs are treated as stale and never used as a fallback.
Every tool gets a rule: reexecute or use-cached. Read-only tools like web_search run again by default. Tools that write, like send_email, write_file or db_insert, return their cached output so a replay doesn't send the same email twice, and tools the engine doesn't recognize default to cached. A compare view diffs the fork against its parent step by step on inputs, outputs, cost and latency.
Evals as a CI gate
An eval suite is a set of cases taken from real executions, and any run in the dashboard can be saved as a case. Running a case reuses the fork code: the source execution is forked at its first step, the case input is applied, and replay produces the output that gets checked. Assertions include contains, regex, json_path, cost_under, latency_under, step_count, similarity and llm_judge, and the score is the fraction that pass.
In GitHub Actions, one step starts a run through the API and a tm-eval command polls the status endpoint with a minimum score. If the score is too low it exits with an error, which fails the check and blocks the PR. The docs suggest keeping llm_judge out of the gate because every judge call costs money.
Claude Code hooks
Claude Code runs each hook as its own short-lived process, so I wrote a bridge. An installer registers it for 11 hook events, from SessionStart through tool calls and subagents to SessionEnd. Each call reads the event JSON from stdin, finds the session in a state file under ~/.timemachine/claude-code, creates or resumes the execution, and records the step immediately so nothing is lost when the process exits. On SessionEnd it parses the transcript JSONL for assistant messages and file edits that hooks never see.
An MCP server lets Claude Code read the traces back, so I can ask it about recent failed runs without opening a browser. Replaying a session from inside Claude Code is still on the roadmap. The bridge, CLI and MCP server live in the monorepo and are not part of the 0.1.1 npm release.