Architecture

tok0 is a single Rust binary, but internally it’s split into six tightly-bounded subsystems. Each one owns one concern and is testable in isolation.

The big picture

┌────────────────────────────────────────────────────────────┐
│                       tok0 binary                          │
├────────────────────────────────────────────────────────────┤
│ main.rs · clap Commands enum + run_proxy / run_meta        │
├──────────┬───────────┬────────────┬──────────┬─────────────┤
│  bridge  │  engine   │compressors │ insights │  scanner    │
│  (hooks) │ (pipeline)│  (filters) │ (stats)  │ (sessions)  │
├──────────┴───────────┴────────────┴──────────┴─────────────┤
│   rules/         · 120+ embedded TOML compression rules    │
│   extensions/    · user rule packs from git URLs           │
└────────────────────────────────────────────────────────────┘

engine — the shared infrastructure

Lives at apps/cli/src/engine/. Every other subsystem depends on it, none of it depends on them.

Module	Responsibility
config.rs	Read & write ~/.config/tok0/config.toml.
meter.rs	SQLite token meter. Async mpsc writer, flush() on shutdown.
shell.rs	strip_ansi, truncate, execute_command, execute_with_timeout.
compressor.rs	The 8-stage pipeline itself. Pure functions.
rules.rs	Declarative TOML rule engine. Loads + applies FilterRule.
rules_cache.rs	mtime-based cache so rules only re-load when changed.
dispatcher.rs	Routes (command, args) → compressor or TOML rule.
snapshot.rs	Dumps raw output to ~/.cache/tok0/ on filter failure for debugging.
timeout.rs	Hard timeout on every spawned command.
telemetry.rs	Anonymous instance ping (opt-in only, gated behind cloud feature).
updater.rs	Self-update check against GitHub Releases.
cloud.rs	(gated) Team-dashboard API client.

bridge — AI tool integration

Lives at apps/cli/src/bridge/. Owns everything tok0 does outside its own process — installing hooks, verifying signatures, talking to AI tool config files.

Module	Responsibility
setup.rs	tok0 init — detects tools, writes hook configs, prints manual paste-ins.
rewriter.rs	tok0 rewrite — invoked from inside hooks; reformats payloads.
adapters.rs	Per-tool format translators (~20 lines each). Format only, no logic.
status.rs	tok0 status — reports which bridges are wired and healthy.
verify.rs	tok0 verify — SHA-256s every installed hook against the embedded manifest.
trust.rs	Project trust gating. tok0 trust / tok0 untrust.
integrity.rs	The cryptographic spine of trust + verify.

compressors — per-command native filters

Lives at apps/cli/src/compressors/. Organized by ecosystem:

compressors/
├── git/         git, gh, gt, diff
├── js/          npm, pnpm, bun, yarn, deno, vitest, jest, tsc, next, …
├── python/      ruff, pytest, mypy, pip, uv, poetry, conda, black, …
├── go/          go, golangci-lint
├── rust/        cargo, runner
├── dotnet/      dotnet, binlog, trx, format_report
├── java/        gradle, maven
├── ruby/        rake, rspec, rubocop
├── cloud/       aws, gcloud, az, terraform, pulumi, helm, ansible, …
├── db/          mysql, redis, mongo, sqlite3
├── system/      ls, grep, find, diff, ssh, rsync, archive, net, proc, …
├── pkg/         brew, apt, dnf, pacman, snap, flatpak, nix, winget, …
├── build/       make, cmake, bazel, ninja, meson, scons, …
├── lint/        shellcheck, hadolint, stylelint, yamllint, markdownlint, …
├── security/    semgrep, trivy, bandit, snyk, grype, sonar
├── cicd/        act, precommit
├── api/         httpie, grpcurl, newman, loadtest
├── versionmgr/  nvm, fnm, pyenv, rbenv, rustup, asdf, mise, volta
└── (15 more language ecosystems)

Each *_cmd.rs exposes one pub fn run(args: <Args>) -> Result<()> and follows the same shape: capture output, apply filter, fall back on error, propagate exit code, record to meter.

insights — local analytics

Lives at apps/cli/src/insights/. Reads from the SQLite meter and presents.

Module	Command
stats.rs	tok0 stats — total tokens saved, ratios per command, 30-day chart.
costs.rs	tok0 costs — translates token savings into $ at provider rates.
adoption.rs	tok0 adoption — % of agent commands that hit a compressor.
profiler.rs	tok0 profile — per-compressor time + size breakdown.
doctor.rs	tok0 doctor — full diagnostic across config, hooks, db.
rules_cli.rs	tok0 rules list/show/test — inspect loaded rules.

scanner — historical adoption

Lives at apps/cli/src/scanner/. Reads AI tool session histories (Claude Code transcripts, Cursor session logs) to build a “missed savings” report — what tok0 would have saved if it had been installed last month.

Module	Responsibility
session_reader.rs	Per-tool transcript parser.
command_catalog.rs	300+ command classification rules.
opportunity.rs	The user-facing report.

extensions — community rule packs

Lives at apps/cli/src/extensions/. Lets users install rule packs from arbitrary git URLs:

tok0 extensions install https://github.com/user/repo

Every extension goes through trust prompting before any rule from it is loaded. See Extensions.

Threading model

tok0 is single-threaded by design. There is no tokio, no async-std, no futures. The metering writer runs on a dedicated background thread connected via mpsc, and that’s the entire concurrency story. Everything else is synchronous, blocking I/O.

This is a deliberate constraint, not an oversight. Async runtimes add 5–10ms of cold-start time we cannot afford for a tool that wraps every shell call. Single-threaded execution also makes the pipeline trivially deterministic and the binary 30% smaller.

Build flags

Feature	Default	Adds
default	yes	The minimal CLI. Everything in this doc.
cloud	no	tok0 auth, tok0 cloud team, real telemetry pings.

CI runs both configurations on every PR.