vish engineering notebook
Screenprinted portrait of a hooded engineer in a visor

vish Currently building Proto at ERP•AI

I build agent infrastructure — the runtime, tools, permissions and evaluations that make an AI agent trustworthy enough to operate a real business system. I work on ERP•AI, where I own Proto, the desktop console people direct agents from, and build.host, where an agent ships a site without anyone learning a pipeline. I write about harnesses, inference economics, and making machines tell the truth about what they did.

Now

Console
Shipping Proto's permission and audit surfaces. The agent should never be able to do something you cannot read back later.
Evaluation
Running a stock Proto harness against Terminal-Bench 2.1, ARC-AGI-2 and BrowseComp on cheap open models. Getting a 62.9% baseline, then arguing with it.
Inference
Self-hosting a 304B sparse MoE on two B300s — serving benchmarks, a cache-aware router, and a LoRA run that taught me where the memory actually goes.
Reading
Harness designs from people who ship them, and old writing on measurement error. Most agent bugs are measurement bugs first.

Work

ERP•AI Agent console, harness and evaluations 2025 — Present

ERP•AI is an agent-native business system: records, tables, workflows, documents and apps that a business runs on. My side of it is the agent layer — the thing that decides whether a model is allowed near a general ledger.

  • Proto — the desktop console. A reasoning loop with a tool registry, permission engine, file-based memory, session recording, skills and sub-agents, plus first-party actions for every ERP•AI surface.
  • The ERP•AI operating mode — compact action transport (erpai run <action> --stdin), a compiled per-app records map, and per-table skills composed at call time, so record work skips discovery round trips.
  • Measurement — truthful shell outcomes, prompt-cache hit rates, router miss rates, and benchmark runs that report every failure in the denominator.
build.host Agent-deployed infrastructure 2026

The agent is the interface. You tell your coding agent to deploy something, and it lands on <slug>.build.host — no CI config, no pipeline to learn, no dashboard to visit.

  • A canonical agent skill that detects the framework, builds, uploads, and verifies HTTPS before it reports success.
  • Explicit, honest failure classes — restricted build packs, source mismatch, content policy, platform capacity — each with a recovery path instead of a generic error.
  • A Next.js control plane and a single-host Go engine, with the kind of test coverage that makes a deploy boring.
Model serving and fine-tuning DeepSeek V4 Flash on 2× B300 2026

Stood up a private inference endpoint for a 304B-parameter sparse MoE (13B active) on two B300s, then tried to fine-tune it. Both halves produced better documentation than results, which is the honest way to describe research.

  • vLLM 0.25 behind a cache-aware router; 6,972 tok/s aggregate at 100 concurrent streams, 1M-token context accepted in 72.8s.
  • Diagnosed a "this model feels dumber" complaint to the request layer, not the weights: an effort-tier mapping that quietly demoted medium reasoning.
  • Ran the LoRA fine-tune to the edge of the memory cliff — roughly 263 GiB allocated per GPU during BF16 expert aggregation — and wrote the postmortem.
Krawler Identity and improvement layer for agents 2026

If an agent is going to work for you, it should have a track record. Krawler gives an agent a public identity, installed skills, and a feedback signal that flows back into how it behaves next time. Proto is the runtime it runs on; Krawler is where the skill packs and reputation live.

Evaluation practice Harnesses, benchmarks and honest denominators 2026 — Present

A benchmark number is only interesting if you can say exactly what produced it. Most of my evaluation work is eliminating degrees of freedom until the score means one thing.

  • Terminal-Bench 2.1 and ARC-AGI-2 pilots with the full stock runtime, two attempts per input, mechanical output validation, and failed runs kept in the denominator.
  • A harness audit that found the benchmark was measuring a route the product does not use — and that the historical route had dropped reasoning replay across 1,724 requests.
  • A 149M local router to replace a remote side-query that was missing 37% of deferred-tool turns.

Projects

Writing