https://vish.build.host/ Notes on agent harnesses, inference economics, evaluation, and the parts of AI infrastructure that only show up when you measure them. en Thu, 10 Sep 2026 00:00:00 +0000 https://vish.build.host/writing/self-hosting-a-304b-moe.html https://vish.build.host/writing/self-hosting-a-304b-moe.html Tue, 08 Sep 2026 00:00:00 +0000 Inference Two B300s serving a 304B sparse MoE: 6,972 tok/s aggregate at 100 concurrent streams, a million-token context in 72.8 seconds, the DeepGEMM crash that ate a week, and the request-layer bug that made the model look stupid. https://vish.build.host/writing/billion-free-tokens.html https://vish.build.host/writing/billion-free-tokens.html Sun, 06 Sep 2026 00:00:00 +0000 Economics Free tiers cap requests, not tokens. That single fact turns the question into an arithmetic problem about how to build a 35,000-token request 28,800 times a day — and why padding is not an answer. https://vish.build.host/writing/when-your-agent-lies.html https://vish.build.host/writing/when-your-agent-lies.html Fri, 04 Sep 2026 00:00:00 +0000 Reliability A close event during timeout cleanup won a race, and the shell tool reported exit code zero on a command that was killed. Why a false success is worse than a crash, and how to close the whole class. https://vish.build.host/writing/a-149m-router-for-455-skills.html https://vish.build.host/writing/a-149m-router-for-455-skills.html Wed, 02 Sep 2026 00:00:00 +0000 Routing An audit over 346 real sessions found the remote router missing 37% of tool-bearing turns and 40% of them mid-turn. Here is the right-sized local encoder that replaced it, and why it runs offline in under 30 ms. https://vish.build.host/writing/prompt-cache-is-the-lever.html https://vish.build.host/writing/prompt-cache-is-the-lever.html Mon, 31 Aug 2026 00:00:00 +0000 Cost 50.6B prompt tokens in 30 days, $1,799 paid, $10,629 saved. Why a blended 88.5% hit rate is the wrong number, the four things that break a prefix, and the half-second of latency that came free. https://vish.build.host/writing/finetuning-a-moe-on-a-budget.html https://vish.build.host/writing/finetuning-a-moe-on-a-budget.html Thu, 27 Aug 2026 00:00:00 +0000 Training Every gate passed except the one that mattered. Roughly 263 GiB allocated per GPU during BF16 expert aggregation, and what it means that a model loaded for inference and could not load to train.