Experiments, failures and field notes from building AI systems. 29 articles by Sergei Parfenov.
Review contribution · WebDigestPro ·
A reference architecture for commerce systems used by AI agents. I contributed public review feedback on typed provenance, how refusals carry through derived claims, and the checks that bind an agent’s actions to live permissions and current system state.
An offline checker that ranks machine-translated segments by the risk of a major meaning error: a $1.22 Tinker LoRA, Gemma 4 on DigitalOcean, Sentry tracing.
AnyPS5 is not emulating a console. It is replacing a platform's dependencies one function at a time, with AI help in the loop and a hardware oracle to check the result.
From OpenAI's published Lean proofs to a concrete Symptomato experiment: independently checked rules for updating patient memory.
A synthetic experiment for Symptomato shows how an omitted source can hide a conflict, even when the final accepted value is correct.
A 5/5 mutation score missed an empty-filter regression. A manual catalogue reached 8/8, then a ninth candidate exposed why a redundant-looking test mattered.
ExecCritic found that weak generated tests reduced repair success. A runnable Python example shows how to catch a test that approves the wrong fix.
An AI can recall a fact when asked and miss it when it matters. InMind separates retrieval from application—and suggests a better way to test agent memory.
In 660 of 800 autonomous research runs, the agent found a critical flaw and delivered the result anyway. Self-review is not a control unless it can block the effect.
PeakBench separates logical planning from physical scheduling. Eight frontier models could recover dependencies yet still overload finite infrastructure
Four harnesses took the same public ARC-AGI-3 set from 13% to 100% without touching a single weight. Then Microsoft put the harness inside the training loop.
What actually transfers when you fine-tune an open model on a frontier model's reasoning traces: the mechanics, the evidence it works, the evidence it mostly moves format, and how to tell which one you got.
A Bug Smash hunting story: two rediscovered bugs, one claim-etiquette call, an unreported twin in the newest importer, and a disagreement with an AI about the right fix.
A Python generator in Zulip's Microsoft Teams importer yielded a list and then cleared it. Every batch was the same object. Here's why latent bugs deserve fixes.
Fixing Slack import robustness in Zulip: one malformed timestamp aborted entire migrations, and the NaN case corrupted them silently.
An agent pipeline in production skipped its daily report and no component was at fault. The audit that followed found four bugs in our own code, and every one of them was silent.
In March, the CEO of a research lab with zero products closed a $1.03 billion seed round - the...
A frontier model hallucinates ~1% on a single task. Chain it into a 20-step autonomous agent and the math guarantees failure most of the time, no matter how good the model is. Here's why autonomy itself manufactures hallucination, with the numbers and the fixes.
Running an agent on your own hardware fixes data sovereignty and nothing else. Prompt injection, silent provenance failures, and privilege escalation all survive the move to local. Here's where local agents are genuinely safe to deploy in 2026, and where 'local' is just a comforting word.
Reading Lilian Weng's harness engineering survey as a reliability engineer - what self-improving harness papers actually show, and the three invariants every working loop converges on.
Four provenance-tracking arms, identical gates, one uncompacted oracle: measuring exactly what memory compaction does to agent gate decisions - including the preregistered hypothesis that failed.
A typed provenance vector is useless if downstream code ignores it, and impossible if it can't survive being compressed to fit a 500-step agent's memory. Part 4: enforcement by construction, and compression that keeps the axes. The comment section keeps finding the holes.
Two posts ago I gave you a boolean trust tag. A commenter took it apart, and he was right. Here's the better model: trust is a vector over axes, provenance is what propagates, and the consumer applies the policy. Part 3 of a series that my comment section is co-writing.
Anthropic's Claude Fable 5 launched June 9 and was suspended worldwide by a US export-control directive on June 12. Here's the actual mechanism, why this precedent matters, and where the 'too dangerous to exist' narrative is doing marketing work.
Every capacity fix - retries, fallbacks, caching - buys availability by acting on output it didn't freshly earn. Why uptime and correct uptime are different SLOs, and how to engineer the second one.
I wrote a post on model distillation. The comments were thoughtful, specific, and technically sharp - and that's exactly what made me check whether any of them were written by people.
Companies spend 44 cents of every AI-token dollar fixing bugs the AI itself wrote. The speedup is real - but it's borrowed against future maintenance. Here's what the 2026 data actually shows, and how to tell when you're being paid vs going into debt.
A hands-on knowledge-distillation project: Qwen2-VL-7B → 2B for UI-screenshot understanding, trained, evaluated and benchmarked end-to-end on an M4 Pro. 2.4× faster - and why the teacher lost on ROUGE-L.
The dominant production failure mode for LLM agents in 2026 isn't bad reasoning - it's capacity. Here's what the data shows, why nobody demos it, and the capacity-engineering patterns that actually keep agents alive under load.
A practical, no-hype explainer of knowledge distillation in LLMs - the actual mechanics, why distilling from a closed API is different, and what the OpenAI/Anthropic vs DeepSeek allegations are really about.