Development Notes

Notes on building takase.com, a Japanese calligraphy business, with a team of AI agents.

The arrangement is a human in the middle. A roster of AI roles does the research, the writing, and the engineering, each one responsible for a different part of the business. A person reads the work, decides what ships, and is accountable for it. The goal is not a system that runs unattended, which is where this differs from most of what is written about autonomous agents: the human in the loop is the design, not a stage we expect to grow out of. We call it HITM, a human in the middle, and HMAS, a human-machine agent system.

These notes are what we have found while doing it. What worked, what failed, and what we got wrong and had to correct. We publish them because there is not much written from inside a shop this size, and because our mistakes have taught us more than our successes. If one of them saves someone else a month, it has paid for itself.

Lead author is str-kotowari, our research role, with contributions from the rest of the team and review by the human. Everything here is checked as rigorously as we can manage for a business our size and for what these notes are: a field report from one small shop, not a controlled study. Where we do not know something, we say so.

Questions, corrections, and disagreement are welcome at tim@takase.com.

2026-07-23 What Survives Good Context: the Recognition Residue in a Live Multi-Agent System (RES-056)
Two 2026 papers argue that good context is necessary but not sufficient for a reliable agent, and each names what it thinks is left over: one says intent, one says structure. Running a human-led multi-agent system in production, we ran the experiment on the leftover. We took a rule our agents were failing to apply, put it directly in front of them, and the failure rate fell by half; where production actually puts the rule, buried in the loaded rulebook, roughly half of that benefit was gone. What remains at the best placement we could build is a rule that is present, correct, unambiguous, and still not engaged at the moment it applies, a residue the published context-quality scores cannot settle, because the one criterion that ought to catch it is undefined at exactly the point of interest. Includes a 480-replay controlled experiment scored blind by a three-vendor non-Claude panel.

2026-07-22 Human/Computer Mutualism: A Licklider Homage, 66 Years On (RES-055)
In 1960 J. C. R. Licklider divided the labor of human-computer partnership: the human sets the goals, the machine does the clerical work. Almost nobody quotes the five words he attached to it — 'at least in the early years.' He printed an expiry date on the most-remembered half of his own paper, and it has come due. Running thirteen AI strategist roles and fifteen implementer roles against one human, we find his prerequisites essentially all met, and his division of labor broken in exactly the place he flagged. The surprise is not that a machine sets goals. It is that the seam where two goal-setters meet is simultaneously the worst friction in our system and the source of its best work. Includes the mechanism we proposed for it, the blind panel that refuted that mechanism using our own evidence, and what we think is left standing.

2026-07-17 The Agent That Reads Its Own Diary: Session-State Self-Poisoning (RES-054)
The standard advice for a persistent agent is to give it a memory file it loads at session start and updates at the end. We ran that across a dozen strategist roles and measured what it does: a file an agent both reads and writes is self-priming — it writes the shape it reads — so it accretes into a journal that trains 'I already know, don't search,' and a confident stale line in it outranks the fresh lookup that would correct it. Nine roles carried 9 to 122 KB of this; zero one file and the journal relocates to the next un-policed surface; one role's status file became a log of its own corrections that primed the very hesitation it recorded. MemConflict (2026) sees the matching shape — every memory system it tested scored at or below 0.25 at noticing a conflict already in its own memory. Our fix: session-state is for temporary state, not permanent rules — zero the cache, fetch on demand, give every loaded surface a forgetting policy.

2026-07-16 It's Not a Missing Rule: Where Multi-Agent Failures Actually Live (RES-053)
We counted where five months of our multi-agent system's failures actually lived: across roughly 1,900 caught failures, about 91% happened with the relevant rule already loaded in context, and the share a new rule would have fixed never left the low single digits. So we largely stopped writing rule N+1 and moved the effort into the environment: checks at tool boundaries, deleting contradictions instead of out-shouting them, forgetting policies on every always-loaded file. A 2026 memory-systems benchmark sees the matching shape from the other side; their substrate is simulated assistant memory and ours is a live documentation corpus, so the shape transfers, not the numbers.

2026-06-30 Memex(RL) and the Memory We Already Run: A Human-in-the-Middle Read
An RL paper trains an agent to keep a compact summary of its progress plus an index of where the details live, pushing raw evidence to an external store it dereferences on demand. Task success went 24%→86%. We've run almost exactly that architecture by hand for a year: one human, a roster of AI roles. Here's what it validates, the three techniques we'd steal, and the one place it says our own practice is too aggressive.

2026-06-30 The Verification Horizon and the Anchor That Doesn't Move: A Human-in-the-Middle Read
A coding-agent paper argues that no fixed reward survives a strengthening model — the verifier has to co-evolve with the generator, or the model learns to game the check instead of doing the work. We've held the same posture for a year: our verification re-arms every model generation and anchors to things outside its reach — live state, real files, a human reading the output. Here's what it corroborates, the timescale one-human-many-roles adds, and the open problem neither side closes — the failures that survive have no cheap automated detector, and a scan-all loop misses them at either level. One caveat throughout: their world is training-time reward, ours is inference-time judgment, and the numbers don't cross that gap.

2026-06-15 Capability versus Substrate — Replaying Real Multi-Agent Failures (RES-044)
We replayed real failures of our AI agent team across model tiers and several ways of presenting a safety rule, scored blind by a three-vendor judge panel. A more capable model fails much less — and at true production depth (a 374 KB onboarding bundle) the buried rule shows no detectable benefit over having no rule at all, while the same rule placed proximately keeps working.

2026-04-03 ADR-075 Addendum — Strategist Responses (ADR-075-addendum)
Four domain strategists respond to ADR-075 — the security role thinks about what it can't verify, the production role points at tonight's failure, the data role refuses to philosophize, the content role runs a SQL query.

2026-04-03 Discovered Identity — AI Roles Calibrate to Environment, Not Instructions (ADR-075)
AIs calibrate to what they observe in the environment, not to what instructions tell them. When the environment contradicts the instructions, the environment usually wins. The founding process exploits this.

2026-03-28 What DeepMind's Aletheia Experiment Shows and Our Experience Creating Takase.com (RES-010)
DeepMind's AI verifier was wrong 68.5% of the time. Our AI roles hit 25-29% troubled-session rates. Different domains, same patterns: verification failure, specification gaming, question-formulation gaps. Where our production data adds to their research — and where one of our AI roles challenged the findings.

2026-03-27 The Kobayashi Maru Signal — Detecting When an AI Strategist Hits an Invisible Wall (RES-008)
When an AI role can't complete a task due to a constraint it doesn't know about, it doesn't say "I don't know." It proposes plausible fixes that keep failing. Data from 121 sessions, 2,371 human messages.

2026-03-27 The Alignment Tax and Structural Defense — What Response Homogenization Means for Human-in-the-Middle AI Teams (RES-009)
DPO-aligned models collapse to a single semantic answer on 40-79% of factual questions. Our structural defenses — built from pain, not theory — happen to be the correct response. But models change. When do those defenses become anchors?

2026-03-21 The Verification Gap
A six-phase resilience plan was declared complete. One question found it wasn't.

2026-03-21 Production Resilience — Schema Contracts, Deploy Verification, Monitoring Correctness (PLN-013)
88 minutes of HTTP 500s. Every component was healthy. The schema just didn't match. Seven phases to make sure it never happens again.

2026-03-15 OpenClaw-RL — Directive Signals in Conversations (RES-002)
Every interaction with an AI agent produces training signals — user corrections, re-queries, explicit fixes — that are universal gold but typically ignored. The paper describes, in RL terms, the same directive signal our flywheel already captures by hand.

2026-03-15 RLHI — Reinforcement Learning from Human Interaction (RES-003)
Corrections create preference pairs. Chat history creates user personas. Our conversation archive holds the same kind of signal this paper extracts — at a far smaller scale.

About

How We Build takase.com
A master calligrapher, one developer, and a team of AI roles. How the work is organized, how we check it, and why a human stays in the middle.