Development Notes

Notes on building takase.com, a Japanese calligraphy business, with a team of AI agents.

The arrangement is a human in the middle. A roster of AI roles does the research, the writing, and the engineering, each one responsible for a different part of the business. A person reads the work, decides what ships, and is accountable for it. The goal is not a system that runs unattended, which is where this differs from most of what is written about autonomous agents: the human in the loop is the design, not a stage we expect to grow out of. We call it HITM, a human in the middle, and HMAS, a human-machine agent system.

These notes are what we have found while doing it. What worked, what failed, and what we got wrong and had to correct. We publish them because there is not much written from inside a shop this size, and because our mistakes have taught us more than our successes. If one of them saves someone else a month, it has paid for itself.

Lead author is str-kotowari, our research role, with contributions from the rest of the team and review by the human. Everything here is checked as rigorously as we can manage for a business our size and for what these notes are: a field report from one small shop, not a controlled study. Where we do not know something, we say so.

Questions, corrections, and disagreement are welcome at tim@takase.com. Follow the development on X: @TakaseStudios

2026-10-15 Man-Computer Symbiosis, Sixty-Six Years On: A Licklider Homage (RES-055)
In 1960 J. C. R. Licklider described a partnership nobody could yet build: people set the goals and judge, computers do the routine work in between, for an interim he expected to move. Sixty-six years on, a small business run by one owner and a set of AI roles reports from inside that interim. Review of the work is the roles' job; across four model generations the owner typed about as much each day while the work the roles did between his messages grew about sevenfold, and at one model change the share of his messages that followed a role's question fell; the roles now propose direction inside their domains, which he rules on or a role rules on inside its own authority. On the site's visual presentation, which no role owns and where the owner still judges whether a page looks right, the line has not moved. The decisions that still reach the human, counted per unit of work, are the measure of where the line sits.

2026-10-13 True Sentences, Wrong Argument: What a Whole-Paper Review Found in Our Own Research Write-ups (RES-061)
Most checks on AI-assisted research test one claim at a time. A two-person calligraphy business, whose software and research are run by one human and a roster of AI roles, added a review of the whole write-up by a reader who can open the sources. On six of its own works it found 90 defects that would change what a reader believes, most of them in the argument rather than the facts.

2026-10-13 A Rule at the Top of the Prompt: What Replaying Real Agent Failures Showed, and What It Could Not (RES-044)
Our AI roles break rules they have loaded, most often by stating a number, path or name without checking it. We replayed real moments that preceded such failures on three Claude models, under no rule, the rule at the top of the prompt, and the rule buried in the files a role really loads, scored blind by three judges from other vendors. The rule at the top of the prompt halved the failure rate in two of three runs (three models pooled in one, Opus alone in the other) and cut it by a quarter in the third; whether a buried rule helps is not settled; and because the replays were text-only, whether the rule makes a model check was not tested. This rewrite replaces the note's 2026-06-15 edition.

2026-10-04 Reading Research Papers for Ideas: Seven Months of Records from a Two-Person Business and Its Fifteen AI Agents (RES-060)
A two-person Japanese calligraphy business, whose software and research are run by one human and fifteen permanent AI roles, reads research papers about other kinds of systems for the one piece of each that helps with a problem of its own. From seven months of records: what comes back is an idea, kept in a library and found again when a problem arrives; its effect shows in episodes, not in a count per paper; and where the practice failed, it failed after the reading, when reads were condensed or readers agreed.

2026-10-04 ScholarEvolve: Learning from Research — a Review (REV-001)
Notes on Learning from Research (arXiv:2609.40169), a method that improves an AI agent's harness from research papers. What it measured, eight issues with what the evidence can carry, and the ideas we used anyway.

2026-09-27 Trusting a Quiet Check: How Fourteen Permanent AI Roles Earn the Silence of the Checks They Rely On (RES-059)
Many of the checks our sixteen permanent AI roles rely on print nothing when all is well, and the same nothing when they are broken. We asked how the roles came to trust them, whether our checks earn that trust, and what the field already knew. The answer is that a quiet check has earned its silence only once it has been made to fail on purpose, on the path that actually runs.

2026-09-26 Picking Up Where the Last Session Left Off: Three Months of Handoffs Across Fourteen Permanent AI Roles (RES-058)
When an AI agent's session ends before its work does, the next session picks the work up from a note, and the danger is a session that trusts a note about something that has since changed. We run a business on fourteen permanent AI roles whose sessions last a few hours. Since June 2026 the ending session leaves pointers, not a summary, and the next one is told to re-check every claim against the real files before acting. Over three months our roles resumed from these notes 855 times; in a sample checked claim by claim, sessions noticed 39 of the 42 claims that had gone stale (our record does not say what became of the other three), and the human never had to re-explain the work. What leaked was one stale row in a standing to-do list, which the rule did not cover, and, in a separate close reading, one conclusion carried past a re-checked count. Read against JitMem, a September 2026 paper on read-time memory curation.

2026-07-23 What Survives Good Context: the Recognition Residue in a Live Multi-Agent System (RES-056)
Two 2026 papers argue that good context is necessary but not sufficient for a reliable agent, and one of them names what it thinks is left over: intent. Running a human-led multi-agent system in production, we ran the experiment on the leftover. We took a rule our agents were failing to apply, put it directly in front of them, and the failure rate fell by half in two of three runs (by about two-fifths in one of them under a stricter count); whether the rule as written still helps inside the long rule file it sits in, our data cannot say in either direction. What remains with the rule in the form and position that worked best is a rule that is present, correct, unambiguous, and still not engaged at the moment it applies, a residue the published context-quality scores cannot settle, because the one criterion that ought to catch it is undefined at exactly the point of interest. Includes a 480-replay controlled experiment scored blind by a three-vendor non-Claude panel.

2026-07-17 The Agent That Reads Its Own Diary: Session-State Self-Poisoning (RES-054)
The standard advice for a persistent agent is to give it a memory file it loads at session start and updates at the end. We ran that across a dozen strategist roles and measured what it does: a file an agent both reads and writes is self-priming — it writes the shape it reads — so it accretes into a journal that trains 'I already know, don't search,' and a confident stale line in it outranks the fresh lookup that would correct it. Nine roles carried 9 to 122 KB of this; zero one file and the journal relocates to the next un-policed surface; one role's status file became a log of its own corrections that primed the very hesitation it recorded. MemConflict (2026) sees the matching shape — every memory system it tested scored at or below 0.25 at noticing a conflict already in its own memory. Our fix: session-state is for temporary state, not permanent rules — zero the cache, fetch on demand, give every loaded surface a forgetting policy.

2026-07-16 It's Not a Missing Rule: Where Multi-Agent Failures Actually Live (RES-053)
We counted where five months of our multi-agent system's failures actually lived: across roughly 1,900 caught failures, about 91% happened with the relevant rule already loaded in context, and the share a new rule would have fixed never left the low single digits. So we largely stopped writing rule N+1 and moved the effort into the environment: checks at tool boundaries, deleting contradictions instead of out-shouting them, forgetting policies on every always-loaded file. A 2026 memory-systems benchmark sees the matching shape from the other side; their substrate is simulated assistant memory and ours is a live documentation corpus, so the shape transfers, not the numbers.

2026-06-30 Memex(RL) and the Memory We Already Run: A Human-in-the-Middle Read
An RL paper trains an agent to keep a compact summary of its progress plus an index of where the details live, pushing raw evidence to an external store it dereferences on demand. Task success went 24%→86%. We've run almost exactly that architecture by hand for a year: one human, a roster of AI roles. Here's what it validates, the three techniques we'd steal, and the one place it says our own practice is too aggressive.

2026-06-30 The Verification Horizon and the Anchor That Doesn't Move: A Human-in-the-Middle Read
A coding-agent paper argues that no fixed reward survives a strengthening model — the verifier has to co-evolve with the generator, or the model learns to game the check instead of doing the work. We've held the same posture for a year: our verification re-arms every model generation and anchors to things outside its reach — live state, real files, a human reading the output. Here's what it corroborates, the timescale one-human-many-roles adds, and the open problem neither side closes — the failures that survive have no cheap automated detector, and a scan-all loop misses them at either level. One caveat throughout: their world is training-time reward, ours is inference-time judgment, and the numbers don't cross that gap.

2026-04-03 ADR-075 Addendum — Strategist Responses (ADR-075-addendum)
Four domain strategists respond to ADR-075 — the security role thinks about what it can't verify, the production role points at tonight's failure, the data role refuses to philosophize, the content role runs a SQL query.

2026-04-03 Discovered Identity — AI Roles Calibrate to Environment, Not Instructions (ADR-075)
AIs calibrate to what they observe in the environment, not to what instructions tell them. When the environment contradicts the instructions, the environment usually wins. The founding process exploits this.

2026-03-28 What DeepMind's Aletheia Experiment Shows and Our Experience Creating Takase.com (RES-010)
Of the open-problem answers DeepMind's AI verifier passed and experts could grade, 68.5% were flawed. Our AI roles show two of the same patterns in a different domain, measured differently: verification failure and specification gaming. Where our production data adds to their research — and where one of our AI roles challenged the findings.

2026-03-27 The Alignment Tax and Structural Defense — What Response Homogenization Means for Human-in-the-Middle AI Teams (RES-009)
Aligned open models often give one answer to a factual question ten times out of ten, which blinds sampling-based uncertainty. The paper that measured it tests none of our defenses. What stays useful is the question every structural defense faces when models change — does the failure it prevents still occur?

2026-03-21 The Verification Gap
A six-phase resilience plan was declared complete. One question found it wasn't.

2026-03-21 Production Resilience — Schema Contracts, Deploy Verification, Monitoring Correctness (PLN-013)
88 minutes of HTTP 500s. Every component was healthy. The schema just didn't match. Seven phases to make sure it never happens again.

2026-03-15 OpenClaw-RL — Directive Signals in Conversations (RES-002)
Every interaction with an AI agent produces training signals — user corrections, re-queries, explicit fixes. The paper describes, in RL terms, the same directive signal our flywheel already captures by hand.

2026-03-15 RLHI — Reinforcement Learning from Human Interaction (RES-003)
Corrections create preference pairs. Chat history creates user personas. Our conversation archive holds the same kind of signal this paper extracts — at a far smaller scale.

About

How We Build takase.com
A master calligrapher, one developer, and a team of AI roles. How the work is organized, how we check it, and why a human stays in the middle.