<p>I had Claude credits left in my week and wanted to secure my agent-dediced macbook. It
ran on my Mac M1
for a few days while I was on vacation. I made it kind of fancy with Discord support and
remote management through tailscale, it was really fun getting notified every now and
then that another commit was pushed to the open PR.</p>
<p>It used a DIY mvp version of a <a href="https://github.com/snarktank/ralph">Ralph loop</a>.
I am the supply chain! ...The bots tell me they won't draw Simpsons characters for me
and I'm not about to break out an image editor, so we get this footless child instead.</p>
<p>Look at the commits in <a href="https://github.com/kylep/multi/pull/52">this PR</a>, over 80!
A common thing about AI is you get all these impressive stats that way oversell the value
that they represent. These 80 commits are cool but if I'd done it by hands it would have
been like... 4, maybe 5.</p>
<p>As another win, I used my PRD and DD agents from the
<a href="/ai-native-sdlc-first-try.html">prior post</a> to design this out and
that worked wonderfully.</p>
<h2 id="ralph">Ralph?</h2>
<p>Ralph is an autonomous agent pattern that runs Claude Code in a loop
until a task list is empty. The task list here was dynamic, Claude kept looking for new
ways to improve laptop security without hurting agent autonomy. Later it crunched
through security findings from compliance scanners. The agent read each
finding, fixed it through Ansible, verified the fix, and
moved to the next one. I steered it from Discord a bit.</p>
<h1 id="the-setup">The setup</h1>
<p>The machine is a Mac M1 running as an AI workstation. This is "Claude's laptop", and I
factory reset it to remove anything Claude shouldn't have. I segmented it from my
network too, just in case.</p>
<p>Claude Code runs in bypass-permissions mode with
unrestricted tool access. The entire machine config is managed
by an <a href="https://github.com/kylep/multi/tree/main/infra/mac-setup">Ansible playbook</a>,
so any change the agent makes is reproducible on a fresh
install.</p>
<p>Before this project, I'd done some security work setting up pre-commit hooks for
semgrep and gitleaks, a <a href="/ai-security-toolkit.html">security toolkit</a> Docker image, and
hook scripts that blocked known-bad commands.</p>
<h1 id="loop-claude-go-do-security">Loop: Claude, go "do security"</h1>
<p>This loop was <em>okay, I guess</em>. The idea of the loop is really exciting. I can't wait to
get a loop of, like "work on anything in Linear" or something similarly broad that is
able to decompose stuff properly.</p>
<p>Anyways, for this project, the focus was limited to laptop security.
A Python script spawns Claude Code every few minutes to keep in the 5-hour token window.
We never really went above 50%. I had a custom token use mcp counting things up too,
but the math is fuzzy, especially since Claude was offering double quota at night that
week.</p>
<p>Each iteration:</p>
<ol>
<li>Tries to find something to do to improve security (tool or LLM decides)</li>
<li>Pick a finding that hasn't been attempted</li>
<li>Fix it through the Ansible playbook (not ad-hoc shell)</li>
<li>Run the playbook to deploy the fix</li>
<li>Verify the fix took effect</li>
<li>Commit and push</li>
</ol>
<pre class="mermaid">
stateDiagram-v2
    [*] --> Ideate
    Ideate --> PickFinding
    PickFinding --> FixInPlaybook
    FixInPlaybook --> Deploy
    Deploy --> Verify
    Verify --> Commit: pass
    Verify --> PickFinding: fail (retry)
    Commit --> Sleep
    Sleep --> ReadLog
</pre>
<p>The loop had those cost/token controls, a lock file so
only one instance ran at a time, and a timeout per
phase to kill things when they get stuck. It had an adversarial subagent loop, I love those, to
make sure that the changes actually worked, helped security, and did not hurt agent autonomy.
It pushed to GitHub and pinged me on Discord after each verified commit, so I could review from my phone.</p>
<h2 id="llm-only-no-tools">LLM only, no tools</h2>
<blockquote>
<p>"Claude, find me some ways to make my laptop secure and implement the best one through Ansible".</p>
</blockquote>
<p>It found a few real, if basic, issues. Things like:</p>
<ul>
<li>The application firewall was off</li>
<li>A <code>sudoers.d</code> file granted passwordless sudo to the agent user</li>
<li><code>exports.sh</code> and <code>.mcp.json</code> were world-readable (mode 0644)</li>
<li>Screen lock wasn't configured</li>
</ul>
<p>But 60 iterations later, most of the time had gone to things that didn't matter or couldn't work.</p>
<h2 id="the-loop-gets-stuck">The loop gets stuck</h2>
<p>It turns out you need to be proactive about preventing loops.</p>
<p>During this test, Claude spent multiple days iterating on a hook script called <code>protect-sensitive.sh</code>.
It'd move on then just keep coming back for it. The hook itself just tried to prevent the agent from reading sensitive files.
I had given some instructions to not get stuck in loops like that, but I guess it didn't take.</p>
<h2 id="explain-yourself-claude-agent-rcas">Explain Yourself, Claude: Agent RCAs</h2>
<p>After 68 commits in the week, I had Claude run an RCA.</p>
<blockquote>
<p>LLM intuition is a weak security discovery mechanism. The
loop should execute against a scanner's scored finding list,
not improvise the list itself.</p>
</blockquote>
<p>The result Looked really good, 68 commits in 3 days is cool stat, but the actual value
delivered was pretty meh. You need to be really intentional and good about goal-setting.</p>
<p>The RCA asked if it could just use a scanner like I did with the SCA stuff and that
immediately resulted in way more fixes and a much better loop.</p>
<h1 id="scanner-driven-task-lists-are-way-better">Scanner-driven task lists are way better</h1>
<p>The RCA recommended three scanners:</p>
<table>
<thead>
<tr>
<th>Tool</th>
<th>Scope</th>
<th>Why</th>
</tr>
</thead>
<tbody>
<tr>
<td><a href="https://cisofy.com/lynis/">Lynis</a></td>
<td>General system audit</td>
<td>Broad, scored, Linux-first but works on macOS</td>
</tr>
<tr>
<td><a href="https://rkhunter.sourceforge.net/">rkhunter</a></td>
<td>Rootkit detection</td>
<td>Catches things Lynis doesn't check</td>
</tr>
<tr>
<td><a href="https://github.com/usnistgov/macos_security">mSCP</a></td>
<td>macOS CIS Level 1</td>
<td>NIST-maintained, macOS-native compliance checks</td>
</tr>
</tbody>
</table>
<p>All three install through the Ansible playbook. A LaunchDaemon
runs them daily at 06:00 and writes results to
<code>/var/log/security-scans/</code>.</p>
<p>It was admittedly kind of a pain in the ass to help Claude escalate perms to root over
and over during the mSCP thing, some of those tests need root to run and Claude couldn't
trigger them itself.</p>
<h1 id="whatd-it-get-me">What'd it get me?</h1>
<p>Other than the fun of having it work while I was away, here's what Claude tells me it
implemented to secure my laptop from this. Worth noting that this was all done through
Ansible, so when I factory reset my laptop again (I will), I get this back too:</p>
<ul>
<li>Enabled the macOS application firewall and stealth mode to block unsolicited inbound connections</li>
<li>Enabled Gatekeeper to enforce code signing and notarization on all executables</li>
<li>Disabled SSH root login and enforced protocol v2 only</li>
<li>Locked down <code>sshd_config</code> permissions (0600) so it can't be read by other users</li>
<li>Set screen lock to trigger after 10 minutes idle, requiring password immediately</li>
<li>Disabled the guest account entirely</li>
<li>Disabled automatic login</li>
<li>Removed password hints from the login window</li>
<li>Enabled automatic app updates</li>
<li>Disabled AirDrop to prevent unsolicited file transfer attempts</li>
<li>Disabled AirPlay receiver so the machine can't accept screen share requests</li>
<li>Disabled Siri data sharing and search query telemetry</li>
<li>Disabled assistive voice data donation</li>
<li>Disabled diagnostic report submission to Apple</li>
<li>Disabled AI writing tools, mail summaries, Notes transcription, and external intelligence integrations</li>
<li>Disabled personalized advertising</li>
<li>Forced on-device-only dictation (no audio sent to Apple servers)</li>
<li>Enabled secure keyboard entry in Terminal to prevent other processes from reading keystrokes</li>
<li>Installed rkhunter for rootkit detection, running daily</li>
<li>Installed Lynis for system audit scoring, running daily</li>
<li>Installed mSCP for CIS Level 1 compliance checks, running daily</li>
</ul>
<p>So overall, pretty meh. It was fun though and I think if done well and let to run
perpetually, it could be a neat approach to having an agent actively defend my system.</p>


<h2 id="table-of-contents">Table of contents</h2>
<ul>
<li><a href="#the-goal">The goal</a></li>
<li><a href="#the-journalist-agent">The journalist agent</a></li>
<li><a href="#how-the-pieces-connect">How the pieces connect</a></li>
<li><a href="#controller-and-runtime">Controller and runtime</a></li>
<li><a href="#agenttask-crd">AgentTask CRD</a></li>
<li><a href="#openrouter-for-api-auth">OpenRouter for API auth</a></li>
<li><a href="#permissions-with-allowedtools">Permissions with allowedTools</a></li>
<li><a href="#webhook-trigger-on-demand">Webhook: trigger on demand</a></li>
<li><a href="#helm-chart-and-deployment">Helm chart and deployment</a></li>
</ul>
<h2 id="the-goal">The goal</h2>
<p>I want AI agents that run autonomously on a cron schedule or in
response to events, managed by Kubernetes.</p>
<p>The demo task is a daily AI news digest. Every morning at 8am,
an agent searches the web for yesterday's AI news, writes a
digest to the wiki, and commits. It's also triggerable on demand
via webhook.</p>
<p>K8s handles the scheduling. A custom controller watches a CRD
and creates Jobs. The agent runs in a container, writes to a
shared volume, and the controller tracks the result. Agents
commit locally but don't push. I review and push manually.</p>
<h2 id="the-journalist-agent">The journalist agent</h2>
<p>I wrote a new Claude Code agent for this:
<a href="https://github.com/kylep/multi/blob/main/.claude/agents/journalist.md">journalist</a>.
Sonnet, not Opus. This runs daily, and the task is search and
summarize. Sonnet is cheaper and good enough.</p>
<p>The agent searches for yesterday's AI news, writes a digest to
<code>wiki/journal/YYYY-MM-DD/ai-news.md</code> with source URLs for every
item, and commits. It determines the date itself via <code>date</code>.</p>
<p>Early versions used <code>WebSearch</code>, which burned through context
with HTML scraping. I built a small
<a href="/wiki/custom-tools/google-news-mcp.html">GNews MCP server</a>
that wraps the GNews API. Structured JSON responses, fewer
tokens. The agent runs six parallel queries (<code>OpenAI</code>,
<code>Anthropic</code>, <code>Google AI</code>, <code>NVIDIA AI</code>, <code>AI startup</code>, plus
tech headlines) and deduplicates the results.</p>
<p>I also wrote a
<a href="/wiki/custom-tools/discord-mcp.html">Discord MCP server</a>
so the agent posts its digest to the <code>#news</code> channel in my
Discord bot server.</p>
<h2 id="how-the-pieces-connect">How the pieces connect</h2>
<pre class="mermaid">
graph TD
    AT[daily-ai-news AgentTask] -->|watched by| CTRL[Controller]
    CTRL -->|8am cron| JOB[K8s Job]
    JOB -->|init| GIT[git fetch + reset]
    GIT -->|then| AGENT[Journalist Agent]
    AGENT -->|writes| WIKI[wiki/journal/YYYY-MM-DD/ai-news.md]
    AGENT -->|commits| PVC[Shared PVC]
    AGENT -->|posts via MCP| DISCORD[Discord #news]
    WH[Webhook :8080] -->|creates| AT
</pre>
<p>The controller is a Go binary running as a Deployment. It watches
<code>AgentTask</code> custom resources in the <code>ai-agents</code> namespace.</p>
<p>Each Job has two containers:</p>
<ol>
<li><strong>Init container</strong> (<code>alpine/git</code>): fetches the repo, resets
to the configured branch, then <code>chown</code>s the workspace to
uid 1000 (the agent user).</li>
<li><strong>Main container</strong> (<code>ai-agent-runtime</code>): writes MCP config,
then runs <code>claude --mcp-config ... --agent journalist -p "..." --allowedTools ...</code></li>
</ol>
<h2 id="controller-and-runtime">Controller and runtime</h2>
<p>The
<a href="https://github.com/kylep/multi/tree/main/infra/ai-agents/agent-controller/pkg/controller">controller</a>
is a Go reconcile loop that lists all AgentTasks every 30 seconds.
For scheduled tasks, it compares <code>lastRunTime</code> against the cron
expression and creates a Job when due. For manual and webhook
tasks, it creates a Job immediately when phase is <code>Pending</code>.</p>
<p>The runtime image is an Alpine container with Claude Code and
OpenCode baked in (<code>infra/ai-agents/ai-agent-runtime/Dockerfile</code>). Runs
as uid 1000.</p>
<h2 id="agenttask-crd">AgentTask CRD</h2>
<p>The custom resource definition gives <code>kubectl get agenttasks</code>
for free. Here's the daily news task:</p>
<pre><code class="language-yaml">apiVersion: agents.kyle.pericak.com/v1alpha1
kind: AgentTask
metadata:
  name: daily-ai-news
spec:
  agent: journalist
  runtime: claude
  prompt: >-
    Search for yesterday's most notable AI news...
  schedule: "0 8 * * *"
  trigger: scheduled
  readOnly: false
  allowedTools: >-
    WebSearch,WebFetch,Read,Glob,Grep,Write,
    Bash(mkdir *),Bash(git add *),
    Bash(git commit *),Bash(date *)
</code></pre>
<p>The controller splits <code>allowedTools</code> on commas and passes each
one as a separate <code>--allowedTools</code> flag to Claude Code. This is
how you run Claude Code autonomously without
<code>--dangerously-skip-permissions</code>.</p>
<table>
<thead>
<tr>
<th>Field</th>
<th>Purpose</th>
</tr>
</thead>
<tbody>
<tr>
<td><code>agent</code></td>
<td>Name from <code>.claude/agents/</code></td>
</tr>
<tr>
<td><code>runtime</code></td>
<td><code>claude</code> or <code>opencode</code></td>
</tr>
<tr>
<td><code>prompt</code></td>
<td>The <code>-p</code> argument</td>
</tr>
<tr>
<td><code>schedule</code></td>
<td>Cron expression (empty = one-shot)</td>
</tr>
<tr>
<td><code>trigger</code></td>
<td><code>manual</code>, <code>scheduled</code>, or <code>webhook</code></td>
</tr>
<tr>
<td><code>readOnly</code></td>
<td>Whether this agent modifies the repo</td>
</tr>
<tr>
<td><code>allowedTools</code></td>
<td>Comma-separated tool permissions</td>
</tr>
</tbody>
</table>
<h2 id="openrouter-for-api-auth">OpenRouter for API auth</h2>
<p>Claude Code normally uses <code>ANTHROPIC_API_KEY</code> to talk to
Anthropic's API directly. In a container, I route it through
<a href="/openrouter-ai-tools.html">OpenRouter</a> instead. One API key
for all model providers, unified billing.</p>
<p>The Helm chart's Secret injects these env vars into every
agent container:</p>
<pre><code class="language-yaml">ANTHROPIC_BASE_URL: https://openrouter.ai/api
ANTHROPIC_AUTH_TOKEN: &#x3C;openrouter-key>
ANTHROPIC_API_KEY: ""
CLAUDE_CODE_DISABLE_NONESSENTIAL_TRAFFIC: "1"
</code></pre>
<p><code>ANTHROPIC_API_KEY</code> must be empty or Claude Code tries to use
it and the auth conflicts.</p>
<h2 id="permissions-with-allowedtools">Permissions with allowedTools</h2>
<p>Claude Code prompts for permission before using tools. In a
container with no TTY, it just hangs. The fix is
<code>--allowedTools</code>, which auto-approves specific tools.</p>
<p>Each tool needs its own flag. Comma-separated doesn't work:</p>
<pre><code class="language-bash"># wrong
claude -p "..." --allowedTools "WebSearch,Write,Read"

# right
claude -p "..." \
  --allowedTools "WebSearch" \
  --allowedTools "Write" \
  --allowedTools "Read"
</code></pre>
<p>Tools with specifiers use parentheses for scoping:</p>
<pre><code class="language-bash">--allowedTools 'Bash(mkdir *)'    # only mkdir
--allowedTools 'Bash(git add *)'  # only git add
--allowedTools 'Write'            # any file
</code></pre>
<p>I tried scoping Write to a specific path with
<code>Write(apps/blog/blog/markdown/wiki/journal/**)</code> but it didn't
match inside the container. The pattern works locally but fails
when the working directory is <code>/workspace/repo</code>. Unrestricted
<code>Write</code> works. The agent definition and Bash scoping still
constrain what gets written.</p>
<h2 id="webhook-trigger-on-demand">Webhook: trigger on demand</h2>
<p>The controller exposes <code>:8080/webhook</code> for on-demand runs.
Requires a bearer token via <code>AI_WEBHOOK_TOKEN</code> env var (skipped
if empty for local dev):</p>
<pre><code class="language-bash">curl -X POST http://localhost:8080/webhook \
  -H "Authorization: Bearer $AI_WEBHOOK_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{
    "agent": "journalist",
    "prompt": "Search for today AI news...",
    "allowedTools": "WebSearch,WebFetch,Read,Write,..."
  }'
</code></pre>
<p>The handler creates an AgentTask CR with the allowedTools, and
the reconcile loop picks it up.</p>
<h2 id="helm-chart-and-deployment">Helm chart and deployment</h2>
<p>The
<a href="https://github.com/kylep/multi/tree/main/infra/ai-agents/agent-controller/helm">Helm chart</a>
packages everything: CRD, controller Deployment, ServiceAccount,
RBAC, Secrets, and PVC. hostPath volume, single node.</p>
<pre><code class="language-bash">helm install agent-controller ./helm \
  -n ai-agents --create-namespace \
  -f values-override.yaml
</code></pre>
<p>The OpenRouter key has special characters that <code>--set</code> mangles.
Use a values file instead:</p>
<pre><code class="language-yaml">secrets:
  openrouterApiKey: "&#x3C;your-openrouter-key>"
  discordBotToken: "&#x3C;your-discord-bot-token>"
  discordGuildId: "&#x3C;your-guild-id>"
repo:
  branch: kyle/blog-k8s-autonomous-agents-mvp
</code></pre>


<h2 id="table-of-contents">Table of contents</h2>
<ul>
<li><a href="#a-warning-this-didnt-work-well">A warning: This didn't work well</a></li>
<li><a href="#why-an-org-chart-for-ai-agents">Why an org chart for AI agents</a></li>
<li><a href="#the-agents">The agents</a></li>
<li><a href="#the-org-chart">The org chart</a></li>
<li><a href="#pai-the-executive-assistant">Pai: the executive assistant</a>
<ul>
<li><a href="#agent-invocation-via-bash">Agent invocation via Bash</a></li>
</ul>
</li>
<li><a href="#the-wiki-layer">The wiki layer</a></li>
<li><a href="#pai-in-action">Pai in action</a>
<ul>
<li><a href="#the-quarterly-health-check">The quarterly health check</a></li>
</ul>
</li>
</ul>
<h1 id="a-warning-this-didnt-work-well">A warning: This didn't work well</h1>
<p>I found the process of building and testing this really fun. Trying to actually use it
afterwards was less fun.</p>
<ul>
<li><code>2026-03-11</code>: this is a "Hey this is neat" thing, not a  "this works really well!" one.</li>
<li><code>2026-03-15</code>: I've deleted basically all these agents and am down to 7.</li>
<li>I haven't given up on this idea but this approach was too ambitious and didn't have
the other infra in place it'd need to succeed.</li>
</ul>
<h1 id="why-an-org-chart-for-ai-agents">Why an org chart for AI agents</h1>
<p>Because it's cool, mostly. Also I'd hoped it would work really well.
But mostly, it's cool.</p>
<p>In this experiment I defined a team of Claude Code agents that each had a named role,
themed around being a virtual agent company. I figured I could run them manually or in
K8s using the Claude CLI:</p>
<pre><code class="language-bash">claude --agent cmo -p "What are my top 5 posts by pageviews?"
</code></pre>
<p>In theory I could manage agent context with this approach effectively and get each one
to stay in its own lane without bloating Claude's global context. CMO started with CMO
prompt, CDO with CDO prompt, so on.</p>
<p>The hope was to have these things each coordinate on their own, with my Pai agent acting
as a top-level orchestrator. They should have been able to use the wiki to exchange context.</p>
<p>I repurposed my OpenClaw agent
<a href="/openclaw-linear-skill.html">Pai</a>'s name when defining my orchestrator. I hoped to be
able to ask for things in Telegram and Pai would decompose the request into a
multi-agent flow and orchestrate from there.</p>
<p>The C-suite agents (CMO, CFO, CTO, CDO, CSO) each were envisioned as sub-orchestrators
that ran their own adversarial loops to keep standards high and sub-route requests.</p>
<h1 id="the-agents">The agents</h1>
<table>
<thead>
<tr>
<th>Agent</th>
<th>Role</th>
<th>Model</th>
<th>Key Tools</th>
</tr>
</thead>
<tbody>
<tr>
<td>Pai</td>
<td>Orchestration</td>
<td>Sonnet</td>
<td>Bash, Linear MCP</td>
</tr>
<tr>
<td>CMO</td>
<td>Traffic and growth</td>
<td>Sonnet</td>
<td>GA4 Analytics MCP</td>
</tr>
<tr>
<td>CFO</td>
<td>AI spend</td>
<td>Sonnet</td>
<td><a href="/openrouter-ai-tools.html">OpenRouter MCP</a></td>
</tr>
<tr>
<td>CTO</td>
<td>Delivery, blockers</td>
<td>Sonnet</td>
<td>Linear MCP, Bash</td>
</tr>
<tr>
<td>CDO</td>
<td>Knowledge management</td>
<td>Sonnet</td>
<td>Wiki read/write, Bash</td>
</tr>
<tr>
<td>CSO</td>
<td>Security and privacy</td>
<td>Sonnet</td>
<td>File tools, Bash</td>
</tr>
<tr>
<td>AR</td>
<td>Agent onboarding, mediation</td>
<td>Sonnet</td>
<td>File tools, Bash</td>
</tr>
<tr>
<td>Publisher</td>
<td>Blog content pipeline</td>
<td>Sonnet</td>
<td>Bash, file tools</td>
</tr>
<tr>
<td>SEO</td>
<td>Search audits</td>
<td>Sonnet</td>
<td>GA4, WebSearch</td>
</tr>
<tr>
<td>Cost Tracker</td>
<td>Spend reports</td>
<td>Haiku</td>
<td>OpenRouter MCP</td>
</tr>
<tr>
<td>Librarian</td>
<td>Wiki read/write</td>
<td>Haiku</td>
<td>Wiki file tools</td>
</tr>
<tr>
<td>Privacy Auditor</td>
<td>Flag confidential data</td>
<td>Haiku</td>
<td>File tools</td>
</tr>
<tr>
<td>Researcher</td>
<td>Gather sourced facts</td>
<td>Sonnet</td>
<td>WebSearch, Read</td>
</tr>
<tr>
<td>Writer</td>
<td>Draft posts from briefs</td>
<td>Sonnet</td>
<td>Read, Write</td>
</tr>
<tr>
<td>Fact Checker</td>
<td>Verify claims</td>
<td>Haiku</td>
<td>WebSearch, Read</td>
</tr>
<tr>
<td>Reviewer</td>
<td>Style and structure</td>
<td>Haiku</td>
<td>Read, Grep</td>
</tr>
<tr>
<td>QA</td>
<td>Production readiness</td>
<td>Sonnet</td>
<td>Bash, Playwright</td>
</tr>
</tbody>
</table>
<p>My only real use case at the time was writing blog posts, so the Publisher was special,
getting its own sub-org with a four-stage content
pipeline: researcher, writer, fact-checker, and reviewer.</p>
<p>QA ran after the pipeline to verify the post actually
built, rendered, and linked correctly before it went live.
QA used <a href="/playwright-mcp.html">Playwright MCP</a> to take screenshots
and inspect the rendered page.</p>
<p>Every agent connected to real
<a href="https://modelcontextprotocol.io/">MCP</a> servers to do cool stuff. The
CMO queried real GA4 data. The CFO pulled real OpenRouter
bills.</p>
<h1 id="the-org-chart">The org chart</h1>
<p>This was accurate as of 2026-03-12. The
<a href="/wiki/projects/agent-team/org-chart.html">bot-wiki org chart</a>
stayed up to date as agents were added or reorganized.</p>
<pre class="mermaid">
graph TD
    Kyle["Kyle"]
    Pai["Pai: Executive Assistant"]
    CMO["CMO: Grow Readership"]
    CFO["CFO: Optimize Spend"]
    CTO["CTO: Delivery"]
    CDO["CDO: Knowledge"]
    CSO["CSO: Security"]
    AR["AR: Agent Resources"]
    Publisher["Publisher: Content Pipeline"]

    SEO["SEO Subagent"]
    CostTracker["Cost Tracker"]
    Librarian["Librarian"]
    PrivacyAuditor["Privacy Auditor"]

    Researcher["Researcher"]
    Writer["Writer"]
    FactChecker["Fact Checker"]
    Reviewer["Reviewer"]
    QA["QA"]

    Kyle --> Pai
    Kyle --> CMO
    Kyle --> CFO
    Kyle --> CTO
    Kyle --> CDO
    Kyle --> CSO
    Kyle --> AR
    Kyle --> Publisher

    Pai -.->|orchestrates| CMO
    Pai -.->|orchestrates| CFO
    Pai -.->|orchestrates| CTO
    Pai -.->|orchestrates| CDO
    Pai -.->|orchestrates| CSO
    Pai -.->|orchestrates| AR
    Pai -.->|orchestrates| Publisher

    CMO --> SEO
    CFO --> CostTracker
    CDO --> Librarian
    CSO --> PrivacyAuditor

    CMO -.->|reads/writes| Librarian
    CFO -.->|reads/writes| Librarian
    CTO -.->|reads/writes| Librarian

    Publisher --> Researcher
    Publisher --> Writer
    Publisher --> FactChecker
    Publisher --> Reviewer
    Publisher --> QA
</pre>
<p>Every agent was directly
invocable by me, but ideally I didn't need to do that often.</p>
<p>Solid lines meant "reports to." Dashed lines from Pai meant
"orchestrates." Dashed lines to the Librarian meant "can
read/write wiki through."</p>
<p>The Privacy Auditor was the gatekeeper. Any agent writing
content that would end up in git or on the internet should
have checked with the Privacy Auditor first. It scanned for
leaked analytics data, spend numbers, secrets, and anything
else that shouldn't be public. This came from a scare where one of my agents with access
to mildly confidential data referenced it in content that
would have been public. I feel strongly that all agent flows
need a privacy step at minimum, and ideally agents should
have purposefully curtailed access to sensitive data.</p>
<p>The Librarian was an experiment in context management and
shared state. Any agent could talk to it directly to persist
notes, plans, or evidence to the wiki. I had already built a
<a href="/wiki-rag.html">RAG system</a> over the wiki. Once the wiki
got big enough that agents couldn't skim it all, I'd have
them query the RAG index instead. At that point, the agents
just used git and local file storage for wiki access. The CDO
owned the wiki strategy, but the Librarian did the actual
reading and writing.</p>
<h1 id="pai-the-executive-assistant">Pai: the executive assistant</h1>
<p>Agent definitions lived in <code>.claude/agents/</code>. Here was Pai's
frontmatter:</p>
<pre><code class="language-yaml"># .claude/agents/pai.md
name: pai
description: >-
  Pai — Executive assistant that orchestrates
  multi-agent workflows
model: sonnet
tools:
  - Bash
  - Read
  - Glob
  - Grep
  - Write
  - mcp__linear-server__list_issues
  - mcp__linear-server__list_projects
</code></pre>
<h2 id="agent-invocation-via-bash">Agent invocation via Bash</h2>
<p>Pai invoked other agents with <code>claude --agent &#x3C;name> -p "prompt"</code> through the Bash tool.
In theory I could have made an MCP for Pai that ran that in k8s pods for OpenClaw.
Each call was a fresh session. Agents didn't share memory or context with each other.</p>
<pre><code class="language-text">*Update*: The alternative I didn't try was just to use Claude Subagents for local work.
That's much easier and works better.
</code></pre>
<p>The alternatives I considered:</p>
<ul>
<li><strong>Shared memory / message bus.</strong> Tools like LangGraph or
CrewAI give agents shared state. More powerful, but more
moving parts. I wanted something I could debug by reading
a bash script.</li>
<li><strong>Single mega-agent.</strong> One agent definition with all the
tools. Simpler, but the context window filled up fast and
the agent lost focus. Splitting by role kept each
session lean.</li>
<li><strong>MCP-based orchestration.</strong> Route through an MCP server
that managed agent sessions. Interesting, but overkill
for a blog. I'd rather have built on <code>claude --agent</code>
which already worked.</li>
<li><strong><a href="https://github.com/steveyegge/beads">Beads</a>.</strong> A
structured task memory for AI agents, backed by Dolt
(version-controlled SQL). I looked into it, but it was
more of an alternative to Linear than to the wiki. It
solved task tracking and agent handoff, not knowledge
management. I liked Linear for tasks and markdown files
for knowledge.</li>
</ul>
<p>I went with the dumb approach: Bash calls and text passing.
It was easy to understand, easy to debug, and the wiki
handled long-term memory. If I outgrew it, I'd upgrade.</p>
<h1 id="the-wiki-layer">The wiki layer</h1>
<p>Each agent had a page in the
<a href="/bot-wiki.html">Bot-Wiki</a> that documented its goal, tools,
subagents, and example prompts.</p>
<pre><code class="language-text">agent-team/
├── index.md          # org chart and coordination model
├── pai.md            # orchestration agent
├── cmo.md            # traffic and growth
├── cfo.md            # AI spend
├── cto.md            # delivery and blockers
├── cdo.md            # knowledge management
├── cso.md            # security and privacy
├── publisher.md      # blog pipeline
└── phase-2.md        # future async architecture
</code></pre>
<p>Every agent session was stateless. The CMO didn't remember
what the CTO said last week. But if the CTO wrote its
findings to the wiki through the Librarian, the CMO could
read them next time it ran. The wiki was how agents shared
context across sessions.</p>
<p>The Librarian (a Haiku subagent under the CDO) handled
all wiki read/write operations. Any agent could invoke it
directly with <code>claude --agent librarian -p "..."</code> to
persist notes, plans, evidence, or whatever else needed to
survive between sessions.</p>
<p>The CDO owned the strategy: what got documented, how pages
were structured, when content was stale. This was cheaper
than giving every agent write access to the full file
system. The Librarian ran on Haiku, knew the wiki format,
and wouldn't accidentally clobber unrelated files.</p>
<p>In time I would have had the CFO tweak the models the
agents used to manage costs, too.</p>
<h1 id="pai-in-action">Pai in action</h1>
<h2 id="the-quarterly-health-check">The quarterly health check</h2>
<p>I asked Pai for a quarterly health check. It invoked three
agents and synthesized everything into one report.</p>
<p><strong>Traffic</strong> (from CMO): GA4 only had a week of data. The
site was new enough that there was no meaningful baseline yet.
Almost entirely direct traffic, organic search barely
registering. CMO recommended connecting Search Console and
revisiting in 30 days.</p>
<p><strong>AI spend</strong> (from CFO): well under budget with no pressure.
OpenRouter's API only surfaced account-level totals, no
per-model breakdown. CFO noted the biggest future lever was
routing non-reasoning tasks to flash-tier models.</p>
<p><strong>Delivery</strong> (from CTO): most issues done or in backlog,
one in progress (this blog post). No hard blockers, but
three dependency chains worth watching:</p>
<ol>
<li>RSS feed was high priority but hadn't started. Four
distribution issues were gated on it.</li>
<li>CMO and CFO baseline runs hadn't started. They gated
two future blog posts.</li>
<li>This post had an open PR to close.</li>
</ol>
<p>Pai wrote a log entry automatically:</p>
<pre><code class="language-markdown">## 21:30 — Quarterly health check

| Agent | Prompt Summary | Result |
|-------|---------------|--------|
| cmo | 90-day traffic trend | success |
| cfo | 90-day AI spend | success |
| cto | Blocked/stalled issues | success: 3 clusters |

**Synthesis:** Early launch phase, minimal spend,
no hard blockers but three dependency chains.
</code></pre>


<h2 id="table-of-contents">Table of contents</h2>
<ul>
<li><a href="#claude-code-hooks">Claude Code hooks</a></li>
<li><a href="#how-hooks-work">How hooks work</a></li>
<li><a href="#blocking-destructive-commands">Blocking destructive commands</a></li>
<li><a href="#protecting-sensitive-files">Protecting sensitive files</a></li>
<li><a href="#audit-trail">Audit trail</a></li>
<li><a href="#wiring-it-up">Wiring it up</a></li>
<li><a href="#prompt-injection-defense">Prompt injection defense</a></li>
<li><a href="#why-global-config-not-per-repo">Why global config, not per-repo</a></li>
<li><a href="#same-idea-different-tools">Same idea, different tools</a></li>
</ul>
<h1 id="claude-code-hooks">Claude Code hooks</h1>
<p>Claude Code runs shell commands, reads files, and edits code
on your machine. Hooks let you intercept those actions with
shell scripts that run before or after each tool call. The
scripts get JSON on stdin describing what Claude is about to
do, and they can block it by exiting with code 2.</p>
<p>I set up three hooks: one that blocks destructive commands,
one that blocks access to sensitive files, and one that logs
everything to a JSONL audit trail.</p>
<h1 id="how-hooks-work">How hooks work</h1>
<p>A hook is a shell script registered in
<code>~/.claude/settings.json</code>. There are two lifecycle events:</p>
<ul>
<li><strong>PreToolUse</strong>: runs before Claude executes a tool. Exit 0
to allow, exit 2 to block. The block message goes to stderr.</li>
<li><strong>PostToolUse</strong>: runs after. Good for logging. Can run async
so it doesn't slow things down.</li>
</ul>
<p>Each hook gets a JSON object on stdin with <code>tool_name</code>,
<code>tool_input</code>, <code>session_id</code>, and <code>cwd</code>. You can use <code>jq</code> to
pull out whatever you need.</p>
<p>A <code>matcher</code> field controls which tools trigger the hook. It
takes a regex, so <code>Bash</code> matches only the Bash tool while
<code>Read|Edit|Write|Bash</code> matches all four.</p>
<h1 id="blocking-destructive-commands">Blocking destructive commands</h1>
<p>This one intercepts every Bash tool call and blocks anything
that could wreck the system.</p>
<pre><code class="language-bash">#!/usr/bin/env bash
# ~/.claude/hooks/block-destructive.sh
# PreToolUse hook, matcher: Bash

set -euo pipefail

INPUT=$(cat)
COMMAND=$(echo "$INPUT" | jq -r '.tool_input.command // empty')

if [[ -z "$COMMAND" ]]; then
  exit 0
fi

BLOCKED=""

case "$COMMAND" in
  *"rm -rf /"*|*"rm -rf ~"*)
    BLOCKED="recursive delete of root or home" ;;
  *"git push --force"*|*"git push -f "*)
    BLOCKED="force push" ;;
  *"git reset --hard"*)
    BLOCKED="hard reset" ;;
  *"DROP TABLE"*|*"DROP DATABASE"*)
    BLOCKED="database drop" ;;
  *":(){ :|:&#x26; };:"*)
    BLOCKED="fork bomb" ;;
  *"curl"*"|"*"sh"*|*"curl"*"|"*"bash"*)
    BLOCKED="piped remote execution" ;;
  *"chmod 777"*)
    BLOCKED="world-writable permissions" ;;
  *"mkfs."*)
    BLOCKED="filesystem format" ;;
esac

# Check dd writing to /dev/
if [[ -z "$BLOCKED" ]] &#x26;&#x26; echo "$COMMAND" | \
   grep -qE 'dd\s+if=.*of=/dev/'; then
  BLOCKED="raw device write"
fi

if [[ -n "$BLOCKED" ]]; then
  echo "BLOCKED by block-destructive hook: $BLOCKED" >&#x26;2
  exit 2
fi

exit 0
</code></pre>
<p>Testing it:</p>
<pre><code class="language-bash">$ echo '{"tool_name":"Bash","tool_input":{"command":"rm -rf /"}}' \
  | ~/.claude/hooks/block-destructive.sh
BLOCKED by block-destructive hook: recursive delete of root or home
# exit code: 2

$ echo '{"tool_name":"Bash","tool_input":{"command":"git push --force origin main"}}' \
  | ~/.claude/hooks/block-destructive.sh
BLOCKED by block-destructive hook: force push
# exit code: 2

$ echo '{"tool_name":"Bash","tool_input":{"command":"ls -la"}}' \
  | ~/.claude/hooks/block-destructive.sh
# exit code: 0  (allowed)
</code></pre>
<p>When Claude tries a blocked command, it sees the stderr
message and knows the action was rejected. It can then
rephrase or ask for confirmation.</p>
<h1 id="protecting-sensitive-files">Protecting sensitive files</h1>
<p>The second hook blocks access to credentials, SSH keys, and
cloud configs. It handles both file-path tools (Read, Edit,
Write) and Bash commands that might <code>cat</code> or exfiltrate
those files.</p>
<pre><code class="language-bash">#!/usr/bin/env bash
# ~/.claude/hooks/protect-sensitive.sh
# PreToolUse hook, matcher: Read|Edit|Write|Bash

set -euo pipefail

INPUT=$(cat)
TOOL=$(echo "$INPUT" | jq -r '.tool_name // empty')

check_path() {
  local filepath="$1"
  case "$filepath" in
    */.env|*/.env.*|*.env)
      echo "BLOCKED by protect-sensitive hook: .env file" >&#x26;2
      exit 2 ;;
    */.ssh/id_*)
      echo "BLOCKED by protect-sensitive hook: SSH key" >&#x26;2
      exit 2 ;;
    */.aws/credentials*)
      echo "BLOCKED by protect-sensitive hook: AWS creds" >&#x26;2
      exit 2 ;;
    */.kube/config*)
      echo "BLOCKED by protect-sensitive hook: kubeconfig" >&#x26;2
      exit 2 ;;
  esac
}

if [[ "$TOOL" == "Bash" ]]; then
  COMMAND=$(echo "$INPUT" | jq -r '.tool_input.command // empty')
  if echo "$COMMAND" | \
     grep -qE '(cat|less|head|tail|curl -d @|base64|scp)\s+\.env'; then
    echo "BLOCKED by protect-sensitive hook: .env access via bash" >&#x26;2
    exit 2
  fi
  if echo "$COMMAND" | \
     grep -qE '(cat|less|head|tail)\s+.*(\.ssh/id_|\.aws/credentials|\.kube/config)'; then
    echo "BLOCKED by protect-sensitive hook: sensitive file access via bash" >&#x26;2
    exit 2
  fi
else
  FILEPATH=$(echo "$INPUT" | jq -r '.tool_input.file_path // empty')
  if [[ -n "$FILEPATH" ]]; then
    check_path "$FILEPATH"
  fi
fi

exit 0
</code></pre>
<pre><code class="language-bash">$ echo '{"tool_name":"Read","tool_input":{"file_path":"/home/user/.ssh/id_ed25519"}}' \
  | ~/.claude/hooks/protect-sensitive.sh
BLOCKED by protect-sensitive hook: SSH key
# exit code: 2

$ echo '{"tool_name":"Bash","tool_input":{"command":"cat .env"}}' \
  | ~/.claude/hooks/protect-sensitive.sh
BLOCKED by protect-sensitive hook: .env access via bash
# exit code: 2
</code></pre>
<p>This isn't foolproof. Claude could read a sensitive file
by some path I didn't think of. But it catches the obvious
cases and raises the bar.</p>
<h1 id="audit-trail">Audit trail</h1>
<p>The third hook runs after every tool call and appends a JSON
line to a log file. It runs async so it doesn't slow Claude
down.</p>
<pre><code class="language-bash">#!/usr/bin/env bash
# ~/.claude/hooks/audit-log.sh
# PostToolUse hook, async

LOG="/Users/kp/gh/multi/logs/claude-audit.jsonl"

INPUT=$(cat)
TOOL=$(echo "$INPUT" | jq -r '.tool_name // "unknown"')
SESSION=$(echo "$INPUT" | jq -r '.session_id // "unknown"')
CWD=$(echo "$INPUT" | jq -r '.cwd // "unknown"')

case "$TOOL" in
  Bash)
    PARAM=$(echo "$INPUT" | jq -r '.tool_input.command // empty') ;;
  Read|Edit|Write)
    PARAM=$(echo "$INPUT" | jq -r '.tool_input.file_path // empty') ;;
  *)
    PARAM="" ;;
esac

jq -nc \
  --arg ts "$(date -u +%Y-%m-%dT%H:%M:%SZ)" \
  --arg sid "$SESSION" \
  --arg tool "$TOOL" \
  --arg param "$PARAM" \
  --arg cwd "$CWD" \
  '{timestamp: $ts, session_id: $sid, tool: $tool, param: $param, cwd: $cwd}' \
  >> "$LOG"

exit 0
</code></pre>
<p>After a session, the log looks like:</p>
<pre><code class="language-json">{"timestamp":"2026-03-11T13:14:18Z","session_id":"abc123","tool":"Bash","param":"echo hello","cwd":"/Users/kp/gh/multi"}
</code></pre>
<p>You could pipe this into whatever alerting or analysis you
want. I'm keeping it simple for now, just a file on disk.</p>
<p>The end goal is to use this log to build something like
<a href="https://github.com/ldayton/Dippy">Dippy</a>, a smart
auto-approve hook. Dippy parses bash commands with an AST
and auto-approves safe ones while still prompting for
anything destructive. I want to do the same thing but
trained on my own audit trail, so the approve/deny decisions
reflect how I actually work. That's a future post.</p>
<h1 id="wiring-it-up">Wiring it up</h1>
<p>All three hooks go in <code>~/.claude/settings.json</code> under a
<code>hooks</code> key:</p>
<pre><code class="language-json">{
  "hooks": {
    "PreToolUse": [
      {
        "matcher": "Bash",
        "hooks": [{
          "type": "command",
          "command": "/Users/kp/.claude/hooks/block-destructive.sh"
        }]
      },
      {
        "matcher": "Read|Edit|Write|Bash",
        "hooks": [{
          "type": "command",
          "command": "/Users/kp/.claude/hooks/protect-sensitive.sh"
        }]
      }
    ],
    "PostToolUse": [
      {
        "hooks": [{
          "type": "command",
          "command": "/Users/kp/.claude/hooks/audit-log.sh",
          "async": true
        }]
      }
    ]
  }
}
</code></pre>
<p>Use absolute paths, not <code>~</code>. JSON doesn't expand tilde.</p>
<p>The <code>async: true</code> on the audit hook means Claude doesn't
wait for it to finish before moving on. Logging shouldn't
slow down the workflow.</p>
<h1 id="prompt-injection-defense">Prompt injection defense</h1>
<p>One thing I haven't built yet but plan to: a PostToolUse
hook that scans tool output for prompt injection attempts.</p>
<p>The idea is to grep the output for patterns like "ignore
previous instructions" or "you are now in admin mode" and
flag them. This matters most for tools that ingest external
data. If Claude is reading web pages, parsing documents,
or pulling content from APIs, a malicious payload in that
content could try to hijack the session.</p>
<p>Lasso Security published a
<a href="https://www.lasso.security/blog/prompt-injection-taxonomy-techniques">prompt injection taxonomy</a>
cataloging these techniques: encoding, obfuscation,
role-playing, context manipulation, and more. A PostToolUse
hook is the right place to catch
them because you see the actual content before Claude acts
on it.</p>
<p>I'll cover this in a future post once I have it running on
real workloads.</p>
<h1 id="why-global-config-not-per-repo">Why global config, not per-repo</h1>
<p>Claude Code hooks can be configured at three levels:</p>
<ul>
<li><code>~/.claude/settings.json</code> (global, user-level)</li>
<li><code>.claude/settings.json</code> (per-repo, checked into git)</li>
<li><code>.claude/settings.local.json</code> (per-repo, gitignored)</li>
</ul>
<p>Safety hooks belong in global config. If they're per-repo,
a malicious repository can ship its own <code>.claude/settings.json</code>
that disables your protections or adds hooks that exfiltrate
data. You clone it, run Claude Code, and the repo's hooks
run instead of yours.</p>
<p>This is a real attack vector. In 2025, Check Point Research
found
<a href="https://research.checkpoint.com/2026/rce-and-api-token-exfiltration-through-claude-code-project-files-cve-2025-59536/">CVE-2025-59536</a>:
an attacker plants a <code>.claude/settings.json</code> with hooks in
a repo. You clone it, run Claude Code, approve the trust
dialog (which doesn't mention hooks), and the hook commands
execute immediately. Reverse shell, data exfiltration,
whatever the attacker wants. Anthropic patched it in August
2025, but the lesson stands: keep safety hooks in global
config where only you control them.</p>
<h1 id="same-idea-different-tools">Same idea, different tools</h1>
<p>Claude Code isn't the only AI coding tool with hook support.
Here's how the same concepts map elsewhere.</p>
<table>
<thead>
<tr>
<th>Feature</th>
<th>Claude Code</th>
<th>Cursor</th>
</tr>
</thead>
<tbody>
<tr>
<td>Hook format</td>
<td>Shell scripts</td>
<td>Shell scripts</td>
</tr>
<tr>
<td>Config</td>
<td><code>~/.claude/settings.json</code></td>
<td><code>.cursor/hooks.json</code></td>
</tr>
<tr>
<td>Events</td>
<td><code>PreToolUse</code>, <code>PostToolUse</code></td>
<td><code>preToolUse</code>, <code>postToolUse</code></td>
</tr>
<tr>
<td>Input</td>
<td>JSON on stdin</td>
<td>JSON on stdin</td>
</tr>
<tr>
<td>Block mechanism</td>
<td>Exit code 2 + stderr</td>
<td>Exit code 2 + stderr</td>
</tr>
<tr>
<td>Async support</td>
<td><code>"async": true</code></td>
<td>Not documented</td>
</tr>
</tbody>
</table>
<p>Cursor added hooks in version 1.7 (October 2025) with a
nearly identical model: shell scripts, JSON stdin, exit
codes. It also supports a "prompt" handler type where an
LLM evaluates a natural language condition to allow or deny,
which is interesting but slower. The same shell scripts
could work in both tools with minor config changes.</p>
<p>OpenCode takes a different approach. Instead of shell
scripts, it uses JavaScript or TypeScript plugins placed in
<code>.opencode/plugins/</code> or <code>~/.config/opencode/plugins/</code>. You
register functions on events like <code>tool.execute.before</code> and
<code>tool.execute.after</code>. More powerful (you get full access to
the session and project context), but less portable than a
shell script you can reuse across tools.</p>


<p>The <a href="/bot-wiki.html">bot-wiki post</a> introduced a structured wiki
designed for machine consumption. The whole point was to build
something a RAG pipeline could actually use. This is that pipeline.</p>
<h2 id="the-stack">The stack</h2>
<p>The wiki has 21 markdown pages with structured frontmatter:
title, summary, keywords, scope. The RAG pipeline is a single
Python script that:</p>
<ol>
<li>Embeds all 21 pages into vectors using OpenRouter's API</li>
<li>Stores them in a FAISS flat index (129 KB, committed to git)</li>
<li>At query time, embeds the question, finds the nearest pages,
and feeds them to an LLM for a grounded answer</li>
</ol>
<p>Dependencies: <code>faiss-cpu</code>, <code>openai</code> (for the OpenAI-compatible
API), and <code>pyyaml</code>. That's it.</p>
<h2 id="why-faiss">Why FAISS</h2>
<p><a href="https://github.com/facebookresearch/faiss">FAISS</a> is Meta's
library for similarity search over dense vectors. It runs
in-process, no server needed. You give it vectors, it gives
you nearest neighbors. The index is a flat file you can
commit to git.</p>
<p>I picked it because this is 21 vectors. There's no need for
a database, a server, or a managed service. <code>pip install faiss-cpu</code>, build an index, save it, done. The index is
129 KB. It loads in milliseconds.</p>
<p>If you're working at larger scale or want more features,
the main alternatives are:</p>
<ul>
<li><a href="https://github.com/chroma-core/chroma">Chroma</a>: embedded
vector database with a nice Python API. Good if you want
metadata filtering or persistence without managing files
yourself. Still runs locally.</li>
<li><a href="https://github.com/qdrant/qdrant">Qdrant</a>: vector search
engine written in Rust. Runs as a server, supports
filtering and payloads. Better fit when you have millions
of vectors or need a proper API.</li>
<li><a href="https://www.pinecone.io/">Pinecone</a>: fully managed cloud
vector database. Zero ops, but you're paying per query and
your data lives on someone else's servers.</li>
</ul>
<p>If you're already running PostgreSQL,
<a href="https://github.com/pgvector/pgvector">pgvector</a> is worth
a look. It adds vector columns and similarity search to
your existing database. No new service to run, and your
vectors live next to your relational data.</p>
<p>For 21 pages committed to a git repo, FAISS is the obvious
choice. No dependencies beyond the pip package, no running
services, no accounts.</p>
<h2 id="why-whole-page-chunks">Why whole-page chunks</h2>
<p>The wiki pages are short, 18 to 65 lines each. Splitting them
by section would lose context without gaining anything. The
frontmatter was designed to enrich embeddings, so each chunk
looks like this:</p>
<pre><code>Title: Claude Code
Summary: Anthropic's CLI-based AI coding assistant. Runs
  in terminal, edits files directly, supports MCP tool
  integrations and custom hooks.
Keywords: claude-code, anthropic, cli, ai-coding-assistant,
  mcp, hooks
Scope: Covers Claude Code setup, configuration, MCP
  integration, hooks, and usage patterns.

Claude Code is Anthropic's CLI tool for AI-assisted software
engineering. It operates directly in the terminal...
</code></pre>
<p>Prepending the summary, keywords, and scope to the body gives
the embedding model more signal about what the page covers.
21 chunks, 5,434 tokens total.</p>
<h3 id="chunking-in-general">Chunking in general</h3>
<p>Most RAG guides recommend chunks of 256 to 512 tokens. That's
a reasonable default for long documents where you want to
retrieve a specific paragraph, not the whole thing. Smaller
chunks give more precise retrieval. Larger chunks give the
LLM more context per result.</p>
<p>The tradeoffs:</p>
<ul>
<li><strong>Too small</strong> (under 100 tokens): chunks lose meaning on
their own. "See above" or "as mentioned" with no referent.
Retrieval finds fragments, not answers.</li>
<li><strong>Too large</strong> (over 1000 tokens): embeddings become diluted.
A chunk about five topics matches all five weakly instead
of one strongly. You also burn more of the LLM's context
window per retrieved result.</li>
<li><strong>Overlap</strong> helps when chunks split mid-thought. A common
pattern is 512-token chunks with 50-100 tokens of overlap.
This adds redundancy but reduces the chance of cutting a
relevant passage in half.</li>
</ul>
<p>The wiki pages average about 260 tokens each, which lands
right in the sweet spot. They're also self-contained by
design, each page covers one topic with clear boundaries.
No reason to split them further.</p>
<p>If I were indexing longer documents (blog posts, docs, book
chapters), I'd split by section headers first, then by token
count within sections. Semantic boundaries beat arbitrary
ones.</p>
<h2 id="building-the-index">Building the index</h2>
<pre><code class="language-bash">python3 -m venv .venv
.venv/bin/pip install -r bin/requirements-rag.txt
.venv/bin/python bin/wiki-rag.py build
</code></pre>
<pre><code>Embedding 21 wiki pages...
Indexed 21 pages, 1536 dimensions, 5434 tokens
Index: 129,069 bytes, metadata: 29,663 bytes
Saved to blog/markdown/wiki/.index
</code></pre>
<p>The embedding model is <code>text-embedding-3-small</code> via OpenRouter.
1536 dimensions per vector. The FAISS index uses <code>IndexFlatIP</code>
(inner product on normalized vectors = cosine similarity). No
approximate search needed at this scale.</p>
<p>The index and a <code>metadata.json</code> file (slug, title, summary,
full chunk text for each page) get saved to
<code>blog/markdown/wiki/.index/</code>. Both files are committed to git.
Any tool that wants to query the wiki can just read the index
directly without needing the embedding API.</p>
<p>The build is a manual step, not part of <code>npm run build</code>. Wiki
content doesn't change often, and there's no reason to pay for
embeddings on every deploy.</p>
<h2 id="querying">Querying</h2>
<p>A query hits two models. First, the question gets embedded
into a vector using the same embedding model that indexed
the wiki pages. FAISS compares that vector against the 21
stored vectors and returns the closest matches. Those
matches are wiki pages that are semantically similar to the
question, not keyword matches, but meaning matches.</p>
<p>Then the retrieved pages get passed to an LLM (Claude
Sonnet) as context, along with the original question. The
LLM reads the pages and writes an answer grounded in what
they actually say. Without the retrieval step, the LLM
would just guess from its training data. With it, the LLM
answers from your specific documents.</p>
<p>That's the "retrieval-augmented" part of RAG: find relevant
documents first, then generate an answer from them.</p>
<pre class="mermaid">
graph LR
    Q[Question] --> E[Embed question]
    E --> F[FAISS search]
    F --> R[Top 3 wiki pages]
    R --> P[Build prompt]
    P --> LLM[Claude Sonnet]
    LLM --> A[Answer + sources]
</pre>
<pre><code class="language-bash">.venv/bin/python bin/wiki-rag.py query \
  "What MCP servers does this project use?"
</code></pre>
<pre><code>Question: What MCP servers does this project use?

Top 3 matches:
  1. [0.644] MCP Integrations (wiki/mcp)
  2. [0.563] Linear MCP (wiki/mcp/linear)
  3. [0.524] GA4 MCP (wiki/mcp/ga4)

Answer:
Based on the provided context, this project uses three MCP
servers:

1. Playwright, for browser automation
2. Linear, for project management
3. GA4, for accessing website traffic data and reports

All MCP servers run as subprocesses communicating via JSON-RPC
over stdio, and are configured in ~/.claude.json or a
project-level .claude.json file.
</code></pre>
<p>The retrieval nailed the right pages. Here's a different query:</p>
<pre><code class="language-bash">.venv/bin/python bin/wiki-rag.py query "How do I run security scans?"
</code></pre>
<pre><code>Question: How do I run security scans?

Top 3 matches:
  1. [0.484] Security Toolkit (wiki/devops/security-toolkit)
  2. [0.439] DevOps &#x26; Security (wiki/devops)
  3. [0.343] OpenClaw Kubernetes Security (wiki/openclaw/kubernetes)
</code></pre>
<p>The answer included the actual docker commands from the security
toolkit wiki page, copy-paste ready. The third result
(Kubernetes security) is a reasonable stretch, the word
"security" overlaps, but the top two are exactly right.</p>
<h2 id="the-script">The script</h2>
<p>The interesting parts are small. Embedding is a single API call
since all 21 pages fit in one batch:</p>
<pre><code class="language-python">client = OpenAI(
    base_url="https://openrouter.ai/api/v1",
    api_key=os.environ["OPENROUTER_API_KEY"],
)
response = client.embeddings.create(
    model="openai/text-embedding-3-small",
    input=texts,
)
</code></pre>
<p>OpenRouter's embedding endpoint is OpenAI-compatible, so the
<code>openai</code> Python package works with just a base URL swap.</p>
<p>FAISS search returns similarity scores and indices:</p>
<pre><code class="language-python">scores, indices = index.search(query_vec, k)
for score, idx in zip(scores[0], indices[0]):
    page = metadata[idx]
    # score is cosine similarity (0 to 1)
</code></pre>
<p>The retrieved pages get stuffed into a prompt with instructions
to only use the provided context and cite sources. The LLM
(Claude Sonnet via OpenRouter) generates the answer.</p>
<h2 id="cost">Cost</h2>
<p>Indexing 21 pages used 5,434 tokens at
<code>text-embedding-3-small</code> rates ($0.02/1M tokens). That's
$0.0001. Each query embeds a short question (maybe 20 tokens)
plus one chat completion. A query costs roughly $0.002 to
$0.01 depending on the response length.</p>
<p>Building the index and running test queries cost $0.03
total on OpenRouter. That covers the embedding API calls
and the chat completions for query-mode answers. The
development itself (writing the script, iterating on the
blog post) ran through Claude Code on an Anthropic Max
subscription, so that cost isn't on the OpenRouter bill.</p>


<p>I added a new top-level section to this site: the
<a href="/wiki.html">Bot-Wiki</a>.</p>
<p>It looks like a wiki. It reads like a wiki. But it's not really for
humans. It's a structured knowledge base designed for AI agents and
RAG pipelines to consume.</p>
<h2 id="why-a-wiki-when-i-already-have-a-blog">Why a wiki when I already have a blog?</h2>
<p>Blog posts are stories. They have a narrative arc, opinions, tangents.
They're written for people who want to understand how I got from point
A to point B. That's great for learning from my experience but
terrible for a robot trying to answer "what MCP servers does this
project use?"</p>
<p>The wiki is the opposite. Each page is a factual reference card.
Structured metadata, explicit scope declarations, keyword lists for
embedding search, and cross-references for graph traversal. The tone
is robots writing notes for other robots.</p>
<h2 id="whats-in-it">What's in it</h2>
<p>The wiki covers the same ground as my recent blog posts but organized
by topic rather than chronology:</p>
<ul>
<li><strong>AI Tools</strong>: Claude Code, OpenCode, OpenRouter, local LLMs, image gen</li>
<li><strong>MCP Integrations</strong>: Playwright, Linear, GA4</li>
<li><strong>OpenClaw</strong>: the agent, its skills, Kubernetes security</li>
<li><strong>DevOps</strong>: security toolkit, lint toolkit, Ruler</li>
<li><strong>Blog Architecture</strong>: how this site is built, the markdown pipeline,
testing setup</li>
</ul>
<p>Each page has frontmatter designed for machine consumption:</p>
<pre><code class="language-yaml">title: "Claude Code"
summary: "CLI AI coding assistant by Anthropic with MCP support"
keywords:
  - claude-code
  - anthropic
  - mcp
related:
  - wiki/ai-tools/opencode
  - playwright-mcp
scope: "Setup, config, MCP integration, and usage patterns"
last_verified: 2026-03-10
</code></pre>
<p>The <code>summary</code> and <code>scope</code> fields help a RAG system decide whether to
retrieve the full page. The <code>keywords</code> list feeds embedding search.
The <code>related</code> links let an agent traverse the knowledge graph.
<code>last_verified</code> tells the agent how stale the content might be.</p>
<h2 id="how-this-connects-to-rag">How this connects to RAG</h2>
<p>This is step one of a larger plan. Next I'm building a RAG pipeline
that indexes the wiki content, generates embeddings, and lets agents
query it during conversations. The wiki's structured metadata makes
chunking and retrieval much cleaner than trying to RAG over blog
posts directly.</p>
<p>The directory structure maps to the URL structure which maps to the
knowledge hierarchy. An agent asking about MCP servers gets pointed
to <code>wiki/mcp/</code> and its children, not to a 2000-word blog post that
mentions MCP in passing.</p>
<h2 id="how-its-built">How it's built</h2>
<p>The wiki reuses the blog's existing infrastructure. A new <code>WikiService</code>
reads markdown from <code>markdown/wiki/</code> instead of <code>markdown/posts/</code>,
builds a tree from the directory structure, and generates navigation
automatically. Each <code>index.md</code> file gets a tree of its children
rendered above its own content.</p>
<p>The routing plugs into the same Next.js catch-all that handles blog
posts. Wiki pages get the same layout, sidebar, and responsive
behavior. They just have different content and metadata.</p>
<p>No images in the wiki. It's all text, all the time. Robots don't
need hero images.</p>
<p>You can find it at <a href="/wiki.html">/wiki.html</a>, or use the new Wiki
link in the header.</p>


<p>Linear's official MCP server launched in May 2025. It lets the robots lean in on the
project management work. I want to get some ideation-bots going.</p>
<h2 id="supported-commands">Supported Commands</h2>
<table>
<thead>
<tr>
<th>Category</th>
<th>Commands</th>
</tr>
</thead>
<tbody>
<tr>
<td>Issues</td>
<td>list, get, create/update, status, statuses</td>
</tr>
<tr>
<td>Projects</td>
<td>list, get, create/update, labels</td>
</tr>
<tr>
<td>Documents</td>
<td>list, get, create, update, search</td>
</tr>
<tr>
<td>Comments</td>
<td>list, create/update, delete</td>
</tr>
<tr>
<td>Attachments</td>
<td>get, create, delete, extract images</td>
</tr>
<tr>
<td>Milestones</td>
<td>list, get, create/update</td>
</tr>
<tr>
<td>Cycles</td>
<td>list</td>
</tr>
<tr>
<td>Teams</td>
<td>list, get</td>
</tr>
<tr>
<td>Users</td>
<td>list, get</td>
</tr>
<tr>
<td>Labels</td>
<td>list issue labels, create issue label</td>
</tr>
</tbody>
</table>
<h2 id="setup">Setup</h2>
<p>Linear hosts the MCP server at <code>https://mcp.linear.app/mcp</code>. Use that endpoint, not the
old SSE one which is deprecated.</p>
<h3 id="claude-code">Claude Code</h3>
<pre><code class="language-bash">claude mcp add --transport http linear-server https://mcp.linear.app/mcp
</code></pre>
<p>Restart and authenticate via OAuth.</p>
<hr>
<h1 id="idea-nightly-ideation-engine">Idea: Nightly Ideation Engine</h1>
<p>Instead of using Linear as a human-operated task tracker, an AI agent can read youri
codebase, compare it against goals, and file suggestions directly.</p>
<p>What I'm hoping to build is a system where a scheduled agent runs nightly. It reads
blog posts, checks page speed, reviews SEO metadata, and compares everything against a
set of high-level goals:</p>
<ul>
<li>100% accurate content</li>
<li>Fast loading</li>
<li>Low cost</li>
<li>Looks good</li>
<li>Good SEO</li>
<li>Interesting tutorials about emerging tooling</li>
</ul>
<p>It should create Linear issues for anything worth improving, labeled "Suggestion" and
drop them into the Backlog.</p>
<h3 id="the-pipeline">The pipeline</h3>
<p>Linear's built-in statuses act as gates between human and agent
work.</p>
<pre class="mermaid">
stateDiagram-v2
    [*] --> Backlog: AI files suggestion
    Backlog --> Todo: Human promotes
    Backlog --> Canceled: Human rejects
    Todo --> InProgress: Agent picks up
    InProgress --> InReview: Agent finishes
    InReview --> Done: Human approves
    InReview --> InProgress: Human pushes back
</pre>
<p>Each step has a clear owner:</p>
<table>
<thead>
<tr>
<th>Status</th>
<th>Owner</th>
<th>What happens</th>
</tr>
</thead>
<tbody>
<tr>
<td>Backlog</td>
<td>AI</td>
<td>Nightly agent creates suggestion issues</td>
</tr>
<tr>
<td>Todo</td>
<td>Human</td>
<td>You move the good ones forward</td>
</tr>
<tr>
<td>In Progress</td>
<td>Agent</td>
<td>A working agent picks it up</td>
</tr>
<tr>
<td>In Review</td>
<td>Agent</td>
<td>Agent finishes, leaves a comment</td>
</tr>
<tr>
<td>Done</td>
<td>Human</td>
<td>You verify and close</td>
</tr>
<tr>
<td>Canceled</td>
<td>Human</td>
<td>Bad idea, toss it</td>
</tr>
</tbody>
</table>
<h3 id="agent-notes-via-comments">Agent notes via comments</h3>
<p>When agents work on an issue, they leave comments. This beats
internal context files because comments are visible, searchable,
and survive across sessions. If one agent starts a task and another
finishes it, the full history is right there on the issue.</p>
<p>I might also run multiple planning agents that each comment on a
ticket before I review it. A fact-checker, a SEO reviewer, and a
style auditor could all weigh in on the same Backlog issue. By the
time I look at it, there's already a thread of perspectives.</p>
<pre><code class="language-bash"># An agent checking for work:
# list_issues with label="Suggestion", state="Todo"

# An agent leaving notes:
# save_comment with issueId and markdown body
</code></pre>
<h3 id="filtering">Filtering</h3>
<p>The nightly agent queries existing Backlog suggestions before
creating new ones, avoiding duplicates. Working agents filter
by label and status to find their next task:</p>
<pre><code>list_issues → label: "Suggestion", state: "Todo"
</code></pre>
<h2 id="defining-state-as-prompts">Defining State as Prompts</h2>
<p>Linear's community Terraform provider exists but only covers teams,
labels, workflow states, and templates. No custom views. The GraphQL
API has a <code>customViewCreate</code> mutation, but nobody's wrapped it in
IaC tooling yet.</p>
<p>So instead of declaring views in HCL, I described mine as a prompt
and had Claude set it up via Playwright MCP:</p>
<blockquote>
<p>Create a custom view called "Suggestion Backlog" on the Pericak
team. Filter to issues where label includes "Suggestion" and
status is "Backlog". Description: "AI-generated improvement
suggestions awaiting human review".</p>
</blockquote>
<p>Claude navigated to Linear in Chromium, clicked "Add new view",
filled in the name, added both filters, and hit Save. The view
now lives as a tab on the team page.</p>
<p>This is nondeterministic state management. There's no lock file,
no plan/apply, no drift detection. But it works, and the prompt
is the documentation. If the view ever gets deleted, re-run the
prompt.</p>
<h3 id="a-linear-admin-agent">A Linear-Admin agent</h3>
<p>A "Linear-Admin" agent could own all workspace
configuration: views, labels, workflow states, notification
settings. You'd feed it a doc describing your desired workspace
setup, and it would use Playwright to converge Linear's UI to
match. Like Terraform, but with a browser instead of an API.</p>
<p>The tradeoffs are real. Playwright clicks are slow, brittle if
Linear redesigns their UI, and hard to make idempotent. But for
a personal workspace where the alternative is "click around in
the UI manually," the prompt-as-IaC approach is surprisingly
practical.</p>
<p>Still faster than a person.</p>


<p>Playwright MCP gives Claude Code a real browser to drive. Coding agents are kind of
awful at doing the right thing unless they can verify their work. If you want to do any
front-end work and not be the human in the loop every step along the way, it's gotta be
able to operate Chromium and review the photos. Opus 4.6 is multi modal and any later
models will just be better, so it can consume the photos itself.</p>
<h1 id="setup">Setup</h1>
<pre><code class="language-bash">claude mcp add playwright -- npx @playwright/mcp@latest --browser chromium
</code></pre>
<p>Or add it to <code>.mcp.json</code> at the project root to share it with the team:</p>
<pre><code class="language-json">{
  "mcpServers": {
    "playwright": {
      "command": "npx",
      "args": ["@playwright/mcp@latest", "--browser", "chromium"]
    }
  }
}
</code></pre>
<p>Restart Claude Code. Run <code>/mcp</code> to confirm it's listed. If you've got a session
already, tell it to store its memory first and to load it back after, works well.</p>
<h1 id="what-playwright-does">What Playwright Does</h1>
<p>At work we migrated from Selenium/WebDriver to this to address flakiness long before
the MCP was out, but at this point my interest in playwright is more about the AI
integration.</p>
<p>Playwright MCP exposes the browser as individual tools Claude can call. This post demos
the main ones, with examples against my dev blog server at <code>http://localhost:3000</code>.</p>
<p>I'm pretty sure I'll be able to post agents at this post even to tell it how I want
them to work, and ask them to write a claude.md about it.</p>
<h2 id="browser_navigate">browser_navigate</h2>
<pre><code>browser_navigate: { url: "http://localhost:3000" }
</code></pre>
<p>Loads a URL. Returns the final URL (catches redirects) and a console
error/warning count alongside the snapshot.</p>
<h2 id="browser_snapshot">browser_snapshot</h2>
<pre><code>browser_snapshot: {}
</code></pre>
<p>Returns an accessibility tree of the current page. This is the main tool for
reading page state: what's visible, what's interactive, what the structure is.
Faster and more structured than HTML, and the <code>ref</code> values it returns are what
<code>browser_click</code> and <code>browser_type</code> use to target elements.</p>
<h2 id="browser_take_screenshot">browser_take_screenshot</h2>
<pre><code>browser_take_screenshot: {}
</code></pre>
<p>Returns a PNG of the current viewport. This is the tool Claude uses to judge
look and feel. It's multimodal, so it actually reads the image and can catch
layout problems, broken images, or rendering bugs that don't show up in the
accessibility tree. It can't extract <code>ref</code> values from a screenshot though, so
use <code>browser_snapshot</code> when you need to drive further actions.</p>
<h2 id="browser_click">browser_click</h2>
<pre><code>browser_click: { element: "Posts", ref: "..." }
</code></pre>
<p>Clicks by visible label or element reference. Drives navigation and interactive
components.</p>
<h2 id="browser_type">browser_type</h2>
<pre><code>browser_type: { element: "search input", ref: "...", text: "Playwright" }
</code></pre>
<p>Types into an input. The <code>ref</code> comes from <code>browser_snapshot</code>.</p>
<h2 id="browser_evaluate">browser_evaluate</h2>
<pre><code>browser_evaluate: { function: "() => document.title" }
</code></pre>
<p>Runs JavaScript in the page context. For reading state that isn't in the DOM.</p>
<h2 id="browser_wait_for">browser_wait_for</h2>
<pre><code>browser_wait_for: { text: "Playwright MCP" }
browser_wait_for: { textGone: "Loading..." }
browser_wait_for: { time: 2 }
</code></pre>
<p>Three modes: wait for text to appear, wait for text to disappear, or sleep for
N seconds. No explicit timeout parameter. It uses Playwright's default (30s).
Needed for anything that renders async.</p>
<h2 id="browser_console_messages">browser_console_messages</h2>
<pre><code>browser_console_messages: {}
browser_console_messages: { level: "error" }
</code></pre>
<p>Returns the full console output: message text, level, and source location.
The <code>level</code> filter works like log levels: <code>error</code> gives you errors only, <code>info</code>
gives you everything. The navigate and snapshot tools show a count
(<code>1 errors, 0 warnings</code>), but this is where you get the actual content.</p>
<p>Claude can read and interpret it. Errors include full stack traces and, for
things like React hydration mismatches, the actual HTML diff showing which
component diverged between server and client. Ask Claude to look at the console
and it'll summarize the errors and suggest fixes rather than dumping the raw
text at you.</p>
<p>First thing to check when something looks wrong but the DOM appears fine.</p>
<p>Also worth having in every test suite: a test that loads each key page and
asserts no console errors. It catches a lot of regressions for free.</p>
<pre><code class="language-typescript">test('no console errors on homepage', async ({ page }) => {
  const errors: string[] = [];
  page.on('console', msg => {
    if (msg.type() === 'error') errors.push(msg.text());
  });
  await page.goto('/');
  expect(errors).toHaveLength(0);
});
</code></pre>
<h2 id="browser_network_requests">browser_network_requests</h2>
<pre><code>browser_network_requests: {}
browser_network_requests: { includeStatic: true }
</code></pre>
<p>The network log since the last navigation. Returns method, URL, and status code
for each request. Good for catching 404s on assets or confirming a request
fired. Excludes static resources by default. Pass <code>includeStatic: true</code> to see
everything.</p>
<p>No timing data. If you need to assert on response duration, use Playwright's
<code>page.on('response', ...)</code> in a test instead:</p>
<pre><code class="language-typescript">test('API calls complete within 500ms', async ({ page }) => {
  const timings: number[] = [];
  page.on('response', async response => {
    if (response.url().includes('/api/')) {
      const timing = response.timing();
      timings.push(timing.responseEnd - timing.requestStart);
    }
  });
  await page.goto('/');
  expect(Math.max(...timings)).toBeLessThan(500);
});
</code></pre>
<p><code>page.on('response', ...)</code> fires for every network response. Filter by URL
pattern to target whatever you care about: API calls, a specific domain,
a file type. Same pattern works for asserting no 404s:</p>
<pre><code class="language-typescript">test('no broken asset requests', async ({ page }) => {
  const failures: string[] = [];
  page.on('response', response => {
    if (response.status() >= 400) failures.push(`${response.status()} ${response.url()}`);
  });
  await page.goto('/');
  expect(failures).toHaveLength(0);
});
</code></pre>
<h2 id="browser_resize">browser_resize</h2>
<pre><code>browser_resize: { width: 375, height: 812 }
</code></pre>
<p>Resizes the viewport. Triggers CSS media queries based on width, so layout
reflows and responsive breakpoints work as expected. Good for a quick visual
check. Pair it with <code>browser_take_screenshot</code> to see what changed.</p>
<p>It doesn't change the user agent, device pixel ratio, or pointer type. That
means UA-sniffing code still thinks it's a desktop browser, <code>srcset</code> serves
desktop-resolution images, and <code>hover: none</code> / <code>pointer: coarse</code> media queries
don't fire. For proper mobile regression coverage, use Playwright's device
emulation in the test config instead:</p>
<pre><code class="language-typescript">import { devices } from '@playwright/test';

// playwright.config.ts
export default defineConfig({
  projects: [
    { name: 'desktop', use: { viewport: { width: 1280, height: 720 } } },
    { name: 'mobile',  use: { ...devices['iPhone 13'] } },
  ],
});
</code></pre>
<p><code>devices['iPhone 13']</code> sets viewport, user agent, device scale factor, and
touch support together. Your tests run against both profiles automatically.</p>
<hr>
<h1 id="gherkin-specs-as-source-of-truth">Gherkin Specs as Source of Truth</h1>
<p>AI-generated test code has a maintenance problem. The agent writes tests, they
drift from what the UI actually does, someone edits them directly, and now you
have no reliable source of truth.</p>
<p>What I'm experimenting with here is to keep specs in Gherkin, generate tests from
them, and treat the generated code as a build artifact. Claude rewrites the tests from
spec whenever they go stale. Nobody edits the generated files.</p>
<pre><code>tests/
  spec/
    index.feature
    post.feature
    navigation.feature
  playwright/
    index.spec.ts      # generated, do not edit
    post.spec.ts       # generated, do not edit
    navigation.spec.ts # generated, do not edit
  playwright.config.ts
</code></pre>
<p>The actual specs for this blog live in
<a href="https://github.com/kylep/multi/tree/main/apps/blog/blog/tests/spec/"><code>tests/spec/</code></a>.
Here's the index one:</p>
<pre><code class="language-gherkin"># tests/spec/index.feature

Feature: Blog index page

  Scenario: Page loads with post list
    Given I navigate to the blog homepage
    Then I see a list of posts
    And each post has a title, date, and summary

  Scenario: Posts link to their detail page
    Given I navigate to the blog homepage
    When I click the first post title
    Then I am on a post detail page
    And the post has a title and content

  Scenario: Pagination controls are present
    Given I navigate to the blog homepage
    Then I see pagination controls
</code></pre>
<p>The instruction to Claude is: read the <code>.feature</code> file, generate a matching
<code>spec.ts</code>. If something breaks, fix the spec first, then regenerate. The feature
file is what documents how the UI is supposed to behave.</p>
<p>A <code>CLAUDE.md</code> in <code>tests/</code> gives Claude guidance when generating tests, things
like keeping the suite under a minute, no sleeps, one assertion per scenario.
If you're using <a href="/ruler-cross-tool-ai-rules.html">Ruler</a> to share rules across tools,
put that content in <code>tests/.ruler/</code> instead and let Ruler generate the
<code>CLAUDE.md</code> alongside your <code>AGENTS.md</code>.</p>
<h1 id="claude-verifying-its-own-changes">Claude Verifying Its Own Changes</h1>
<p>This is the part I wanted to try. Claude makes a change, then uses the MCP
browser to check whether it worked, before handing back.</p>
<pre class="mermaid">
graph TD
    A[Claude makes a change] --> B[Build blog]
    B --> C[Start dev server]
    C --> D[Navigate to affected page]
    D --> E[Screenshot + read DOM]
    E --> F{Looks right?}
    F -->|No| H[Investigate and fix]
    H --> B
    F -->|Yes| I{Add a test?}
    I -->|No| G[Done]
    I -->|Yes| J[Write Gherkin scenario]
    J --> K[Generate Playwright spec]
    K --> L[Run test to verify]
    L --> G
</pre>
<p>It's nondeterministic. That's the strength and the problem. Claude
isn't following a checklist. It navigates to what it changed, reads what's
there, and decides whether it matches intent. That catches a class of dumb
mistakes a checklist wouldn't, but it also means "looks right" depends on the
agent's judgment, which isn't always reliable.</p>
<p>The multimodal part matters here. When Claude calls <code>browser_take_screenshot</code>,
it doesn't get back an opaque file path. It sees the rendered page. It can tell
if an image failed to load, if a layout broke, if text is overlapping. That's
more than a DOM inspection gives you. It's also not a substitute for explicit
assertions, because "looks right" is still a judgment call.</p>
<pre class="mermaid">
graph TD
    A[Claude ready to commit] --> B[Start dev server]
    B --> C[Run Playwright tests]
    C --> D{Tests pass?}
    D -->|Yes| E[Commit]
    D -->|No| F[Investigate and fix]
    F --> C
</pre>
<p>This part is deterministic as reasonably quick in headless mode.
Screenshots are for the stuff that's hard to assert: visual regressions, style
issues, anything where you'd want a human to glance at it.</p>
<h2 id="playwright-config-for-the-blog">Playwright Config for the Blog</h2>
<pre><code class="language-bash">cd apps/blog
npm init playwright@latest
</code></pre>
<pre><code class="language-typescript">// playwright.config.ts
import { defineConfig } from '@playwright/test';

export default defineConfig({
  testDir: './tests/playwright',
  use: {
    baseURL: 'http://localhost:3000',
  },
  webServer: {
    command: 'npm run dev',
    url: 'http://localhost:3000',
    reuseExistingServer: true,
  },
});
</code></pre>
<p>The generated specs live in
<a href="https://github.com/kylep/multi/tree/main/apps/blog/blog/tests/playwright/"><code>tests/playwright/</code></a>.
Run them with:</p>
<pre><code class="language-bash">cd apps/blog/blog
npx playwright test
</code></pre>
<p>Claude gets pass/fail output with assertion details. Unambiguous, scriptable,
no judgment required.</p>
<p>Note: <code>npx playwright test</code> runs headless by default. No browser window appears.
Pass <code>--headed</code> to watch it run, or set <code>headless: false</code> in the config.</p>
<h1 id="where-this-is-going-agent-teams">Where This Is Going: Agent Teams</h1>
<p>Claude Code has a multi-agent feature in preview. The setup that makes sense
here is a builder agent making changes and a separate verifier agent running
tests and reporting back. They work in parallel. Failures come back as tasks.</p>
<p>Not wired up yet. But the spec files, generated tests, and MCP config are what
make it possible to hand verification to a different agent without losing the
context of what the tests mean. I ended up wiring this into a <a href="/agent-org-chart.html">QA subagent in
the agent team</a>.</p>


<p>I'm building a tabletop RPG game that generates images on the fly: character portraits,
enemy illustrations, scene art. Cloud APIs work but get expensive when every combat
encounter wants several images. More importantly, I need character consistency. If I
generate a portrait of "Aria the rogue" in session one, future images of her should
look like the same character, not a random new person.</p>
<p>ComfyUI solves both. It runs locally on Apple Silicon, exposes a REST API, and
supports IP-Adapter for using an existing portrait as a visual reference for new
generations.</p>
<h1 id="hardware-reality">Hardware Reality</h1>
<p>MacBook Air M2, 16GB unified memory. The Air throttles under sustained load since
it has no active cooling. Fine for a game that generates at natural breaks (between
scenes, at combat start) rather than continuously.</p>
<p>16GB comfortably fits:</p>
<ul>
<li><strong>SDXL</strong> (fp16, ~6.5GB): mature IP-Adapter ecosystem, good portrait quality</li>
<li><strong>FLUX.2-klein-4B</strong> (fp8, ~8-9GB): Apache 2.0, released November 2025, 4-step
distilled generation, supersedes FLUX.1-schnell. IP-Adapter support less
mature than SDXL.</li>
<li><strong>FLUX.1-dev</strong> (fp8, ~8GB): previous generation but still the best quality
option that fits 16GB. FLUX.2-dev exists but won't fit.</li>
</ul>
<p>For this post I'm using SDXL. Its IP-Adapter tools are battle-tested for exactly
this use case, and 6.5GB leaves plenty of headroom for additional models.</p>
<h1 id="install-comfyui">Install ComfyUI</h1>
<p>ComfyUI has a <a href="https://github.com/Comfy-Org/desktop">Desktop app</a> that handles
everything automatically. Use that if you just want the UI. For API access and
full control, manual install is cleaner.</p>
<pre><code class="language-bash">git clone https://github.com/comfyanonymous/ComfyUI
cd ComfyUI
python3.13 -m venv venv
source venv/bin/activate
python --version  # must say 3.13 — if it says 3.9 you have an old venv, rm -rf venv and redo
pip install -r requirements.txt
</code></pre>
<p>On Apple Silicon, the requirements file pulls a PyTorch build with MPS support
automatically. No CUDA, no special flags.</p>
<p>Start it:</p>
<pre><code class="language-bash">source venv/bin/activate
python main.py --listen 0.0.0.0 --port 8188
</code></pre>
<p><code>--listen 0.0.0.0</code> binds to all interfaces instead of localhost only. Needed for
the API to be reachable from other processes or devices on your network.</p>
<p>I keep a <code>start-comfyui.sh</code> in <code>~/models/</code> next to the model README so there's
one place for everything model-related:</p>
<pre><code class="language-bash">#!/bin/bash
source /path/to/ComfyUI/venv/bin/activate
cd /path/to/ComfyUI
python main.py --listen 0.0.0.0 --port 8188
</code></pre>
<p>Open <code>http://localhost:8188</code>. You should see the node graph UI with a default
workflow loaded. No models needed for the server to start — the UI comes up
regardless.</p>
<h1 id="model-storage">Model Storage</h1>
<p>Models don't need to live inside the repo. ComfyUI will look in additional
directories if you tell it to via <code>extra_model_paths.yaml</code>:</p>
<pre><code class="language-bash">cp extra_model_paths.yaml.example extra_model_paths.yaml
</code></pre>
<p>Then add an entry pointing at wherever you keep models. For example, if you
store them under <code>~/models/</code>:</p>
<pre><code class="language-yaml">my_models:
  base_path: ~/models/
  checkpoints: sdxl/
  ipadapter: ipadapter/
  clip_vision: clip_vision/
</code></pre>
<p>Adjust the subdirectory names to match your layout. ComfyUI merges these with
its own paths and shows everything in the UI dropdowns.</p>
<h1 id="models">Models</h1>
<p>Download SDXL (~6.5GB):</p>
<pre><code class="language-bash">hf download stabilityai/stable-diffusion-xl-base-1.0 \
  sd_xl_base_1.0.safetensors \
  --local-dir ~/models/sdxl/
</code></pre>
<p>For IP-Adapter, clone the custom node into ComfyUI's <code>custom_nodes/</code> directory:</p>
<pre><code class="language-bash">cd /path/to/ComfyUI/custom_nodes
git clone https://github.com/cubiq/ComfyUI_IPAdapter_plus
</code></pre>
<p>No pip install needed — it has no extra dependencies beyond what ComfyUI
already installs. ComfyUI picks it up automatically on next startup.</p>
<p>Then download the IP-Adapter weights:</p>
<pre><code class="language-bash"># IP-Adapter for SDXL
hf download h94/IP-Adapter \
  sdxl_models/ip-adapter_sdxl.bin \
  --local-dir ~/models/ipadapter/

# CLIP vision encoder (required by IP-Adapter — must be ViT-bigG, not ViT-L)
hf download laion/CLIP-ViT-bigG-14-laion2B-39B-b160k \
  --local-dir ~/models/clip_vision/CLIP-ViT-bigG-14-laion2B-39B-b160k/
</code></pre>
<p>Restart ComfyUI after installing custom nodes. It rescans the nodes directory on
startup.</p>
<h1 id="quick-test">Quick Test</h1>
<p>Before touching the API, confirm generation works at all.</p>
<p>The built-in Templates target newer models and won't work with SDXL. ComfyUI
ships a test workflow in the repo — open it with <strong>File > Open</strong>:</p>
<pre><code>/path/to/ComfyUI/tests/inference/graphs/default_graph_sdxl1_0.json
</code></pre>
<p><img src="/images/comfyui-web-interface.png" alt="ComfyUI web interface"></p>
<p>It's a base+refiner workflow with two Load Checkpoint nodes. The top one
should already show <code>sd_xl_base_1.0.safetensors</code>. The bottom one defaults to
<code>sd_xl_refiner_1.0.safetensors</code> — change it to <code>sd_xl_base_1.0.safetensors</code>
too. We're not doing base+refiner, just running base twice.</p>
<p>Drop a prompt into one of the CLIP Text Encode nodes and hit <strong>Run</strong>:</p>
<pre><code>a knight in plate armor holding a mace and shield, fantasy RPG portrait,
detailed face, dramatic lighting, painterly style, 4k, highly detailed
</code></pre>
<p>First run is slow while the model loads. On M2 Air, SDXL at 20 steps takes
90-120 seconds. Subsequent runs with the model cached are faster but still
in that range — the Air throttles under sustained load with no active cooling.</p>
<p><img src="/images/oss-img-gen-knight.png" alt="Knight portrait generated with SDXL"></p>
<h1 id="img2img-post-battle">Img2Img: Post-Battle</h1>
<p>Txt2img generates from noise. Img2img starts from an existing image and
nudges it toward a new prompt. Good way to stay close to a character while
changing context or condition.</p>
<p>Add two nodes to the current workflow. Right-click empty canvas space (not
on a node) to get the picker:</p>
<ul>
<li><strong>image → Load Image</strong> — then select your knight from <code>ComfyUI/output/</code>
using the dropdown in the node, or upload it from there</li>
<li><strong>latent → VAE Encode</strong> — converts a pixel image into the latent space
the sampler works in, replacing the EmptyLatentImage</li>
</ul>
<p>Then make three connections. To draw a wire: click and drag from an output
dot (right side of a node) to an input dot (left side of another node).
The dots are color-coded by type and will only connect to compatible ports.</p>
<ol>
<li>Drag from the <strong>IMAGE</strong> dot (blue, right side of Load Image) to the
<strong>pixels</strong> dot on VAE Encode</li>
<li>Drag from either <strong>Load Checkpoint VAE</strong> dot → <strong>VAE Encode vae</strong> dot</li>
<li>Drag from <strong>VAE Encode LATENT</strong> dot → <strong>KSampler latent_image</strong> dot
(disconnects EmptyLatentImage automatically)</li>
</ol>
<p>The test workflow uses <strong>KSampler Advanced</strong>, which has no <code>denoise</code> slider.
Instead set <strong>start_at_step</strong> to <code>6</code>. With 20 total steps that runs the last
14, equivalent to about 70% denoise. Lower start_at_step = more faithful to
original, higher = more change.</p>
<p>Update the positive prompt — that's the CLIP Text Encode node wired to the
KSampler's <strong>positive</strong> input, as opposed to the negative one (which holds
your "ugly, blurry..." text):</p>
<pre><code>a knight in plate armor, damaged armor, battle-worn, dented helmet,
scratches and dents, mud-stained, exhausted, fantasy RPG portrait,
detailed face, dramatic lighting, painterly style, 4k, highly detailed
</code></pre>
<p>Hit <strong>Run</strong>. If it's changing too much, drop start_at_step to 4. If the
damage isn't reading, push to 8.</p>
<p><img src="/images/img2image-knight.png" alt="Knight after img2img battle damage pass"></p>
<p>Img2img works, but it's coarse. Fine details like face structure get lost
as you increase denoise, and at low denoise the prompt has limited
influence. Good for "same scene, different state." Not what you want for
generating the same character across completely different scenes.</p>
<p>That's what IP-Adapter is for. Instead of perturbing the reference image,
it encodes it through CLIP vision into an embedding that gets injected
alongside the text conditioning during generation. The sampler still starts
from pure noise, but the attention layers are nudged toward the character's
appearance at every step. Face and body survive across very different
scenes because the conditioning runs throughout, not just at the start.</p>
<h1 id="the-api">The API</h1>
<p>ComfyUI's API accepts a workflow as JSON and queues the generation. The workflow
you build in the UI can be exported in API format: go to Settings, enable Dev
Mode Options, then use the "Save (API Format)" button on the workflow.</p>
<p>The API format represents each node as a dict entry keyed by node ID. Here's a
minimal SDXL workflow:</p>
<pre><code class="language-json">{
  "4": {
    "class_type": "CheckpointLoaderSimple",
    "inputs": { "ckpt_name": "sd_xl_base_1.0.safetensors" }
  },
  "5": {
    "class_type": "EmptyLatentImage",
    "inputs": { "batch_size": 1, "height": 1024, "width": 768 }
  },
  "6": {
    "class_type": "CLIPTextEncode",
    "inputs": {
      "clip": ["4", 1],
      "text": "a fantasy rogue in a dark alley, RPG portrait, detailed"
    }
  },
  "7": {
    "class_type": "CLIPTextEncode",
    "inputs": {
      "clip": ["4", 1],
      "text": "ugly, blurry, watermark, text, bad anatomy"
    }
  },
  "3": {
    "class_type": "KSampler",
    "inputs": {
      "model": ["4", 0],
      "positive": ["6", 0],
      "negative": ["7", 0],
      "latent_image": ["5", 0],
      "seed": 42,
      "steps": 20,
      "cfg": 7,
      "sampler_name": "dpmpp_2m",
      "scheduler": "karras",
      "denoise": 1
    }
  },
  "8": {
    "class_type": "VAEDecode",
    "inputs": { "samples": ["3", 0], "vae": ["4", 2] }
  },
  "9": {
    "class_type": "SaveImage",
    "inputs": { "filename_prefix": "rpg", "images": ["8", 0] }
  }
}
</code></pre>
<p>Submit it:</p>
<pre><code class="language-bash">curl -X POST http://localhost:8188/prompt \
  -H "Content-Type: application/json" \
  -d '{"prompt": &#x3C;workflow_json>}'
# Returns: {"prompt_id": "abc123...", "number": 1, "node_errors": {}}
</code></pre>
<p>Poll for completion:</p>
<pre><code class="language-bash">curl http://localhost:8188/history/abc123
# When done: history[prompt_id]["status"]["completed"] == true
</code></pre>
<p>Fetch the image:</p>
<pre><code class="language-bash">curl "http://localhost:8188/view?filename=rpg_00001_.png" \
  --output character.png
</code></pre>
<p>Images also land in <code>ComfyUI/output/</code> on disk if you'd rather read them
directly.</p>
<h1 id="image-consistency-with-ip-adapter">Image Consistency with IP-Adapter</h1>
<p>IP-Adapter encodes a reference image using CLIP vision and injects that embedding
alongside the text prompt during generation. The result is a new image that
respects the prompt but is visually pulled toward the reference: same hair, same
armor, similar face structure.</p>
<p>It's not a perfect clone. Think of it as a strong nudge. Consistent text prompts
help too. If every prompt for Aria includes "tall elf rogue, purple hair, leather
armor," you're conditioning from both directions.</p>
<p>Add these nodes to the workflow above:</p>
<pre><code class="language-json">"10": {
  "class_type": "IPAdapterModelLoader",
  "inputs": { "ipadapter_file": "ip-adapter_sdxl.bin" }
},
"11": {
  "class_type": "CLIPVisionLoader",
  "inputs": { "clip_name": "CLIP-ViT-bigG-14-laion2B-39B-b160k/open_clip_model.safetensors" }
},
"12": {
  "class_type": "LoadImage",
  "inputs": { "image": "aria_portrait.png" }
},
"13": {
  "class_type": "IPAdapterAdvanced",
  "inputs": {
    "model": ["4", 0],
    "ipadapter": ["10", 0],
    "image": ["12", 0],
    "clip_vision": ["11", 0],
    "weight": 0.6,
    "weight_type": "linear",
    "combine_embeds": "concat",
    "start_at": 0.0,
    "end_at": 1.0,
    "embeds_scaling": "V only"
  }
}
</code></pre>
<p>Then change node <code>"3"</code>'s <code>model</code> input from <code>["4", 0]</code> to <code>["13", 0]</code>.</p>
<p>The prompt here asks for the same knight in a tavern — completely different
scene, same character used as reference.</p>
<p><img src="/images/oss-img-gen-ipadapter.png" alt="Knight in a tavern, generated with IP-Adapter using the portrait as reference"></p>
<p><code>weight</code> is the main knob. At 0.5 you get style and general appearance influence.
At 0.8 it starts fighting the text prompt. For character identity, 0.6-0.7 is
the range to start with.</p>
<p>The <code>image</code> input in node <code>"12"</code> is a filename relative to <code>ComfyUI/input/</code>.
Copy your reference images there, or upload them via the API:</p>
<pre><code class="language-bash">curl -X POST http://localhost:8188/upload/image \
  -F "image=@aria_portrait.png"
# Returns: {"name": "aria_portrait.png", "subfolder": "", "type": "input"}
</code></pre>
<h2 id="face-consistency">Face Consistency</h2>
<p>Standard IP-Adapter works well for overall style and body type. Faces drift more
than you'd like, especially across very different scenes. IP-Adapter FaceID handles
this by extracting face embeddings specifically rather than encoding the whole image.</p>
<p>Download:</p>
<pre><code class="language-bash">hf download h94/IP-Adapter-FaceID \
  ip-adapter-faceid-plusv2_sdxl.bin \
  --local-dir ~/models/ipadapter/
</code></pre>
<p>FaceID also requires InsightFace, which isn't on HuggingFace. Download the
<code>antelopev2</code> model package from the
<a href="https://github.com/deepinsight/insightface/tree/master/model_zoo">InsightFace model zoo</a>
and place it under <code>~/models/insightface/models/antelopev2/</code> (or wherever your
<code>extra_model_paths.yaml</code> maps <code>insightface</code>).</p>
<p>Install the Python package into the ComfyUI venv:</p>
<pre><code class="language-bash">source /path/to/ComfyUI/venv/bin/activate
pip install insightface onnxruntime
</code></pre>
<p>With FaceID installed, use <code>IPAdapterFaceIDAdvanced</code> instead of <code>IPAdapterAdvanced</code>.
It takes the same inputs but runs the face embedding extraction automatically.
Faces stay recognizable across very different scenes much more reliably than with
the standard adapter.</p>
<p>You can combine both: use <code>IPAdapterAdvanced</code> for overall character appearance
and <code>IPAdapterFaceIDAdvanced</code> for face locking, chaining them in sequence.</p>
<h1 id="game-integration">Game Integration</h1>
<p>The pattern for a TTRPG character system:</p>
<pre class="mermaid">
graph TD
    A[Player describes character] --> B[Generate initial portrait\nno reference image]
    B --> C[Store portrait in character record]
    C --> D[Game event triggers image]
    D --> E[Build scene + character prompt]
    E --> F[Upload portrait, generate\nwith IP-Adapter]
    F --> G[Display in game]
    G --> D
</pre>


<p>No API costs, no data leaving the machine, works offline. The catch is you're
limited to models that fit in RAM. At 16GB, you can run most 7-8B models at
good quality, or push into 12B at a lower quant.</p>
<p>This runs three models locally and compares them:</p>
<table>
<thead>
<tr>
<th>Model</th>
<th>Params</th>
<th>Style</th>
<th>Quant</th>
<th>Size</th>
</tr>
</thead>
<tbody>
<tr>
<td>Qwen3 8B</td>
<td>8B</td>
<td>Instruction</td>
<td>Q4_K_L</td>
<td>5.49 GB</td>
</tr>
<tr>
<td>DeepSeek-R1-Distill-Qwen-7B</td>
<td>7B</td>
<td>Reasoning</td>
<td>Q4_K_L</td>
<td>5.09 GB</td>
</tr>
<tr>
<td>Mistral Nemo Instruct 2407</td>
<td>12B</td>
<td>Instruction</td>
<td>Q4_K_L</td>
<td>7.98 GB</td>
</tr>
<tr>
<td>Mistral Small 3.1 24B</td>
<td>24B</td>
<td>Instruction</td>
<td>Q4_K_S</td>
<td>13.5 GB</td>
</tr>
</tbody>
</table>
<p>The first three fit comfortably in 16GB with room for context. Mistral Small
3.1 at 13.5 GB leaves very little headroom on a 16GB machine. To browse other models,
<a href="https://huggingface.co/bartowski">bartowski's HuggingFace page</a> is a good
starting point for well-tested GGUF quantizations.</p>
<p><strong>Params</strong> (parameters) are the learned weights that make up the model. More
parameters generally means more capable, but also more RAM and slower inference.
7-8B is a practical sweet spot for 16GB machines.</p>
<p><strong>Quantization</strong> is how a full-precision model gets compressed to fit on
consumer hardware. The original weights are 16 or 32-bit floats; quantization
rounds them down to fewer bits, trading a small amount of accuracy for a large
reduction in size and RAM usage. <strong>Q4_K_L</strong> uses 4-bit weights with the
embedding and output layers kept at higher precision (8-bit) to preserve
quality. <strong>Q4_K_S</strong> is the smaller sibling: same 4-bit compression throughout,
including the embedding layers, shaving off a bit of quality for a smaller
file. For Mistral Small 3.1 that's the difference between 13.5 GB (K_S) and
14.8 GB (K_L). On a 16GB machine, that 1.3 GB matters, so K_S is the right
pick there.</p>
<p><strong>Instruction</strong> models are trained to follow directions and answer questions
directly. You ask, they answer. <strong>Reasoning</strong> models think out loud first,
working through the problem step by step before giving a final answer. Slower,
but more reliable on anything involving logic, math, or code.</p>
<h1 id="hardware">Hardware</h1>
<ul>
<li>MacBook Air M2, 16GB unified memory</li>
<li>macOS 26 (Tahoe)</li>
</ul>
<p>No active cooling on the Air, so it'll throttle during long runs. Fine for
interactive use.</p>
<h1 id="setup">Setup</h1>
<p>These steps are shared across all three models.</p>
<h2 id="1-install-llamacpp">1. Install llama.cpp</h2>
<p><a href="https://github.com/ggerganov/llama.cpp">llama.cpp</a> is the runtime that loads
and runs GGUF model files. It handles the actual inference: tokenizing your
input, running it through the model weights, and streaming tokens back to you.
Think of it as the engine; the GGUF file is the model it runs. It also handles
Metal GPU offloading on Apple Silicon, which is what makes local inference fast
enough to be usable.</p>
<pre><code class="language-bash">brew install llama.cpp
</code></pre>
<p>Gives you <code>llama-cli</code> for interactive chat and <code>llama-server</code> for an
OpenAI-compatible HTTP API.</p>
<h2 id="2-install-the-hugging-face-cli">2. Install the Hugging Face CLI</h2>
<p>Installing Python tools with plain <code>pip</code> can stomp on system Python or conflict
with other packages. <code>pipx</code> puts each CLI tool in its own isolated environment
and adds it to your PATH. No venv to activate.</p>
<pre><code class="language-bash">brew install pipx
pipx ensurepath
pipx install "huggingface_hub[cli]"
</code></pre>
<p><code>pipx ensurepath</code> adds pipx's bin dir to your PATH. Restart your terminal after
running it.</p>
<p>Note: the installed command is <code>hf</code>, not <code>huggingface-cli</code>. The name changed in
v1.x.</p>
<h1 id="the-models">The Models</h1>
<h2 id="qwen3-8b">Qwen3 8B</h2>
<p>Alibaba's Qwen3 8B (released April 2025) is a dense instruction-tuned model
that also supports a thinking mode. By default it answers directly. You can
enable step-by-step reasoning with <code>/think</code> in the chat, or disable it with
<code>/no_think</code>.</p>
<pre><code class="language-bash">hf download bartowski/Qwen_Qwen3-8B-GGUF Qwen_Qwen3-8B-Q4_K_L.gguf \
  --local-dir ~/models/qwen3-8b/
</code></pre>
<h2 id="deepseek-r1-distill-qwen-7b">DeepSeek-R1-Distill-Qwen-7B</h2>
<p>Note: despite the name, there is no 8B Qwen-based DeepSeek distill. The 8B
distill uses Llama architecture. This one is 7B on Qwen architecture, released
January 2025.</p>
<p>DeepSeek distilled the chain-of-thought behaviour from their R1 reasoning model
into this smaller base. It always shows its thinking inside <code>&#x3C;think>...&#x3C;/think></code>
tags before the final answer. Verbose for simple questions, but useful for
anything involving logic or code.</p>
<pre><code class="language-bash">hf download bartowski/DeepSeek-R1-Distill-Qwen-7B-GGUF \
  DeepSeek-R1-Distill-Qwen-7B-Q4_K_L.gguf \
  --local-dir ~/models/deepseek-r1-qwen-7b/
</code></pre>
<h2 id="mistral-small-31-24b">Mistral Small 3.1 24B</h2>
<p>Released March 17, 2025. Mistral's most capable small model at the time.
24B parameters, multimodal (supports image input), 128k context window,
Apache 2.0. Mistral officially recommends 32GB RAM for this one, so running
it on a 16GB M2 Air is pushing it. It loads and runs, but context has to stay
small.</p>
<p>Note: bartowski's repo for this model has a <code>mistralai_</code> prefix in the name,
unlike the others.</p>
<pre><code class="language-bash">hf download bartowski/mistralai_Mistral-Small-3.1-24B-Instruct-2503-GGUF \
  Mistral-Small-3.1-24B-Instruct-2503-Q4_K_S.gguf \
  --local-dir ~/models/mistral-small-3.1-24b/
</code></pre>
<p>K_S over K_L here because at 13.5 GB vs 14.8 GB, the difference determines
whether you have any RAM left for context and the OS.</p>
<h2 id="mistral-nemo-instruct-12b">Mistral Nemo Instruct 12B</h2>
<p>Mistral's 12B model (released July 2024, jointly with NVIDIA). Bigger than the
other two, which shows in the quality of longer-form responses. At Q4_K_L it's
7.98 GB, leaving ~6GB free for macOS and context.</p>
<pre><code class="language-bash">hf download bartowski/Mistral-Nemo-Instruct-2407-GGUF \
  Mistral-Nemo-Instruct-2407-Q4_K_L.gguf \
  --local-dir ~/models/mistral-nemo-12b/
</code></pre>
<h1 id="running-them">Running Them</h1>
<p>The flags are the same for all three. Swap the <code>-m</code> path.</p>
<h2 id="interactive-chat">Interactive chat</h2>
<pre><code class="language-bash"># Qwen3 8B
llama-cli \
  -m ~/models/qwen3-8b/Qwen_Qwen3-8B-Q4_K_L.gguf \
  -ngl 99 -c 8192

# DeepSeek-R1-Distill-Qwen-7B
llama-cli \
  -m ~/models/deepseek-r1-qwen-7b/DeepSeek-R1-Distill-Qwen-7B-Q4_K_L.gguf \
  -ngl 99 -c 8192

# Mistral Nemo 12B
llama-cli \
  -m ~/models/mistral-nemo-12b/Mistral-Nemo-Instruct-2407-Q4_K_L.gguf \
  -ngl 99 -c 8192

# Mistral Small 3.1 24B — tight on RAM, keep context small
llama-cli \
  -m ~/models/mistral-small-3.1-24b/Mistral-Small-3.1-24B-Instruct-2503-Q4_K_S.gguf \
  -ngl 99 -c 2048
</code></pre>
<p>Flags:</p>
<ul>
<li><code>-ngl 99</code>: offload all layers to Metal. Without this you're CPU-only and it's
very slow. <code>--fit on</code> instead might be better, not sure.</li>
<li><code>-c 8192</code>: context window. Comfortable with 16GB for the smaller two. Fine
for Mistral Nemo too given the headroom.</li>
</ul>
<p>Interactive mode is auto-enabled when the model has a chat template, which all
three do. The old <code>--interactive-first</code> flag was removed in newer llama.cpp.</p>
<h2 id="openai-compatible-server">OpenAI-compatible server</h2>
<pre><code class="language-bash">llama-server \
  -m ~/models/qwen3-8b/Qwen_Qwen3-8B-Q4_K_L.gguf \
  -ngl 99 -c 8192 --port 8080
</code></pre>
<p>Exposes <code>http://localhost:8080/v1/chat/completions</code>. Swap the model path for
whichever you want to serve.</p>
<h1 id="conclusion">Conclusion</h1>
<h2 id="notes-on-each">Notes on Each</h2>
<p><strong>Qwen3 8B</strong>: Pretty fast. The thinking thing was a bit slow but not bad. It mostly
stuck to the instructions I gave it, like "max 3 paragraphs", and was reasonably
creative. It was also kind of bland and generic, for better or worse.</p>
<p><strong>DeepSeek-R1-Distill-Qwen-7B</strong>: Also ok speed. It had a hard time following
instructions and sticking to the constraints that I'd tried to put in place, like that
same 3 paragraph limit. Overall I was pretty dissapointed given its reputation.</p>
<p><strong>Mistral Nemo 12B</strong> The direct responses felt fast even though it ran at about half
the tokens per second, due to it not working in Reasoning mode. The writing style of
Nemo was by far the best, and it adhered much better to the prompts.</p>
<p><strong>Mistral Nemo 24B</strong> Not enough RAM. I'd figured I could close things and make it work
but it's not worth it. Maybe on a better computer.</p>
<h2 id="not-enough-tokens">Not enough tokens</h2>
<p>The <code>8192</code> tokens they all use got to feeling pretty small, pretty fast. I tried to
push the continuity with a "Choose your own adventure" style conversation and it did
well until it just kind of stopped. I think I've seen claude crushing context down when
it gets near the limit, I wonder if there's a way to do that with this too, or get it
to do some sort of context shifting where it restarts with just the later context when
it runs low. Maybe I can run it in server mode and have a better client manage that,
but that seems like its own post. If you want to run models through an API
rather than locally, <a href="/openrouter-ai-tools.html">OpenRouter</a> is worth a look.</p>
<h2 id="winner">Winner</h2>
<p>Nemo was the pretty clear winner. I'm not sure if that's because of the size of the
model or something else, but it was a significant step up in its ability to stick to
the prompt's instructions, and the writing style was better too.</p>


<p>I wrote about this blog's <a href="/ai-generate-blog-images.html">automated image generation</a> previously. The short version: <code>generate-images.mjs</code> runs before each build, finds posts with missing images, builds a prompt from the <code>imgprompt</code> front matter field (or has <code>gpt-4o-mini</code> summarize the post), then calls DALL-E 3 to produce a 1024x1024 image.</p>
<p>It works, but DALL-E 3 is frustrating. It ignores prompt details, can't make a white background to save its life, and the results look more "AI art" than "clean icon." Time to see what else is out there.</p>
<p><em>Lesson Learned</em>: Don't let Cursor/Claude look online to find the most recent model versions. It got it wrong twice. I had to manually go back and update it after noticing that Gemini's 2.0-flash was not the newer Nano Banana (2.5, 3).</p>
<h1 id="top-models-today">Top Models Today</h1>
<p>This space moves fast. As of February 2026, these are the top 3 image generation models on the <a href="https://arena.ai/leaderboard/text-to-image">LM Arena text-to-image leaderboard</a> that have public APIs. These rankings are based on blind human preference voting and will almost certainly be different by the time you read this.</p>
<p><strong>ELO Scores:</strong> ELO is a rating system where users are shown two AI-generated images side by side (without knowing which model made which) and pick the one they prefer. Wins and losses shift each model's score, so a higher ELO means the model's images are consistently preferred by real people.</p>
<table>
<thead>
<tr>
<th>Model</th>
<th>Provider</th>
<th>Released</th>
<th>ELO</th>
<th>Notes</th>
</tr>
</thead>
<tbody>
<tr>
<td><strong>GPT Image 1.5</strong></td>
<td>OpenAI</td>
<td>December 2025</td>
<td>1248</td>
<td>4x faster than GPT Image 1. Token-based pricing, 20% cheaper than its predecessor.</td>
</tr>
<tr>
<td><strong>Nano Banana Pro</strong></td>
<td>Google (Gemini 3 Pro Image)</td>
<td>November 2025</td>
<td>1237</td>
<td>Native Gemini image gen. Strong multimodal integration.</td>
</tr>
<tr>
<td><strong>Flux 2 Max</strong></td>
<td>Black Forest Labs</td>
<td>November 2025</td>
<td>1169</td>
<td>32B parameter model. Megapixel-based pricing. BFL's highest quality tier.</td>
</tr>
</tbody>
</table>
<p>GPT Image 1.5 and Nano Banana Pro are close in quality. Flux 2 Max sits ~80 ELO points behind them but is the strongest non-Google non-OpenAI option available.</p>
<p><strong>What about open source?</strong> Flux 2 Dev is open-weight and Stable Diffusion 3.5 is fully open source, but neither is practical for this use case. Running them on a MacBook Air M2 (8GB unified memory, passive cooling) means throttling after ~90 seconds, a max practical resolution of 512x512, and needing to close basically every other app first. The quality gap vs. the API models is real too. For a build-time script that needs to reliably produce clean images, cloud APIs win.</p>
<h1 id="setup-gpt-image-15">Setup: GPT Image 1.5</h1>
<p>This is the easiest since the blog already uses OpenAI.</p>
<ol>
<li><strong>API Key</strong>: Already set up. Same <code>OPENAI_API_KEY</code> used for DALL-E 3.</li>
<li><strong>Model change</strong>: Update the <code>model</code> parameter from <code>dall-e-3</code> to <code>gpt-image-1.5</code> in the images API call.</li>
<li><strong>API differences</strong>: GPT Image 1.5 supports both the Images API (<code>/v1/images/generations</code>) and the newer Responses API. The Images API is the closest drop-in replacement. Quality can be set to <code>low</code>, <code>medium</code>, <code>high</code>, or <code>auto</code>. It also supports streaming partial renders and up to 16 reference images for editing.</li>
</ol>
<pre><code class="language-bash">export OPENAI_API_KEY="sk-..."
</code></pre>
<p>If you don't already have a key, create one at <a href="https://platform.openai.com/api-keys">platform.openai.com/api-keys</a>.</p>
<h1 id="setup-nano-banana">Setup: Nano Banana</h1>
<p>Nano Banana is Google's image generation feature built on Gemini. You can access it through the official Google Gemini API.</p>
<ol>
<li><strong>Get an API key</strong>: Go to <a href="https://aistudio.google.com/app/apikey">Google AI Studio</a> and create a new API key.</li>
<li><strong>Install the SDK</strong>:
<pre><code class="language-bash">npm install @google/genai
</code></pre>
</li>
<li><strong>Set the environment variable</strong>:
<pre><code class="language-bash">export GEMINI_API_KEY="your-key-here"
</code></pre>
</li>
<li><strong>API usage</strong>: Nano Banana image gen uses the Gemini API with <code>responseModalities: ["TEXT", "IMAGE"]</code>. The model name for the standard tier is <code>gemini-2.5-flash-preview-image</code> and for Pro it's <code>gemini-3-pro-image</code>.</li>
</ol>
<p>Note: the API flow is different from OpenAI. You send a chat completion-style request with image output enabled, and the response includes inline image data.</p>
<h1 id="setup-flux-2-max">Setup: Flux 2 Max</h1>
<p>Black Forest Labs runs their own API platform.</p>
<ol>
<li><strong>Create an account</strong>: Register at <a href="https://dashboard.bfl.ai">dashboard.bfl.ai</a> and confirm your email.</li>
<li><strong>Add credits</strong>: Navigate to API → Credits and add funds via Stripe. Pricing is simple: 1 credit = $0.01 USD.</li>
<li><strong>Create an API key</strong>: Go to API → Keys, click "Add Key", and copy it immediately (it's only shown once).</li>
<li><strong>Set the environment variable</strong>:
<pre><code class="language-bash">export BFL_API_KEY="your-key-here"
</code></pre>
</li>
<li><strong>API usage</strong>: POST to <code>https://api.bfl.ai/v1/flux-2-max</code> with your prompt, width, and height. The API is async — you submit a task and poll for the result using the returned task ID.</li>
</ol>
<h1 id="setup-verification">Setup Verification</h1>
<p>Once you've configured your API keys, you can verify everything is wired up correctly by running:</p>
<pre><code class="language-bash">node apps/blog/blog/scripts/generate-images.mjs --verify
</code></pre>
<p>This will check that each configured API key is valid and can reach its respective service without actually generating any images. We'll implement this flag in a follow-up change.</p>
<h1 id="cost">Cost</h1>
<p>All prices are USD per image at 1024x1024.</p>
<table>
<thead>
<tr>
<th>Model</th>
<th>Cost</th>
<th>Pricing Model</th>
<th>Notes</th>
</tr>
</thead>
<tbody>
<tr>
<td>DALL-E 3 (prior)</td>
<td>~$0.04</td>
<td>Per image</td>
<td>Fixed price, single quality tier</td>
</tr>
<tr>
<td>GPT Image 1.5</td>
<td>$0.009–$0.14</td>
<td>Per token (output)</td>
<td>Varies by quality: low ~$0.009, medium ~$0.03, high ~$0.14. ~20% cheaper than GPT Image 1</td>
</tr>
<tr>
<td>Nano Banana</td>
<td>~$0.02</td>
<td>Per image</td>
<td>Standard tier (Gemini 2.5 Flash). Pro tier is ~$0.10/image</td>
</tr>
<tr>
<td>Flux 2 Max</td>
<td>~$0.07</td>
<td>Per megapixel</td>
<td>Scales with resolution. 1024x1024 ≈ 1MP ≈ 7 credits</td>
</tr>
</tbody>
</table>
<p>The ambiguity here is mostly around GPT Image 1.5 — the token-based pricing means cost depends heavily on the quality setting and prompt length. For a fair comparison we'll use "medium" quality, which lands close to what we're paying for DALL-E 3 today. Flux 2 Max's megapixel pricing is predictable but gets more expensive at higher resolutions.</p>
<h1 id="results">Results</h1>
<p>Each model was given the same prompt, built from <code>imgprompt: "A cute robot wearing a short chefs hat and holding a paint brush"</code>:</p>
<pre><code class="language-text">Subject: "A cute robot wearing a short chefs hat and holding a paint brush".
Style: minimalist flat vector icon, clean lines, crisp edges, simplified geometric shapes.
Colors: black, white, and one primary accent color only. No gradients.
Composition: centered subject, generous negative space, wide landscape 16:9 aspect ratio.
Background: pure solid white (#FFFFFF), filling the entire image.
No text, no shadows, no textures, no transparency, no background elements.
</code></pre>
<p>All images were center-cropped to 533x300 (16:9) after generation.</p>
<h2 id="gpt-image-15-openai">GPT Image 1.5 (OpenAI)</h2>
<p>Model: <code>gpt-image-1.5</code>, size: 1024x1024, quality: medium. Estimated cost: ~$0.03.</p>
<p><img src="/images/ai-image-gen-bakeoff-openai.png" alt="GPT Image 1.5 result"></p>
<h2 id="gemini-20-flash-google">Gemini 2.0 Flash (Google)</h2>
<p>Model: <code>gemini-2.0-flash-exp-image-generation</code>. Estimated cost: ~$0.02 (free tier may apply).</p>
<p><img src="/images/ai-image-gen-bakeoff-gemini-2.0.png" alt="Nano Banana 2.0 result"></p>
<h2 id="nano-banana-gemini-25-flash">Nano Banana (Gemini 2.5 Flash)</h2>
<p>Model: <code>gemini-2.5-flash-image</code>. Estimated cost: ~$0.02.</p>
<p><img src="/images/ai-image-gen-bakeoff-gemini-2.5.png" alt="Nano Banana 2.5 Flash result"></p>
<h2 id="nano-banana-pro-gemini-3-pro">Nano Banana Pro (Gemini 3 Pro)</h2>
<p>Model: <code>gemini-3-pro-image-preview</code>. Estimated cost: ~$0.10.</p>
<p><img src="/images/ai-image-gen-bakeoff-gemini-3-pro.png" alt="Nano Banana 3 Pro result"></p>
<h2 id="flux-2-max-bfl">Flux 2 Max (BFL)</h2>
<p>Model: <code>flux-2-max</code>, size: 1024x576. Estimated cost: ~$0.04 (~0.6MP at $0.07/MP).</p>
<p><img src="/images/ai-image-gen-bakeoff-flux.png" alt="Flux 2 Max result"></p>
<h1 id="conclusion">Conclusion</h1>
<p>It's pretty close between OpenAI and Nano Banana 2.5. I'll likely switch back and forth between the two, but to pick a flat winner as default I'm going with OpenAI for now.</p>
<p>OpenAI has that annoying same-y generic feel that lets you know it's AI art, and I don't love that, but it's also definitely more cute.</p>
<p>Nano Banana's ones are pretty original. I like 3-pro the most but not the price. 2.5-flash is fun, a bit cleaner than OpenAI's art, just kind of <em>busy</em>. I'll probably pick it when I'm aiming for cleaner images than cute ones.</p>
<p>Flux's was more artsy. It's just kind of... weird? It kept drawing unique stuff but just sort of unsettling.</p>


<p>Making blog thumbnails and images is tedious. I don't want to fight with Gimp or whatever and I don't think anyone cares what the image really is.</p>
<p>To make publishing easier, this site now checks for the image in each markdown post during the build to see if it already exists. If not, a prompt is built from the post title and content and sent to OpenAI's image API. The returned art is saved alongside the post and a 70×70 thumbnail is produced using Sharp.</p>
<p>To make the image for this post it cost me about $0.12 CAD in OpenAI credits. Entertainingly that's probably more than what hosting this site will cost for like a week due to the setup I'm using to host this.</p>
<p>This automation was initially coded with Codex, but it didn't work quite right. I fixed it up with Cursor, then a bit manually to get it across the finish line. This was as much about playing with the AI dev tools as building the feature.</p>
<p>Check it out here <a href="https://github.com/kylep/multi/blob/main/apps/blog/blog/scripts/generate-images.mjs">generate-images.mjs</a>.</p>
<h1 id="how-it-works">How It Works</h1>
<p>The image generation system is integrated into the build pipeline through the <code>package.json</code> build script:</p>
<pre><code class="language-json">"build": "node scripts/generate-images.mjs &#x26;&#x26; next build"
</code></pre>
<p>Before Next.js builds the static site, the <code>generate-images.mjs</code> script runs and:</p>
<ol>
<li><strong>Scans all markdown files</strong> in <code>markdown/posts/</code> for front matter</li>
<li><strong>Checks for missing images</strong> referenced in the <code>image</code> and <code>thumbnail</code> fields</li>
<li><strong>Generates missing images</strong> using OpenAI's DALL·E API if <code>OPENAI_API_KEY</code> is set</li>
<li><strong>Creates thumbnails</strong> by resizing generated images to 70×70 pixels using Sharp</li>
</ol>
<h2 id="the-generation-process">The Generation Process</h2>
<p>It's all pretty straightforward:</p>
<p>High level script logic:</p>
<ol>
<li>If there's no image defined, do nothing</li>
<li>If the image is defined in the front matter but doesn't exist, generate it.</li>
<li>If the image does exist but thumbnail doesn't, resize it and save that too.</li>
</ol>
<p>Generating the image is also simple:</p>
<ol>
<li>If the front matter has <code>imgprompt</code> in it, use that as the <code>summary</code> of the post.</li>
<li>If <code>imgprompt</code> is not present get <code>gpt-4o-mini'</code> to write a <code>summary</code> of the post.</li>
<li>Use <code>dall-e-3</code> to create the image.</li>
<li>Complain that it didn't look right, tweak the prompt, repeat until sufficiently frustrated, move on.</li>
</ol>
<h3 id="prompts">Prompts</h3>
<p>At time of writing (I won't keep this updated, check the code), I used these for the prompts:</p>
<p>Making the summary:</p>
<pre><code class="language-javascript">async function summarizeBlogContent(data, content) {
  const openai = new OpenAI({ apiKey: process.env.OPENAI_API_KEY });
  const prompt = await openai.chat.completions.create({
    model: 'gpt-4o-mini',
    messages: [{ role: 'user', content: `In max 20 words, describe a simple logo image that would represent this blog post: ${content}` }],
  });
  const summary = prompt.choices[0].message.content;
  console.log(`✨📝 Created summary of "${data.title}" → ${summary}`);
  return summary;
}
</code></pre>
<p>Making the image</p>
<pre><code class="language-javascript">async function getPromptFromContent(data, content) {
  const subject = data.imgprompt
    ? data.imgprompt
    : await summarizeBlogContent(data, content);
  const prompt = [
    "Simple minimalist flat vector icon.",
    `Subject: "${subject}".`,
    "Style: clean flat SVG-style vector, crisp edges, no gradients, no shadows, no textures, no lighting effects.",
    "Colors: limited palette, high contrast, modern minimal. Black, white, and primary colors only.",
    "Composition: centered subject with generous whitespace, 1:1 icon framing.",
    "Background: pure solid white (#FFFFFF) and completely filled. Nothing around the logo.",
    "No transparency. No checkerboard. No grid. No background elements.",
    "No text. No words. Just a simple image centered in white space."
  ].join("\n");
  return prompt;
}
</code></pre>
<h3 id="environment-setup">Environment Setup</h3>
<p>Set your OpenAI API key</p>
<pre><code class="language-bash">export OPENAI_API_KEY="your-api-key-here"
</code></pre>
<p>Without this key the build just skips image generation with a warning.</p>
<hr>
<h1 id="outstanding-problem">Outstanding Problem</h1>
<p>Dalle-3 is super frustrating when it comes to just making a white background. I spent a couple hours trying different prompts and it just refuses. Here's what the code came up with as a prompt for this page:</p>
<pre><code class="language-text">Simple minimalist flat vector icon.
Subject: "A simple logo featuring a stylized laptop and a paintbrush, symbolizing effortless content creation and automated design.".                     
Style: clean flat SVG-style vector, crisp edges, no gradients, no shadows, no textures, no lighting effects.                                                                                                
Colors: limited palette, high contrast, modern minimal. Black, white, and primary colors only.   
Composition: centered subject with generous whitespace, 1:1 icon framing.                             
Background: pure solid white (#FFFFFF) and completely filled. Nothing around the logo.                
No transparency. No checkerboard. No grid. No background elements.                                    
No text. No words. Just a simple image centered in white space. 
</code></pre>
<p>It does not seem to care for my preference. If anyone has any tricks for this I'd love to hear about it.</p>


Kyle Pericak

"It works in my environment"

Categories

Tags