<h2 id="table-of-contents">Table of contents</h2>
<ul>
<li><a href="#the-goal">The goal</a></li>
<li><a href="#the-journalist-agent">The journalist agent</a></li>
<li><a href="#how-the-pieces-connect">How the pieces connect</a></li>
<li><a href="#controller-and-runtime">Controller and runtime</a></li>
<li><a href="#agenttask-crd">AgentTask CRD</a></li>
<li><a href="#openrouter-for-api-auth">OpenRouter for API auth</a></li>
<li><a href="#permissions-with-allowedtools">Permissions with allowedTools</a></li>
<li><a href="#webhook-trigger-on-demand">Webhook: trigger on demand</a></li>
<li><a href="#helm-chart-and-deployment">Helm chart and deployment</a></li>
</ul>
<h2 id="the-goal">The goal</h2>
<p>I want AI agents that run autonomously on a cron schedule or in
response to events, managed by Kubernetes.</p>
<p>The demo task is a daily AI news digest. Every morning at 8am,
an agent searches the web for yesterday's AI news, writes a
digest to the wiki, and commits. It's also triggerable on demand
via webhook.</p>
<p>K8s handles the scheduling. A custom controller watches a CRD
and creates Jobs. The agent runs in a container, writes to a
shared volume, and the controller tracks the result. Agents
commit locally but don't push. I review and push manually.</p>
<h2 id="the-journalist-agent">The journalist agent</h2>
<p>I wrote a new Claude Code agent for this:
<a href="https://github.com/kylep/multi/blob/main/.claude/agents/journalist.md">journalist</a>.
Sonnet, not Opus. This runs daily, and the task is search and
summarize. Sonnet is cheaper and good enough.</p>
<p>The agent searches for yesterday's AI news, writes a digest to
<code>wiki/journal/YYYY-MM-DD/ai-news.md</code> with source URLs for every
item, and commits. It determines the date itself via <code>date</code>.</p>
<p>Early versions used <code>WebSearch</code>, which burned through context
with HTML scraping. I built a small
<a href="/wiki/custom-tools/google-news-mcp.html">GNews MCP server</a>
that wraps the GNews API. Structured JSON responses, fewer
tokens. The agent runs six parallel queries (<code>OpenAI</code>,
<code>Anthropic</code>, <code>Google AI</code>, <code>NVIDIA AI</code>, <code>AI startup</code>, plus
tech headlines) and deduplicates the results.</p>
<p>I also wrote a
<a href="/wiki/custom-tools/discord-mcp.html">Discord MCP server</a>
so the agent posts its digest to the <code>#news</code> channel in my
Discord bot server.</p>
<h2 id="how-the-pieces-connect">How the pieces connect</h2>
<pre class="mermaid">
graph TD
    AT[daily-ai-news AgentTask] -->|watched by| CTRL[Controller]
    CTRL -->|8am cron| JOB[K8s Job]
    JOB -->|init| GIT[git fetch + reset]
    GIT -->|then| AGENT[Journalist Agent]
    AGENT -->|writes| WIKI[wiki/journal/YYYY-MM-DD/ai-news.md]
    AGENT -->|commits| PVC[Shared PVC]
    AGENT -->|posts via MCP| DISCORD[Discord #news]
    WH[Webhook :8080] -->|creates| AT
</pre>
<p>The controller is a Go binary running as a Deployment. It watches
<code>AgentTask</code> custom resources in the <code>ai-agents</code> namespace.</p>
<p>Each Job has two containers:</p>
<ol>
<li><strong>Init container</strong> (<code>alpine/git</code>): fetches the repo, resets
to the configured branch, then <code>chown</code>s the workspace to
uid 1000 (the agent user).</li>
<li><strong>Main container</strong> (<code>ai-agent-runtime</code>): writes MCP config,
then runs <code>claude --mcp-config ... --agent journalist -p "..." --allowedTools ...</code></li>
</ol>
<h2 id="controller-and-runtime">Controller and runtime</h2>
<p>The
<a href="https://github.com/kylep/multi/tree/main/infra/ai-agents/agent-controller/pkg/controller">controller</a>
is a Go reconcile loop that lists all AgentTasks every 30 seconds.
For scheduled tasks, it compares <code>lastRunTime</code> against the cron
expression and creates a Job when due. For manual and webhook
tasks, it creates a Job immediately when phase is <code>Pending</code>.</p>
<p>The runtime image is an Alpine container with Claude Code and
OpenCode baked in (<code>infra/ai-agents/ai-agent-runtime/Dockerfile</code>). Runs
as uid 1000.</p>
<h2 id="agenttask-crd">AgentTask CRD</h2>
<p>The custom resource definition gives <code>kubectl get agenttasks</code>
for free. Here's the daily news task:</p>
<pre><code class="language-yaml">apiVersion: agents.kyle.pericak.com/v1alpha1
kind: AgentTask
metadata:
  name: daily-ai-news
spec:
  agent: journalist
  runtime: claude
  prompt: >-
    Search for yesterday's most notable AI news...
  schedule: "0 8 * * *"
  trigger: scheduled
  readOnly: false
  allowedTools: >-
    WebSearch,WebFetch,Read,Glob,Grep,Write,
    Bash(mkdir *),Bash(git add *),
    Bash(git commit *),Bash(date *)
</code></pre>
<p>The controller splits <code>allowedTools</code> on commas and passes each
one as a separate <code>--allowedTools</code> flag to Claude Code. This is
how you run Claude Code autonomously without
<code>--dangerously-skip-permissions</code>.</p>
<table>
<thead>
<tr>
<th>Field</th>
<th>Purpose</th>
</tr>
</thead>
<tbody>
<tr>
<td><code>agent</code></td>
<td>Name from <code>.claude/agents/</code></td>
</tr>
<tr>
<td><code>runtime</code></td>
<td><code>claude</code> or <code>opencode</code></td>
</tr>
<tr>
<td><code>prompt</code></td>
<td>The <code>-p</code> argument</td>
</tr>
<tr>
<td><code>schedule</code></td>
<td>Cron expression (empty = one-shot)</td>
</tr>
<tr>
<td><code>trigger</code></td>
<td><code>manual</code>, <code>scheduled</code>, or <code>webhook</code></td>
</tr>
<tr>
<td><code>readOnly</code></td>
<td>Whether this agent modifies the repo</td>
</tr>
<tr>
<td><code>allowedTools</code></td>
<td>Comma-separated tool permissions</td>
</tr>
</tbody>
</table>
<h2 id="openrouter-for-api-auth">OpenRouter for API auth</h2>
<p>Claude Code normally uses <code>ANTHROPIC_API_KEY</code> to talk to
Anthropic's API directly. In a container, I route it through
<a href="/openrouter-ai-tools.html">OpenRouter</a> instead. One API key
for all model providers, unified billing.</p>
<p>The Helm chart's Secret injects these env vars into every
agent container:</p>
<pre><code class="language-yaml">ANTHROPIC_BASE_URL: https://openrouter.ai/api
ANTHROPIC_AUTH_TOKEN: &#x3C;openrouter-key>
ANTHROPIC_API_KEY: ""
CLAUDE_CODE_DISABLE_NONESSENTIAL_TRAFFIC: "1"
</code></pre>
<p><code>ANTHROPIC_API_KEY</code> must be empty or Claude Code tries to use
it and the auth conflicts.</p>
<h2 id="permissions-with-allowedtools">Permissions with allowedTools</h2>
<p>Claude Code prompts for permission before using tools. In a
container with no TTY, it just hangs. The fix is
<code>--allowedTools</code>, which auto-approves specific tools.</p>
<p>Each tool needs its own flag. Comma-separated doesn't work:</p>
<pre><code class="language-bash"># wrong
claude -p "..." --allowedTools "WebSearch,Write,Read"

# right
claude -p "..." \
  --allowedTools "WebSearch" \
  --allowedTools "Write" \
  --allowedTools "Read"
</code></pre>
<p>Tools with specifiers use parentheses for scoping:</p>
<pre><code class="language-bash">--allowedTools 'Bash(mkdir *)'    # only mkdir
--allowedTools 'Bash(git add *)'  # only git add
--allowedTools 'Write'            # any file
</code></pre>
<p>I tried scoping Write to a specific path with
<code>Write(apps/blog/blog/markdown/wiki/journal/**)</code> but it didn't
match inside the container. The pattern works locally but fails
when the working directory is <code>/workspace/repo</code>. Unrestricted
<code>Write</code> works. The agent definition and Bash scoping still
constrain what gets written.</p>
<h2 id="webhook-trigger-on-demand">Webhook: trigger on demand</h2>
<p>The controller exposes <code>:8080/webhook</code> for on-demand runs.
Requires a bearer token via <code>AI_WEBHOOK_TOKEN</code> env var (skipped
if empty for local dev):</p>
<pre><code class="language-bash">curl -X POST http://localhost:8080/webhook \
  -H "Authorization: Bearer $AI_WEBHOOK_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{
    "agent": "journalist",
    "prompt": "Search for today AI news...",
    "allowedTools": "WebSearch,WebFetch,Read,Write,..."
  }'
</code></pre>
<p>The handler creates an AgentTask CR with the allowedTools, and
the reconcile loop picks it up.</p>
<h2 id="helm-chart-and-deployment">Helm chart and deployment</h2>
<p>The
<a href="https://github.com/kylep/multi/tree/main/infra/ai-agents/agent-controller/helm">Helm chart</a>
packages everything: CRD, controller Deployment, ServiceAccount,
RBAC, Secrets, and PVC. hostPath volume, single node.</p>
<pre><code class="language-bash">helm install agent-controller ./helm \
  -n ai-agents --create-namespace \
  -f values-override.yaml
</code></pre>
<p>The OpenRouter key has special characters that <code>--set</code> mangles.
Use a values file instead:</p>
<pre><code class="language-yaml">secrets:
  openrouterApiKey: "&#x3C;your-openrouter-key>"
  discordBotToken: "&#x3C;your-discord-bot-token>"
  discordGuildId: "&#x3C;your-guild-id>"
repo:
  branch: kyle/blog-k8s-autonomous-agents-mvp
</code></pre>


<h2 id="table-of-contents">Table of contents</h2>
<ul>
<li><a href="#a-warning-this-didnt-work-well">A warning: This didn't work well</a></li>
<li><a href="#why-an-org-chart-for-ai-agents">Why an org chart for AI agents</a></li>
<li><a href="#the-agents">The agents</a></li>
<li><a href="#the-org-chart">The org chart</a></li>
<li><a href="#pai-the-executive-assistant">Pai: the executive assistant</a>
<ul>
<li><a href="#agent-invocation-via-bash">Agent invocation via Bash</a></li>
</ul>
</li>
<li><a href="#the-wiki-layer">The wiki layer</a></li>
<li><a href="#pai-in-action">Pai in action</a>
<ul>
<li><a href="#the-quarterly-health-check">The quarterly health check</a></li>
</ul>
</li>
</ul>
<h1 id="a-warning-this-didnt-work-well">A warning: This didn't work well</h1>
<p>I found the process of building and testing this really fun. Trying to actually use it
afterwards was less fun.</p>
<ul>
<li><code>2026-03-11</code>: this is a "Hey this is neat" thing, not a  "this works really well!" one.</li>
<li><code>2026-03-15</code>: I've deleted basically all these agents and am down to 7.</li>
<li>I haven't given up on this idea but this approach was too ambitious and didn't have
the other infra in place it'd need to succeed.</li>
</ul>
<h1 id="why-an-org-chart-for-ai-agents">Why an org chart for AI agents</h1>
<p>Because it's cool, mostly. Also I'd hoped it would work really well.
But mostly, it's cool.</p>
<p>In this experiment I defined a team of Claude Code agents that each had a named role,
themed around being a virtual agent company. I figured I could run them manually or in
K8s using the Claude CLI:</p>
<pre><code class="language-bash">claude --agent cmo -p "What are my top 5 posts by pageviews?"
</code></pre>
<p>In theory I could manage agent context with this approach effectively and get each one
to stay in its own lane without bloating Claude's global context. CMO started with CMO
prompt, CDO with CDO prompt, so on.</p>
<p>The hope was to have these things each coordinate on their own, with my Pai agent acting
as a top-level orchestrator. They should have been able to use the wiki to exchange context.</p>
<p>I repurposed my OpenClaw agent
<a href="/openclaw-linear-skill.html">Pai</a>'s name when defining my orchestrator. I hoped to be
able to ask for things in Telegram and Pai would decompose the request into a
multi-agent flow and orchestrate from there.</p>
<p>The C-suite agents (CMO, CFO, CTO, CDO, CSO) each were envisioned as sub-orchestrators
that ran their own adversarial loops to keep standards high and sub-route requests.</p>
<h1 id="the-agents">The agents</h1>
<table>
<thead>
<tr>
<th>Agent</th>
<th>Role</th>
<th>Model</th>
<th>Key Tools</th>
</tr>
</thead>
<tbody>
<tr>
<td>Pai</td>
<td>Orchestration</td>
<td>Sonnet</td>
<td>Bash, Linear MCP</td>
</tr>
<tr>
<td>CMO</td>
<td>Traffic and growth</td>
<td>Sonnet</td>
<td>GA4 Analytics MCP</td>
</tr>
<tr>
<td>CFO</td>
<td>AI spend</td>
<td>Sonnet</td>
<td><a href="/openrouter-ai-tools.html">OpenRouter MCP</a></td>
</tr>
<tr>
<td>CTO</td>
<td>Delivery, blockers</td>
<td>Sonnet</td>
<td>Linear MCP, Bash</td>
</tr>
<tr>
<td>CDO</td>
<td>Knowledge management</td>
<td>Sonnet</td>
<td>Wiki read/write, Bash</td>
</tr>
<tr>
<td>CSO</td>
<td>Security and privacy</td>
<td>Sonnet</td>
<td>File tools, Bash</td>
</tr>
<tr>
<td>AR</td>
<td>Agent onboarding, mediation</td>
<td>Sonnet</td>
<td>File tools, Bash</td>
</tr>
<tr>
<td>Publisher</td>
<td>Blog content pipeline</td>
<td>Sonnet</td>
<td>Bash, file tools</td>
</tr>
<tr>
<td>SEO</td>
<td>Search audits</td>
<td>Sonnet</td>
<td>GA4, WebSearch</td>
</tr>
<tr>
<td>Cost Tracker</td>
<td>Spend reports</td>
<td>Haiku</td>
<td>OpenRouter MCP</td>
</tr>
<tr>
<td>Librarian</td>
<td>Wiki read/write</td>
<td>Haiku</td>
<td>Wiki file tools</td>
</tr>
<tr>
<td>Privacy Auditor</td>
<td>Flag confidential data</td>
<td>Haiku</td>
<td>File tools</td>
</tr>
<tr>
<td>Researcher</td>
<td>Gather sourced facts</td>
<td>Sonnet</td>
<td>WebSearch, Read</td>
</tr>
<tr>
<td>Writer</td>
<td>Draft posts from briefs</td>
<td>Sonnet</td>
<td>Read, Write</td>
</tr>
<tr>
<td>Fact Checker</td>
<td>Verify claims</td>
<td>Haiku</td>
<td>WebSearch, Read</td>
</tr>
<tr>
<td>Reviewer</td>
<td>Style and structure</td>
<td>Haiku</td>
<td>Read, Grep</td>
</tr>
<tr>
<td>QA</td>
<td>Production readiness</td>
<td>Sonnet</td>
<td>Bash, Playwright</td>
</tr>
</tbody>
</table>
<p>My only real use case at the time was writing blog posts, so the Publisher was special,
getting its own sub-org with a four-stage content
pipeline: researcher, writer, fact-checker, and reviewer.</p>
<p>QA ran after the pipeline to verify the post actually
built, rendered, and linked correctly before it went live.
QA used <a href="/playwright-mcp.html">Playwright MCP</a> to take screenshots
and inspect the rendered page.</p>
<p>Every agent connected to real
<a href="https://modelcontextprotocol.io/">MCP</a> servers to do cool stuff. The
CMO queried real GA4 data. The CFO pulled real OpenRouter
bills.</p>
<h1 id="the-org-chart">The org chart</h1>
<p>This was accurate as of 2026-03-12. The
<a href="/wiki/projects/agent-team/org-chart.html">bot-wiki org chart</a>
stayed up to date as agents were added or reorganized.</p>
<pre class="mermaid">
graph TD
    Kyle["Kyle"]
    Pai["Pai: Executive Assistant"]
    CMO["CMO: Grow Readership"]
    CFO["CFO: Optimize Spend"]
    CTO["CTO: Delivery"]
    CDO["CDO: Knowledge"]
    CSO["CSO: Security"]
    AR["AR: Agent Resources"]
    Publisher["Publisher: Content Pipeline"]

    SEO["SEO Subagent"]
    CostTracker["Cost Tracker"]
    Librarian["Librarian"]
    PrivacyAuditor["Privacy Auditor"]

    Researcher["Researcher"]
    Writer["Writer"]
    FactChecker["Fact Checker"]
    Reviewer["Reviewer"]
    QA["QA"]

    Kyle --> Pai
    Kyle --> CMO
    Kyle --> CFO
    Kyle --> CTO
    Kyle --> CDO
    Kyle --> CSO
    Kyle --> AR
    Kyle --> Publisher

    Pai -.->|orchestrates| CMO
    Pai -.->|orchestrates| CFO
    Pai -.->|orchestrates| CTO
    Pai -.->|orchestrates| CDO
    Pai -.->|orchestrates| CSO
    Pai -.->|orchestrates| AR
    Pai -.->|orchestrates| Publisher

    CMO --> SEO
    CFO --> CostTracker
    CDO --> Librarian
    CSO --> PrivacyAuditor

    CMO -.->|reads/writes| Librarian
    CFO -.->|reads/writes| Librarian
    CTO -.->|reads/writes| Librarian

    Publisher --> Researcher
    Publisher --> Writer
    Publisher --> FactChecker
    Publisher --> Reviewer
    Publisher --> QA
</pre>
<p>Every agent was directly
invocable by me, but ideally I didn't need to do that often.</p>
<p>Solid lines meant "reports to." Dashed lines from Pai meant
"orchestrates." Dashed lines to the Librarian meant "can
read/write wiki through."</p>
<p>The Privacy Auditor was the gatekeeper. Any agent writing
content that would end up in git or on the internet should
have checked with the Privacy Auditor first. It scanned for
leaked analytics data, spend numbers, secrets, and anything
else that shouldn't be public. This came from a scare where one of my agents with access
to mildly confidential data referenced it in content that
would have been public. I feel strongly that all agent flows
need a privacy step at minimum, and ideally agents should
have purposefully curtailed access to sensitive data.</p>
<p>The Librarian was an experiment in context management and
shared state. Any agent could talk to it directly to persist
notes, plans, or evidence to the wiki. I had already built a
<a href="/wiki-rag.html">RAG system</a> over the wiki. Once the wiki
got big enough that agents couldn't skim it all, I'd have
them query the RAG index instead. At that point, the agents
just used git and local file storage for wiki access. The CDO
owned the wiki strategy, but the Librarian did the actual
reading and writing.</p>
<h1 id="pai-the-executive-assistant">Pai: the executive assistant</h1>
<p>Agent definitions lived in <code>.claude/agents/</code>. Here was Pai's
frontmatter:</p>
<pre><code class="language-yaml"># .claude/agents/pai.md
name: pai
description: >-
  Pai — Executive assistant that orchestrates
  multi-agent workflows
model: sonnet
tools:
  - Bash
  - Read
  - Glob
  - Grep
  - Write
  - mcp__linear-server__list_issues
  - mcp__linear-server__list_projects
</code></pre>
<h2 id="agent-invocation-via-bash">Agent invocation via Bash</h2>
<p>Pai invoked other agents with <code>claude --agent &#x3C;name> -p "prompt"</code> through the Bash tool.
In theory I could have made an MCP for Pai that ran that in k8s pods for OpenClaw.
Each call was a fresh session. Agents didn't share memory or context with each other.</p>
<pre><code class="language-text">*Update*: The alternative I didn't try was just to use Claude Subagents for local work.
That's much easier and works better.
</code></pre>
<p>The alternatives I considered:</p>
<ul>
<li><strong>Shared memory / message bus.</strong> Tools like LangGraph or
CrewAI give agents shared state. More powerful, but more
moving parts. I wanted something I could debug by reading
a bash script.</li>
<li><strong>Single mega-agent.</strong> One agent definition with all the
tools. Simpler, but the context window filled up fast and
the agent lost focus. Splitting by role kept each
session lean.</li>
<li><strong>MCP-based orchestration.</strong> Route through an MCP server
that managed agent sessions. Interesting, but overkill
for a blog. I'd rather have built on <code>claude --agent</code>
which already worked.</li>
<li><strong><a href="https://github.com/steveyegge/beads">Beads</a>.</strong> A
structured task memory for AI agents, backed by Dolt
(version-controlled SQL). I looked into it, but it was
more of an alternative to Linear than to the wiki. It
solved task tracking and agent handoff, not knowledge
management. I liked Linear for tasks and markdown files
for knowledge.</li>
</ul>
<p>I went with the dumb approach: Bash calls and text passing.
It was easy to understand, easy to debug, and the wiki
handled long-term memory. If I outgrew it, I'd upgrade.</p>
<h1 id="the-wiki-layer">The wiki layer</h1>
<p>Each agent had a page in the
<a href="/bot-wiki.html">Bot-Wiki</a> that documented its goal, tools,
subagents, and example prompts.</p>
<pre><code class="language-text">agent-team/
├── index.md          # org chart and coordination model
├── pai.md            # orchestration agent
├── cmo.md            # traffic and growth
├── cfo.md            # AI spend
├── cto.md            # delivery and blockers
├── cdo.md            # knowledge management
├── cso.md            # security and privacy
├── publisher.md      # blog pipeline
└── phase-2.md        # future async architecture
</code></pre>
<p>Every agent session was stateless. The CMO didn't remember
what the CTO said last week. But if the CTO wrote its
findings to the wiki through the Librarian, the CMO could
read them next time it ran. The wiki was how agents shared
context across sessions.</p>
<p>The Librarian (a Haiku subagent under the CDO) handled
all wiki read/write operations. Any agent could invoke it
directly with <code>claude --agent librarian -p "..."</code> to
persist notes, plans, evidence, or whatever else needed to
survive between sessions.</p>
<p>The CDO owned the strategy: what got documented, how pages
were structured, when content was stale. This was cheaper
than giving every agent write access to the full file
system. The Librarian ran on Haiku, knew the wiki format,
and wouldn't accidentally clobber unrelated files.</p>
<p>In time I would have had the CFO tweak the models the
agents used to manage costs, too.</p>
<h1 id="pai-in-action">Pai in action</h1>
<h2 id="the-quarterly-health-check">The quarterly health check</h2>
<p>I asked Pai for a quarterly health check. It invoked three
agents and synthesized everything into one report.</p>
<p><strong>Traffic</strong> (from CMO): GA4 only had a week of data. The
site was new enough that there was no meaningful baseline yet.
Almost entirely direct traffic, organic search barely
registering. CMO recommended connecting Search Console and
revisiting in 30 days.</p>
<p><strong>AI spend</strong> (from CFO): well under budget with no pressure.
OpenRouter's API only surfaced account-level totals, no
per-model breakdown. CFO noted the biggest future lever was
routing non-reasoning tasks to flash-tier models.</p>
<p><strong>Delivery</strong> (from CTO): most issues done or in backlog,
one in progress (this blog post). No hard blockers, but
three dependency chains worth watching:</p>
<ol>
<li>RSS feed was high priority but hadn't started. Four
distribution issues were gated on it.</li>
<li>CMO and CFO baseline runs hadn't started. They gated
two future blog posts.</li>
<li>This post had an open PR to close.</li>
</ol>
<p>Pai wrote a log entry automatically:</p>
<pre><code class="language-markdown">## 21:30 — Quarterly health check

| Agent | Prompt Summary | Result |
|-------|---------------|--------|
| cmo | 90-day traffic trend | success |
| cfo | 90-day AI spend | success |
| cto | Blocked/stalled issues | success: 3 clusters |

**Synthesis:** Early launch phase, minimal spend,
no hard blockers but three dependency chains.
</code></pre>


Kyle Pericak

"It works in my environment"

Categories

Tags