← all projects
in progressJuly 2026 to present

Private local AI system

THE-BRAIN: Local Personal AI System

A local-first personal AI system designed around source-backed memory, persistent agent identities, policy-enforced actions, and recovery that works before the more ambitious runtime is allowed to exist.

My role
System owner, product architect, and implementation lead
Context
Independent project, July 2026 to present
Reading time
9 min read
Private local AI system

01

More than a searchable notes folder

THE-BRAIN started from a practical problem: project knowledge was spread across repositories, session histories, documents, decisions, and unfinished work. A larger notes vault would have made storage easier without making the information more trustworthy or usable.

The system is designed as an operational memory layer. Durable claims need sources, scope, timestamps, and review. Agents should receive small context packs compiled for the task instead of raw history. Contradictions and stale facts should become visible records, not quiet prompt noise.

02

Implemented now: the operational foundation

Milestone 1 is complete. The repository contains the signed-off contracts, architecture decisions, acceptance tests, host-readiness evidence, and the operating rules that later milestones must follow. The first production code focused on recovery because a memory system that cannot survive failure is not a memory system I can rely on.

The backup workflow creates encrypted snapshots, handles live SQLite databases through the SQLite backup API, writes a manifest with hashes, excludes credential paths, and records secret-free logs. Restore is attended and defensive: it validates archive members, rejects path traversal and links, checks restored hashes, verifies SQLite integrity, and works through a disposable staging area before accepting the result.

  • Encrypted snapshots and a nightly Windows scheduled task are operational.
  • A real backup and restore rehearsal verified matching content and SQLite integrity.
  • The source repository and heavy runtime data remain separate by design.

03

Designed next: kernel, memory, and persistent agents

The next approved direction is a native-Windows FastAPI kernel with SQLite WAL storage, FTS5 search, source manifests, graph and citation indexes, a compiled Markdown wiki, background jobs, and a context-pack compiler. A phone-first Next.js PWA would reach the local system through Tailscale HTTPS rather than a public endpoint.

The agent model is deliberately stricter than a collection of prompts. JARVIS, EDITH, and the specialist council are persistent identities with registry profiles, lifecycle state, scoped memory, allowed tools, approval tiers, wake triggers, and handoff contracts. EDITH is intended to enforce permission and approval decisions in the kernel. These pieces are the roadmap for Milestone 2 and later; they are not presented as running software today.

04

Policy before autonomy

Ingested documents are treated as untrusted data, never as instructions. A source can be archived, summarized, cited, quarantined, or proposed for review, but it cannot rewrite the system's behavior. Durable memory writes require traceability and remain reviewable.

The same rule applies to actions. External writes, destructive commands, credential changes, and money movement require an approval decision. The point is not to make the assistant timid. It is to make increasing capability possible without hiding risk inside prompt conventions.

05

What's left

Milestone 1 solved recovery, not the assistant. The remaining approved roadmap is substantial and deliberately gated so the project cannot jump from a sound design to unsafe autonomy.

  • Approve and implement Milestone 2: the FastAPI kernel, SQLite schemas, agent registry loader, audit and run logging, jobs table, and worker.
  • Rehearse backup and restore against the real kernel databases, then register kernel and worker supervision in Task Scheduler before memory ingestion begins.
  • Build opt-in repository discovery, redaction, source manifests, FTS5 search, the review inbox, and the Error Book.
  • Add budgeted context packs and source-backed claims before any persistent agent can answer from durable memory.
  • Implement EDITH approvals, local inference adapters, the JARVIS loop, phone-first PWA, MCP server, and FRIDAY briefings in milestone order.

06

Future work (maybe)

Anything beyond the signed roadmap is still a maybe. It belongs here as a direction to test later, not as a promised feature.

  • Encrypted replication between my own devices so the memory layer can survive a workstation replacement without becoming a cloud account.
  • Additional local model and hardware adapters when they improve a measured workflow, not simply because another model exists.
  • A small simulation environment for testing agent handoffs and approval failures before new capabilities touch real repositories or accounts.
  • Selective integrations with calendars, email, and other personal systems only after EDITH's audit and consent model has proved reliable.

Outcomes

  1. 01A signed-off, testable architecture and milestone plan for a local personal AI system.
  2. 02Operational encrypted backup, attended restore, integrity verification, and scheduled execution.
  3. 03A clear implementation boundary between completed Milestone 1 operations and the next kernel/runtime milestone.

What I learned

  1. 01Recovery and data custody belong at the start of a memory-system roadmap, not at the end.
  2. 02Persistent agent identity needs enforcement in state, tools, memory, and handoffs. A character prompt is not enough.
  3. 03Source-backed memory is slower to design than raw retrieval, but it gives future answers something solid to stand on.