Back to portfolio

Designing a Personal AI Operating System

7 min read View repo

The Vision

What if you could tell your computer "when a client emails me with an invoice, save it to their folder" and it just... worked? No IFTTT recipes, no Zapier flows, no code. Just describe what you want in plain English.

PAI (Personal AI OS) is my attempt at building this. It's a CLI tool that parses natural language into automation specifications, connects to your services via MCP (Model Context Protocol), and executes automations on your behalf.

Architecture

PAI follows a deliberately flat architecture — one level of nesting max, with each module having a clear responsibility:

User (CLI) → Intent Engine → Automation spec
                                    ↓
Watcher (poll triggers) → Executor → MCP Manager → MCP Servers → External APIs
                              ↓
                          Database (results, state)

The Intent Engine is the core innovation. It takes a natural language string and produces a structured automation spec through a three-stage pipeline:

  1. Parse — Extract trigger, conditions, and actions from the intent
  2. Clarify — Ask follow-up questions if the intent is ambiguous
  3. Plan — Generate a full automation specification with error handling

For LLM calls, PAI supports both Claude (cloud) and llama.cpp (local). Sensitive domains like health and finance can be routed to the local model automatically — your data never leaves your machine.

MCP as the Integration Layer

Instead of building direct API integrations, PAI connects to external services through MCP servers. This means:

  • Gmail is an MCP server that exposes search, label, and archive tools
  • Outlook connects through a separate MCP server with email and calendar tools
  • GitHub provides PR listing, review fetching, and formatting tools

The MCP manager handles server lifecycle, tool discovery, and routing. When an automation's action says "label email as Important", the executor finds the right MCP server and calls the appropriate tool.

This architecture is inherently extensible — adding a new service means writing an MCP server, not modifying PAI core. The mcp.json config file maps server names to commands:

{
  "servers": {
    "gmail": {
      "command": "uv",
      "args": ["run", "pai-gmail-mcp"]
    }
  }
}

Entity Discovery

One feature I'm particularly proud of is entity discovery. Run pai entities --discover and PAI scans your recent emails, extracting entities (clients, people, projects) using LLM analysis. These entities become building blocks for automations — "when Acme Corp emails me" works because PAI knows Acme Corp sends from @acme.com.

Local-First LLM Support

Not everything needs to go through Claude's API. PAI supports local LLMs via llama.cpp's OpenAI-compatible API. The routing configuration lets you:

  • Force local processing for sensitive domains
  • Use Claude for complex intent parsing
  • Fall back to local when offline
llm:
  routing:
    force_local: false
    sensitive_domains:
      - health
      - finance

What's Next

The current focus is on the watcher system — background processes that poll triggers (new emails, PR reviews) and fire automations when conditions match. The GitHub PR watcher already works, monitoring for review activity and formatting updates for Claude Code consumption.

The longer-term vision is a fully autonomous personal assistant that manages your digital life proactively, not just reactively. Entity discovery was the first step toward PAI understanding your world, not just responding to commands.