UNIT 12/12 — AI · SECURITY · LOCAL-FIRST

Jarvis

A fully local voice assistant with hands, and the safety kernel that keeps them safe

ROLE: Designer & engineer
SECTOR: AI · Security · Local-first
PLATFORM: macOS · local-first
YEAR: 2026 — present
STACK: Python · Ollama · mlx-whisper · Kokoro TTS · MCP · py2app · pytest
STATUS: PERSONAL · IN DAILY USE

[ FIG.01 ] BRIEF

Everything runs on the Mac: openWakeWord and silero-vad for the wake word, mlx-whisper for speech, Ollama-served models chosen by free memory, a 60-voice neural TTS library, persistent mood and memory, and 22 skills across Gmail, Slack, Chrome, calendars, files, apps and reminders. No cloud calls, no telemetry.

[ FIG.02 ] SAFETY KERNEL

The safety kernel is a code-enforced policy chokepoint every skill call passes through: deny by default, an allow-listed argv shell, secret-path and credential-egress blocking, and a no-deletion rule that a user's confirmation cannot override. It is verified by writing exploits against it — a protection that has never been attacked is an assumption.

[ FIG.03 ] RESEARCH

A local-model research pipeline (query rewriting, host ranking, passage extraction, verdict-driven re-reads) is exposed as an MCP server so Claude Code offloads routine lookups to it at zero token cost, with an evaluation harness that regression-tests every fix.

[ FIG.04 ] SCALE

Twenty-five thousand lines of Python, 1,637 tests, 308 commits, shipped as a signed py2app bundle, in daily use on my own machine.

$ jarvis --status
> WAKE WORD · VAD.......... [ OK ]
> MLX-WHISPER.............. [ OK ]
> LOCAL LLM · OLLAMA....... [ OK ]
> NEURAL TTS............... [ OK ]
> SAFETY KERNEL · POLICY... [ OK ]
> 22 SKILLS................ [ OK ]
> RESEARCH · MCP SERVER.... [ OK ]
# running — on my own mac, every day
$

UNIT 12 — LOCAL SYSTEM BUILD

[ FIG.SYS ] ARCHITECTURE

SYSTEM SCHEMATIC — AUTO-GENERATED FROM SPEC

[ FIG.OUT ] OUTCOME

TESTS, INCLUDING ADVERSARIAL ONES AGAINST THE KERNEL

SKILLS WITH HANDS ON REAL ACCOUNTS

CLOUD CALLS

RESEARCH SERVER USED BY CLAUDE CODE DAILY

VERDICT:RUNNING — ON MY OWN MAC, EVERY DAY[ OK ]

NEXT UNIT

nHabit

AI-powered home-rental platform, from empty repo to native iOS and Android