> Interface
> Overview
JARVIS is a local AI assistant for macOS. Talk to it or type to it, and it can chat, answer questions, and control the machine it's running on. Everything runs locally: the language model, the speech-to-text, and the text-to-speech, with no cloud dependency required. It streams responses over a WebSocket to a lightweight single-page UI that opens automatically in the browser.
> How It's Built
- LLM. Qwen3 32B, served locally through Ollama and swappable for
anything else Ollama supports
- Speech-to-text. Whisper, running entirely locally
- Text-to-speech. macOS's built-in say command by default,
with an optional upgrade to an ElevenLabs voice if an API key is configured
- Backend. FastAPI with a WebSocket connection for streaming
- Memory. A SQLite database that remembers conversation context between
sessions
- Wake word. An optional listener for "Hey Jarvis" to trigger voice
input hands-free
> The HUD
The front end is a single HTML file with no framework behind it. The centre is an animated particle sphere that reacts to state (idle, listening, thinking, speaking) surrounded by live readouts: CPU, memory and audio input levels, a voice activity waveform, uptime and battery, and an intelligence stack panel showing which model, speech-to-text, voice and memory backend are currently loaded. Everything on it is real telemetry rather than decoration, which is the only reason it is worth having.
> Plugins
Voice or typed commands are matched against a small plugin system that controls the Mac directly:
"What time is it?" -> reads the time "Open Safari" -> opens any app "Volume up / down / mute" -> controls system volume "Dark mode" / "Light mode" -> toggles macOS appearance "Take a screenshot" -> saves to Desktop "Lock the screen" -> locks the Mac "Empty the trash" -> empties Finder trash "Search for black holes" -> opens a Google search
New plugins are just a small Python class with a regex pattern and an execute()
method, registered with the plugin manager, so adding a new voice command takes a few lines
of code.
> Features
- Typed and voice chat in one interface
- Fully local LLM + speech-to-text, no cloud round-trip required
- Hands-free wake-word activation
- Persistent memory across sessions
- Extensible plugin architecture for system control
> Why I Built This
I wanted a Siri/Alexa-style assistant without handing every conversation to a cloud provider. I wanted something that keeps my data on my own machine, that I can extend with my own plugins, and that I actually understand end to end because I built every layer of it.