I wanted the Star Trek computer in my house, so I built one.
Meet Ada, my always-available household AI, and BM-01, the physical “brain module” I just finished building for her.
This started a few months ago with a pretty simple idea:
Could I build something that feels less like Alexa and more like the computer from Star Trek?
Not just “turn on the lights.”
I wanted to walk into the room, say “Hey Ada,” have a normal conversation, interrupt her naturally, have her remember things, use tools, control the house, recognize the people who live here, survive reboots without babysitting, and generally feel like a computer that is part of the house rather than an app I'm opening.
It got a little out of hand.
https://ibb.co/album/SRMpN7
What Ada is now
Ada runs continuously on a dedicated Lenovo ThinkCentre M720q Tiny running Windows.
The current system includes:
- GPT-Live for realtime voice conversation
- ReSpeaker XVF3800 four-microphone array
- dedicated USB speaker
- local “Hey Ada” wake-word detection
- separate local “Red Alert” detection
- persistent household memory and separate lore memory
- tool/function calling
- direct local WiZ lighting control
- Samsung TV control
- weather, time, search, diagnostics, etc.
- speaker recognition with a conservative “Unknown” fallback
- Red Alert / All Clear household emergency states
- a Flight Data Recorder for diagnostic events
- remote administration over a private connection
- Android controls
- automatic startup/recovery after reboot
- and now BM-01, a physical ESP32-based telemetry/diagnostic computer panel
Ada isn't a commercial product and this isn't a company.
It's basically one guy slowly discovering how many engineering problems are hiding inside the sentence:
“I want the Star Trek computer in my living room.”
The AI turned out not to be the hardest part
One of the biggest things I've learned is that getting an AI model to talk is relatively easy.
Making the whole system pleasant enough to actually live with is much harder.
Audio alone became an absurd rabbit hole.
Ada has to hear someone across a room, ignore normal household noise, deal with a television playing nearby, hear a person while she's speaking, know when she's being interrupted, and not wake up every time somebody on TV says something vaguely resembling “Hey Ada.”
At one point I discovered that the ReSpeaker XVF3800 exposes separate processed USB audio channels and Ada wasn't explicitly using its dedicated ASR channel.
Switching to it produced an immediate improvement.
I also tried energy thresholds, beam direction, moving the microphone, different Windows audio APIs, different gain structures, hardware AEC, software AEC, VAD tuning, interruption thresholds...
Turns out Patrick Stewart is remarkably good at convincing a home AI that he's part of the conversation.
Barge-in taught me a painful lesson
An early version used OpenAI's Realtime system and technically supported interruption.
Except interrupting Ada didn't work.
I spent a bunch of time looking at VAD, microphone echo, model settings, etc.
The actual problem?
My own speaker playback was blocking the asynchronous event loop.
The AI was perfectly capable of being interrupted.
My speaker code wasn't.
I replaced the blocking playback with a callback-driven audio queue.
Suddenly barge-in worked.
That was one of the first times this project taught me that the “boring” infrastructure around the model matters just as much as the model.
Then a simple experiment blew up the entire architecture
After spending a ridiculous amount of time improving the Realtime 2.1 audio stack, I built a clean GPT-Live/WebRTC sandbox in Chromium.
It was supposed to be an experiment.
Instead, my reaction was basically:
“Dude. That was night and day.”
Natural conversation was better.
Interruptions were better.
Simultaneous speech was better.
It felt dramatically closer to the experience I had actually wanted.
So obviously I decided to migrate the whole system.
And obviously that went terribly.
By then the mature version of Ada already had memory, wake detection, smart-home tools, speaker recognition, emergency behavior, remote administration, logging, startup/recovery logic, and a pile of other systems.
Several experimental LIVE builds could do pieces of that, but getting all of Ada back around the new conversational core was much harder than expected.
At one point the new system had even left most of Ada's historical memories behind.
Eventually I stopped trying to transplant the new conversational engine into the old architecture.
I reversed the strategy:
Keep the clean Live/WebRTC conversational core working, and rebuild Ada around it.
That became the modern LIVE branch.
Memory got weird too
Ada's original memory system basically dumped everything she knew into the prompt.
That obviously wasn't going to scale.
Then I tried keyword retrieval.
That worked until I asked the same question using different words.
Then I moved to semantic embeddings.
That worked much better until I asked:
“Why is the sky blue?”
and Ada's memory system decided that my saved memory:
“My favorite color is blue.”
might be relevant.
So memory eventually grew its own categories, semantic thresholds, query classification, cached embeddings, ADD/UPDATE/IGNORE behavior, stale-memory replacement, and limits on how much personal information can be retrieved into one conversation.
There's also a separate lore database because apparently my home computer needs to understand the difference between facts about my household and facts about fictional universes.
I regret nothing.
Wake words are currently my nemesis
“Hey Ada” sounds like it should be easy.
It is not.
The challenge isn't just recognizing it.
It's recognizing different people saying it naturally, from different parts of the room, while not waking because of television dialogue or random household noise.
One experimental detector combined an ONNX wake model, Silero speech detection, and cloud transcription for ambiguous candidates.
It was extremely sophisticated.
During one household test it generated 68 transcription requests in roughly six hours.
And Ada was still annoyingly difficult to wake.
So I removed it.
Later I found cases where the wake model itself was extremely confident but another part of the pipeline vetoed it.
One Red Alert candidate scored 0.954 and was still rejected because the downstream speech gate didn't have the evidence it wanted.
That eventually led to local high-confidence fallback logic.
I'm currently testing a completely different approach: a Microsoft Advanced Custom Keyword model trained specifically on “Hey Ada.”
But I'm not letting it control Ada yet.
It's running in shadow mode.
Ada's existing wake system still has complete authority. The Microsoft recognizer receives a copy of the same microphone stream and simply logs when it thinks I said Hey Ada.
The first real test logged four Microsoft detections and three actual ONNX wake events.
More interestingly, I had actually repeated “Hey Ada” several times because the existing system wasn't waking.
So Microsoft appears to have caught some legitimate attempts earlier.
Promising?
Yes.
Enough evidence to replace the current system?
Nope.
It needs much more real-house testing first.
Red Alert exists because of course it does
Ada also has a separate emergency mode.
Say:
“Red Alert.”
and she can enter a predefined emergency state, including changing the lighting and producing the appropriate alert behavior.
Say:
“All Clear.”
and the system restores the previous lighting state rather than just dumping everything back to some default.
I deliberately keep this path separate from ordinary conversational interpretation.
I don't particularly want an LLM pondering the philosophical meaning of “Red Alert” while I'm trying to activate Red Alert.
There's also a Flight Data Recorder that can preserve diagnostic information around important events.
Yes.
My living room AI has a flight recorder.
This is apparently where my life has gone.
BM-01
BM-01 is the newest piece and the reason I finally decided this project was worth posting.
Originally I just wanted Ada to have a physical computer panel inspired by the blinking displays from Star Trek: The Original Series.
Then I realized it could actually solve a real problem.
Ada has a lot of stages between:
“the PC has power”
and
“the computer is actually ready to hear me.”
Windows might be running while Ada isn't.
Ada might be running while the wake detector isn't ready.
The wake detector might work while GPT-Live is still connecting.
The microphone might fail while everything else appears fine.
Normally I have to open logs to figure out which layer is unhappy.
So BM-01 became a real physical diagnostic annunciator.
The upper panel contains 20 resin jewels arranged in four rows of five.
Each row contains four monitored systems plus one red master-fault annunciator.
AI / Conversation
GPT-Live
OpenAI/API
Conversation engine
Tools
AI FAULT
Audio
Wake detector
XVF3800 microphone
USB speaker
Barge/interruption
AUDIO FAULT
Ada Core
Ada core service
Memory
Flight Data Recorder
Red Alert
CORE FAULT
BM-01 / External
ESP32 controller
LD2410C mmWave radar
BH1750 ambient-light sensor
External services
BM FAULT
The regular lighting isn't random.
I wanted it to resemble the electromechanical blinking of the original TOS computer panels rather than modern RGB rainbow effects.
So the ESP32 runs deterministic timing groups with different periods and phases, roughly inspired by mechanical cam/SPDT switching.
It looks complicated and irregular, but the pattern is deterministic.
Actual telemetry overrides it.
If the microphone fails, for example, the MIC jewel abandons its normal pattern and rapidly flashes.
At the exact same time, the red AUDIO FAULT jewel flashes in perfect synchronization.
If multiple systems in that bank fail, all of their jewels synchronize with the same red master fault.
I tested the complete path by injecting a simulated XVF3800 failure.
MIC + AUDIO FAULT immediately began flashing together.
When the fault cleared, both returned to the normal TOS animation.
That was probably one of the most satisfying tests in the entire project.
BM-01 also has an LD2410C mmWave radar and a BH1750 ambient-light sensor.
The lux sensor automatically adjusts the panel brightness.
The radar provides local presence information that I can eventually use for more context-aware behavior.
And importantly, the telemetry system has an UNKNOWN state.
If Ada doesn't have enough evidence to prove a subsystem is healthy, BM-01 doesn't just pretend everything is fine.
Right now some of the 16 monitored systems still need deeper health instrumentation.
That's unfinished work.
I'd rather have a light say “I don't know” than build a diagnostic panel that confidently lies to me.
Things that failed along the way
A non-exhaustive list:
- keyword memory retrieval
- blindly dumping all memories into every prompt
- Windows robot TTS
- blocking audio playback
- several barge-in approaches
- assuming USB audio device numbers stay put
- accidentally using the wrong processed microphone channel
- energy-based TV rejection
- direction-of-arrival TV rejection
- physically moving the microphone
- several wake thresholds
- a hilariously overcomplicated wake verifier
- parts of the Realtime 2.1 audio architecture
- several LIVE migration attempts
- incomplete historical-memory migration
- Windows trying to autostart an obsolete copy of Ada
- an AI tool wrapper that turned out not to be callable like an ordinary Python function
- a malformed Python docstring that somehow left my smart bulbs permanently red
- a smart-light group/device naming collision that made one porch bulb look broken
- an early BM-01 layout that was simply the wrong size
- Windows batch syntax being defeated by a parenthesis in a status message
I've learned more from that list than from most of the things that worked immediately.
The current architecture, very roughly
Voice:
XVF3800
↓
local wake / emergency detection
↓
Chromium/WebRTC audio path
↓
GPT-Live
↓
Ada tools / memory / household services
↓
USB speaker
Household services:
Ada
├── persistent memory
├── separate lore
├── WiZ lighting
├── Samsung TV
├── weather/search/time
├── diagnostics/FDR
├── Red Alert / All Clear
├── remote administration
└── BM-01 telemetry
BM-01:
Ada/Windows health data
↓
Windows telemetry/serial bridge
↓
ESP32
├── 20-jewel diagnostic panel
├── LD2410C radar
└── BH1750 lux sensor
The ESP32 handles the normal TOS animation locally. Ada does not waste network/model traffic telling 20 lights when to blink.
She only needs to report actual state.
What's next?
Ada isn't finished, and I don't think she ever really will be.
Immediate work is mostly:
- finishing the Microsoft-vs-ONNX wake-word comparison
- improving speaker recognition
- expanding trustworthy BM-01 telemetry
- continuing to improve interruptions/audio behavior
- cleaning up some remaining diagnostics
Longer term, I want to build remote modules around the house.
BM-01 becomes the central computer.
The remote modules become terminals.
Which means there is a very real possibility that this ends with multiple Star Trek computer panels installed around my house.
I'm okay with that.
Biggest lesson so far
The AI model is not the hard part.
Making an AI live in a house is the hard part.
Audio ownership matters.
Turn ownership matters.
Memory relevance matters.
Startup and recovery matter.
Knowing whether a tool actually executed matters.
Knowing whether your microphone silently changed device numbers matters.
Knowing whether your diagnostic system actually knows something is healthy matters.
And testing something for five minutes at a desk is absolutely not the same thing as letting your family live with it for a week.
Probably the biggest design lesson Ada has taught me is:
A complicated solution isn't automatically a better solution.
I spent months engineering the old audio path.
A clean WebRTC experiment kicked its ass.
I built an elaborate wake verifier.
It made Ada harder to wake.
I designed more elaborate BM-01 layouts.
The actual physical hardware told me what the layout needed to be.
So my development process at this point is basically:
Build it. Measure it. Put it in the house. Let reality break it. Keep what survives.
That's Ada.
And apparently I live on a starship now.
