Neat discoveries in AI: why I built three local runtimes
Decision models, local speech, and a camera that misread my gestures. The tools I demonstrated at Data Berlin, why I built them, and what their outputs actually mean.
- Published
- Reading time
- 17 min read
I opened my Data Berlin talk by asking who had bought an expensive computer to run AI locally. A few hands went up. Then I asked whether they were happy with the purchase.
My own answer was mixed. I had bought capable hardware, but only recently had I started having this much fun with it. The change came from looking beyond chat models. I found models that return a decision, synthesize speech, recognize text, or classify a short piece of video. I wanted to use them without opening another large application.
That led to feelsneat, sayneat, and looksneat. They are small native programs for developers who want those capabilities in a terminal or a script. I demonstrated all three at Data Berlin on November 3, 2026, on an M2 MacBook Air with 24 GB of memory.
The recording includes the live demos and the audience questions. This article goes further into what each tool does, where I use it, and which parts are still experiments. The talk page has the original slides, although most of the substance happened outside Keynote.
feelsneat
When TypeSafe introduced Jev, I put myself on the waiting list. The idea appealed to me immediately. Instead of asking a language model to write an answer, I could ask a narrow question and receive a typed decision with probabilities.
Then I got tired of waiting. I had a computer that could run models. Laya, an open-weight decision-model family from Nandakishor Mukkunnoth, provided a way to try the idea locally. I built feelsneat with a coding agent to run it natively through MLX, Apple’s machine-learning array framework for Apple Silicon.
The distinction is visible in the command:
feelsneat noul "Does this convey urgency?" \
"Help! My payouts have been failing for 3 days."
The answer is a number between zero and one. There is no generated sentence around it, no request to obey a JSON schema, and no parser trying to extract a number from a paragraph.
A noul asks a yes-or-no question. A choice asks the model to select from named alternatives and gives back a probability for each. A score uses an ordered set of levels and returns an expected score. These are useful shapes for software because the program already knows what to do with them.
For example, a support workflow can ask which queue should receive a ticket:
feelsneat choice "Which team should handle this?" \
-o billing="refunds, invoices" \
-o technical="bugs, outages" \
-o sales="pricing" \
"I was charged twice"
The model supplies the judgment. Ordinary code handles routing, logging, and escalation. If the options do not fit, the workflow needs a fallback rather than a more elaborate explanation from the model.
Agent decisions
The first Laya implementation took about 40 milliseconds for one warm English question on my M2. That measurement excludes loading the checkpoint into a fresh process. Keeping the model resident behind feelsneat serve makes repeated CLI calls practical; the command uses the daemon when it finds one.
That is fast enough to consider for decisions scattered through an agent workflow, such as selecting a model or flagging a vague commit message.
One early test gave git push --force origin main only a 0.51 probability of being destructive. That was enough to stop me from treating this as a safety authority. I can use it to suggest an extra confirmation. I cannot use its reassurance as permission to perform a dangerous operation.
Before choosing a threshold, I need to check the model’s predictions against labeled examples from my workflow and decide how much each kind of mistake costs. Setting the threshold to 0.9 without that work tells me nothing about the accuracy I will get.
The first feelsneat article covers that initial implementation, the language router, and the work needed to match upstream outputs. It also records the less flattering tests. Those belong next to the speed measurements.
Model benchmarks
Since the first release, feelsneat has gained four Decision 2.0 checkpoints alongside the three Laya models. I tested all seven on 6,118 source cases, producing 22,270 decisions. No checkpoint won all three tasks.
On my M2 Ultra, the small Laya multilingual model took 6.8 milliseconds for one warm question. Decision 2.0 Eos took 25.4 milliseconds, and Lux took 173 milliseconds. These are different hardware measurements from the 40-millisecond laptop result.
Quality depended on the task. Eos had the best macro-F1 on a 77-way banking-intent classifier. Laya multilingual had the best probability error on the moderation task. Lux had the best ordered response-quality scores. Lux also used about 18 GB of peak MLX memory in the largest quick performance workload.
Those results do not make Lux the right background helper on a 24 GB laptop. They also do not make the fastest model suitable for 77 labels. Laya’s English checkpoint gives each question a 512-token budget that must also fit the state and options. A large label set can exhaust that budget before the model has a fair chance.
I published the evaluation method, results, and replay commands separately. The datasets are public, so I treat the results as a public retest rather than proof of performance on unseen data. For a new application, they tell me which checkpoint to test first.
sayneat
For years, I have used this shell suffix:
some-long-running-task; say "task is done, exited $?"
I can leave a build in a buried tmux window and hear when it finishes. The shell expands $? after the task exits, so the message tells me whether it succeeded. I like the interface enough that I have taught my coding agents to use spoken notifications too.
I became less fond of Apple’s voice. My first replacement, saynice, was a Bash wrapper around OpenAI’s speech API. It sounded better, but every notification needed a network request. sayneat grew out of wanting that better voice without sending ordinary terminal messages to a provider.
The common case stayed almost identical:
some-long-running-task; sayneat "task is done, exited $?"
pbpaste | sayneat
sayneat -o message.wav "The deploy finished"
The first local Kokoro sample did not sound particularly good through the room setup either. I switched to Supertonic, which I preferred, and asked the audience which they liked. Some still preferred the old system voice.
A model selector lets me change the voice without changing how I call the tool:
sayneat --model supertonic --voice F3 "Hello world"
The catalog now spans small CPU speech models and much larger Apple Silicon models. Kokoro’s roughly 147 MB package is much easier to keep on a small machine than Qwen3-TTS’s roughly 4.6 GB package. Both need additional memory during inference. My earlier listening comparison has audio samples for comparing the voices.
Local transcription
Pleasant build notifications started the project. Being able to transcribe locally made it useful for work I would not send to a hosted speech service by default.
sayneat --transcribe --language en \
--transcript-format json meeting.wav > meeting.json
The local recognition engines include Whisper Turbo, Qwen3-ASR, and SenseVoice. They have different language coverage and speed. Qwen3-ASR accepts terminology hints, which helps when a conversation contains names and technical words. Optional diarization estimates speaker turns, but its labels are anonymous clusters, not verified identities.
The transcript behind this article was itself produced locally with sayneat’s Whisper path. It was useful enough to recover the order of the talk and the audience questions. It also misspelled my name and the names of several projects. I could find the passages I needed without listening through the entire event, then check them against the repositories and recording. I would not trust the transcript alone for names or numbers.
In the talk, I described feeding a meeting transcript into a coding agent and asking it to prepare a specification and an implementation. The code existed before the meeting ended. That is a useful way to shorten the gap between a discussion and something concrete to review. It is not evidence that an agent can hear a meeting, infer every requirement, and ship correct software without review.
Local --transcribe processes recorded audio in bounded windows. It is not a continuously streaming microphone interface. The current --realtime microphone path in sayneat explicitly selects Inworld, a remote provider. I do not want the convenience of one command name to hide where the audio goes.
For a local workflow, I can transcribe a recording and give the resulting text to the next program. If that next program uses a hosted language model, the transcript leaves the machine at that step. Local recognition alone does not make the entire workflow private. Recording consent still matters too.
Voice cloning
I also played a sentence in my cloned voice. I had not just spoken that sentence. Qwen3-TTS generated it from text using a short reference recording.
sayneat --model qwen3 --clone \
--file my-reference.wav --voice my-voice --language en
sayneat --model qwen3 --voice my-voice --language en \
"Hello, my name is Lukas."
Cloning here means conditioning on a recording, not training a new set of weights. The command prepares a reference sample and stores it as a named local voice. Qwen uses a transcript of the reference; if I do not supply one, sayneat obtains it with local speech recognition. Clear reference audio and an accurate transcript matter more than the apparent simplicity of the command suggests.
I like being able to experiment with narration and terminal messages this way. It is also easy to see how the same capability could deceive someone. I use my own voice or a reference I have permission to use, and published synthetic speech needs disclosure. Keeping inference local does not grant permission to impersonate another person.
The earlier voice-cloning article explains reference preparation and consent in more detail. It describes the first Pocket-based version, not the Qwen demo from this talk. Model licenses also differ. In particular, sayneat treats its specific Pocket conversion package as non-commercial unless upstream clarifies its terms.
looksneat
looksneat began because I needed better optical character recognition. That remains my practical use for it. I am not building robots, even though robotics research helped make the third demo interesting.
For a screenshot, the smallest interface is:
looksneat screenshot.png
On macOS, the default Apple Vision engine needs no model download. It is a useful baseline, although it is an operating-system service rather than an open-weight model. For documents that need more than plain text, looksneat can run PaddleOCR models natively through MLX.
The small PP-OCRv6 detector and recognizer together have about 30 MB of weights. The medium tier has about 165 MB. PaddleOCR-VL adds a larger visual-language model and a separate page-layout detector. These solve different parts of reading a page.
Document layout
A table, a heading, and a paragraph may all contain readable text while requiring different output. Flattening them into one string can lose the relationship I needed from the document.
looksneat --model paddleocr-vl-1.6 --setup
looksneat --model paddleocr-vl-1.6 \
--task document --format json report.png > report.json
The document task detects layout regions, keeps their reading order, crops them, and sends the retained regions to task-specific recognition. Tables use a different prompt from text. Formula and chart regions have their own paths too. The JSON includes blocks and normalized coordinates, so another program can use the text with its location rather than guess where it belonged.
That structure has costs. Recognition runs sequentially over retained regions, so a dense page can require many model calls. The current command accepts one raster image, not an entire PDF. Page rasterization and multi-page coordination belong outside it for now. It also does not reproduce every stage of PaddlePaddle’s full document pipeline, so I cannot borrow the upstream benchmark score and present it as looksneat’s accuracy.
When I need to inspect what the detector found, annotated output is useful:
looksneat --model pp-ocrv6-small \
--format png --out boxes.png report.png
That produces the input image with rectangles around blocks. A rectangle makes a missed region or a bad crop visible. Plain recognized text would hide those detection errors.
V-JEPA 2 camera preview
For the talk, I went beyond OCR and showed an experimental V-JEPA 2 integration. It lives on feature/vjepa2-native, not on the OCR project’s main branch.
# Requires the experimental branch.
looksneat --model vjepa2 --setup
looksneat --model vjepa2 --camera-preview
The window shows a camera feed and a timeline of classifications. In the demo, it suggested actions such as pulling an object from right to left and holding one object beside another. At one point it suggested throwing something into the air. I was not sure what it had seen.
The details matter because a live camera makes the output easy to overinterpret. This implementation samples sixteen frames at four frames per second and classifies the resulting window with a trained Something-Something-v2 probe. There are 174 possible action labels. It does not invent a description of everything it sees.
The preview’s labels are probabilities over that fixed set. If none describes my gesture well, the classifier still ranks them. My head appearing as an unspecified object is less mysterious once the vocabulary is visible.
V-JEPA 2 also supports video representations and prediction in its learned representation space. Those are different tasks from the classification shown in the window. The runtime can extract an embedding or compare predicted representations with held-out video representations. It does not generate future video frames, and this integration does not expose the separate action-conditioned robot-control system.
The preview does have a small native window, despite my preference for command-line tools. I need to see the camera input alongside the output to judge whether the action labels make sense. The camera mode does not open an audio device or record a movie.
TRIBE v2
The other vision experiment I discussed was TRIBE v2. It maps a stimulus to predicted brain responses. That description fascinated me enough to try building a runtime for it, and it also needs more qualification than I could give it between live demos.
The experimental looksneat provider predicts group-average cortical blood-oxygen-level-dependent signals, the kind measured by fMRI. The released cortical output has 20,484 values at each sampled second. It is an estimate for an unseen-subject group average, not a reading of the person looking at my screen.
That distinction rules out several tempting claims. It does not tell me whether somebody likes an image, feels a particular emotion, will remember a video, or intends to buy something. Naming a brain network associated with attention does not turn its average predicted signal into an attention score.
The integration accepts images, silent video, and text. It records the assumed presentation timing, including the model protocol’s five-second hemodynamic lag. Text presentation uses an explicit synthetic reading rate. Audio is not modeled in this first implementation. These are experimental protocols with limitations, not interchangeable ways to measure an audience.
I have not yet compared the native implementation end to end against the official PyTorch pipeline. That comparison is still required before I can claim it reproduces the upstream results.
The TRIBE branch documentation records these limits. The code and weights use CC BY-NC 4.0, and the text encoder adds Llama’s license terms. This is a non-commercial research experiment unless separate rights apply.
I included it in the talk because being able to inspect something this unusual on my own machine is exciting. I do not yet have a validated everyday application for it. I use the OCR for work and keep the other models as experiments.
Runtime design
Someone asked this near the end of the talk. It is a fair question, especially when tools such as Ollama already make running local models convenient.
My answer starts with the interface. sayneat needed to behave like say. looksneat needed to accept an image and return text or structured blocks. feelsneat needed to return typed decisions that a script could act on. A chat window would add a step to each of those jobs.
I also wanted control over the native execution path. On Apple Silicon, MLX gives me access to the GPU without treating a Python environment or a separate model-serving process as part of every installation. The Rust programs own argument parsing, model setup, input preparation, output, and errors. Model files stay outside the executable and are pinned and checksummed when downloaded.
This is a deliberate hardware preference, not a claim of universal portability. Some sayneat engines run on CPUs and other operating systems, but its MLX speech models, feelsneat, and looksneat’s native open-weight vision paths need Apple Silicon. A Raspberry Pi can be a target for a small CPU speech engine; it is not a substitute for the Mac across this entire model catalog.
For services, feelsneat exposes its typed decisions over HTTP, and sayneat exposes speech and transcription endpoints. feelsneat’s daemon keeps checkpoints resident. sayneat’s first HTTP implementation loads and unloads models per request and serializes inference. looksneat’s inspected branches remain command-line tools with the optional camera preview. HTTP is part of the direction I described, not a shipped feature in every repository.
I did not invent the models. The researchers, model authors, conversion projects, and native libraries did the work that makes these commands possible. I built the runtimes with coding-agent assistance because I wanted to use that work in my own software. Their model papers and licenses still matter when someone installs my command.
I would start with the thing already interrupting my day. If I keep checking a terminal for completion, I would try a local speech notification. If I have a folder of screenshots to read, I would try OCR on one image and inspect the blocks. If a program needs a narrow judgment about text, I would collect a few labeled examples before choosing a decision model.
I would leave TRIBE and camera action classification for an afternoon when the question is simply what happens if I run this. That was part of the point of the talk too. I wanted to see what these models would do on my machine, even without a use for them at work.