☠ OPEN-SOURCE. VOICES CLONED LOCALLY. YOUR AUDIO NEVER LEAVES YOUR MACHINE. NO GPU REQUIRED. ☠ ☠ OPEN-SOURCE. VOICES CLONED LOCALLY. YOUR AUDIO NEVER LEAVES YOUR MACHINE. NO GPU REQUIRED. ☠ ☠ OPEN-SOURCE. VOICES CLONED LOCALLY. YOUR AUDIO NEVER LEAVES YOUR MACHINE. NO GPU REQUIRED. ☠ ☠ OPEN-SOURCE. VOICES CLONED LOCALLY. YOUR AUDIO NEVER LEAVES YOUR MACHINE. NO GPU REQUIRED. ☠

local inference · ▓ no cloud séance

PocketTTS-RAVEN

Faster-than-realtime text-to-voice with voice cloning.
Native C++ and browser WASM builds, running locally on CPU.

1Pick voice 2Write line 3Make audio

☠ VOICE_CRYPT.EXE_□×

Temperature 0.50 higher = more expressive, less predictable

Loading engine…

Your audio materializes here as it’s generated

README_4U.TXT — NOTEPAD_□×

UNDER THE HOOD - SEE IT ON GITHUB

HOW TO USE

Type your text.
Press SPEAK to listen.

To make a voice:

Upload an MP3 file, ideally around 6-12 seconds of audio with no music, no background noise, just voice, or use the microphone.
Press "Clone this voice".

WHY DOES THIS THING EXIST?

In the past few weeks I have been working on a realtime conversational AI system. The goal is to have a "creature" that can follow complex multi-step instructions, carry a complex internal state, interact with you and the world, and connect. A disembodied robot of sorts.

Such an undertaking requires low-latency speech, ideally response times below 400ms. If it waits too long, the illusion falls apart.

You can see (and hear) a very very early WIP test of trying to lower latency in a way that feels conversational (active listening - no press to speak, interruption, turn-repair etc) below...

Along the way there were a lot of optimizations, detours, and dead ends. Some of the most useful work landed in STT and TTS (it’s hard enough to make LLMs run fast enough so any ms we can chop off elsewhere counts). I decided to start open-sourcing pieces of the system before unveiling the companion itself. Today I am releasing one of these pieces :)

Making PocketTTS fast enough

One of those pieces is a PocketTTS fork. The original model comes from Kyutai and was released as a Python/PyTorch implementation. That version is already impressive, but Python is not where I wanted the real-time game loop to wait for audio. VolgaGerm’s PocketTTS.cpp port moved the model to ONNX Runtime and C++, and was already fast enough to be exciting, but I noticed there was a lot of leading silence muddying the final experience.

I wanted to see how far I could trim without changing voice quality. On my MacBook M4 Max, this optimized native version reaches roughly 32-33x realtime, with a first usable audio chunk around 30ms on this benchmark text, all on the CPU (3 ar and 2 decode threads).

(own local benchmarks, warm)

Stage	What changed	Result
Python Pocket TTS	Original Python/PyTorch model	8.7x realtime, 210ms first useful audio
PocketTTS.cpp	Clean ONNX Runtime + C++ port	22.4x realtime, 120ms first useful audio
Optimized native path (ours)	Graph rewrites, cache deltas, fused decoder work	33.3x realtime, 30ms first useful audio
Browser build (ours)	WASM SIMD, threads, streaming playback, voice cloning	up to ~14x realtime on M4 Max, ~4x on iPhone 16 Pro

Native numbers are warm medians on a MacBook M4 Max using the page text and the Alba voice (3 threads). On different systems performance might vary.

TTS generations often start with a little silence or a short artifact, so "first useful audio" means the first chunk that contains usable speech, not just any waveform data. First audio chunk emitted is actually pretty close to ours on the other versions but if it is silence it is not a useful signal, so I took that into consideration on the benchmarks above.

On using AI agents for optimizing systems

This project was quite heavily vibecoded. After all, it does not bring a new model to the table, make PocketTTS more accurate, or add some magical new capability. It just makes it faster.
Turns out AI agents are surprisingly efficient at that kind of work. You can set concrete goals, draft investigation plans and let them ruminate, running around in circles, until they eventually exhaust many of the possible paths.

One thing I noticed is that their intuition seems grounded in a lowest denominator mix of best practice and common sense. Some of the ideas that actually yielded the best results were looked down on by the agents at first, and I had to push them to try the thing while they were trying to explain to me why it was apparently a no-go. I think that since these systems are trained on human data, they inherit human assumptions about what is too tedious, or too much of a fool’s errand. Computers, man... processing large amounts of incomprehensible data and doing voodoo hex-code acrobatics was historically the point! Anyway..

Some of the work

These performance gains were achieved while preserving the model’s intended audio quality: no additional low-bit quantization or quality-degrading network pruning was used. Because the runtime and graph rewrites change floating-point execution order, the final waveform can still be slightly different.

More info:

delta-KV cache outputs instead of returning full cache tensors every step
FastKV graph rewrites to remove unnecessary cache gather/scatter work
AttentionTail custom op for the hot AR attention path
decoder cache delta updates
fused ConvTranspose streaming overlap logic
Accelerate-backed decoder Conv ops on Apple Silicon
cross-layer positional deduplication so RoPE/mask work is shared instead of recomputed per layer
merged main + flow model path for fewer runtime calls
KV snapshot cache for restoring warmed voice conditioning state quickly
trim-sustain / leading-burst rejection, not just trimming silence
WASM-specific runtime work: exception-free ORT boundary, fixed non-growable memory, and browser thread-pool tuning

If you are curious about the journey, I wrote up the optimization notes here: PocketTTS optimization notes.

The native version became fast enough that I wanted to see if the same system could be useful in the browser too. To my surprise, this web build can hit around 14x realtime on long text utterances on a M4 Max, and around 4x realtime on an iPhone 16 Pro. In short, the WASM version can be as fast as, or faster than, the clean native C++ version. The payload is about 67MB, it supports voice cloning, and it runs locally. Maybe it can be useful as a library, or a reference, for someone building apps or games, or just tinkering with something that requires TTS. Make something cool!

Thanks a lot for reading. I’ll be releasing the next parts of the project soon, and soon enough the virtual intelligent cursed-hound ai that started this all as well.

July 7, 2026 · find me on x

Github: pkalogiros/pocket-tts-raven

Special thanks to Kyutai and VolgaGerm for Pocket-TTS and pocket-tts.cpp.
And Mr. Poe for the nice poem and inspiration of course.