Zyphra's ZONOS3-Mini Clones Voices With 100M Active Parameters, Runs in a Browser
Zyphra released ZONOS3-Mini on November 9, an Apache 2.0 text-to-speech model with 800M total but about 100M active parameters that clones voices and runs in real time on laptops, phones and in the browser.
Zyphra released ZONOS3-Mini on November 9, 2026, an open-weight text-to-speech model that clones a voice from a short sample and runs in real time on laptops, phones and even inside a browser tab. It is a sparse Mixture of Experts model with 800 million total parameters but only about 100 million active for each token, and the weights ship under the permissive Apache 2.0 license.
That size is the story for developers. Zyphra's previous model, ZONOS2, needed 8 billion parameters, which limited where it could be deployed. ZONOS3-Mini aims to give app builders natural speech and voice cloning without renting a large GPU for every request, and without sending audio to someone else's cloud.
Key facts at a glance
- Released: November 9, 2026, by San Francisco-based Zyphra, with weights on Hugging Face and runtimes on GitHub.
- Size: 800M total parameters, about 100M active per token (64 experts, 2 chosen per token), 11 layers.
- License: Apache 2.0, which allows commercial use without a separate agreement.
- Speed (company figures): 80 ms median time to first audio on one NVIDIA H100 with 8 concurrent requests; under 150 ms on an M3 MacBook Air; under 570 ms in the browser on a Pixel 9.
- Languages: 43 listed on the model card, with 27 default voices included.
Why a small speech model matters now
Open speech models have been getting better but also bigger. Many of the strongest open text-to-speech systems now sit in the billions of parameters. In published comparisons, Fish Audio S2 Pro is listed at 5 billion parameters, MOSS-TTS v1.5 at 8 billion and Qwen3-TTS at 1.7 billion. Running models that size locally usually means a capable desktop GPU, and serving them at scale costs real money per minute of audio.
ZONOS3-Mini takes the opposite route. Because only around 100 million parameters do work for each token, the compute per second of speech is closer to a tiny model, while the full 800 million parameters hold more knowledge. Zyphra says it trained the model on 112 NVIDIA H100 GPUs in 2.5 days, a modest budget by current standards.
For developers, the practical result is choice. The same weights can run on a server for a voice assistant product, on a Mac for a narration tool, or fully on the user's device for a privacy-sensitive app. Text and voice samples do not have to leave the machine when you use the local runtimes.
What changed under the hood
A new audio codec does much of the heavy lifting. A codec turns sound into tokens a model can predict, and back again. ZONOS2 relied on the Descript Audio Codec at 86 frames per second, which Zyphra describes as very hard to model in real time. For ZONOS3-Mini the company trained its own codec, ZAC-1, which runs at just 15 frames per second with 10 codebooks of 4,096 entries each: one semantic codebook for meaning and nine acoustic ones for sound detail. Zyphra reports that, under similar data limits, a model trained on ZAC-1 showed a 10x lower word error rate than one trained on the older codec.
The backbone borrows recent language-model tricks. The 11 layers mix two kinds of sequence processing in a 3:1 ratio: nine Gated DeltaNet layers, a faster linear-attention style design, and two standard global attention layers. The model handles a context of 32,768 positions and can generate up to 10 minutes of audio in one go. Audio comes out at 44.1 kHz, and each frame is rendered as soon as it is predicted, which is what makes streaming responses feel immediate.
Voice identity got a new encoder. Zyphra swapped the speaker encoder it used before for a speaker-verification model called MECT-B2. You can clone a voice from a raw reference clip of up to 60 seconds, or pass a single 192-number speaker embedding. After pretraining, the company applied two rounds of preference tuning, first DPO and then GRPO, to make output more robust.
Fine control over how it speaks
Identity and delivery are adjustable separately. Zyphra exposes controls for age, gender presentation, accent and voice texture, such as a smoother or raspier sound. Delivery controls cover pace, emotion, projection and liveliness, and they work through activation steering, which nudges the model's internal signals rather than retraining it.
Steering can target single words. Learned alignment heads track which word is being spoken during streaming, so a developer can, for example, whisper only the last few words of a sentence. A separate edit model, built on an approach from the earlier VoiceCraft research, can change a sentence after it has been generated while keeping the same voice.
These controls are aimed at games, audiobook narration and voice assistants, where a single flat reading voice quickly sounds robotic. Zyphra notes that the settings are targets, not guarantees: an "older adult" setting aims for an older-sounding voice, and it is worth listening to the results before shipping them.
How the benchmarks stack up, and the caveats
Zyphra built a new test suite, ZTTS2-Eval, for this launch. It covers clean read speech in 35 languages and in-the-wild recordings in 20 languages. The company rolls the results into a Capability Index computed with item response theory over 420 items, each a combination of metric, test set and language.
| Metric (Zyphra's figures) | ZONOS3-Mini | Fish Audio S2 Pro | MOSS-TTS v1.5 | Qwen3-TTS 1.7B |
|---|---|---|---|---|
| Capability Index | 66.2 | 61.7 | 60.2 | 37.7 |
| Clean word error rate (lower is better) | 7.79 | 11.06 | 11.01 | 25.55 |
| Clean speaker similarity | 83.4 | 81.2 | 82.9 | 69.6 |
| Clean MSR-UTMOS quality score | 3.07 | 3.16 | 3.04 | 3.54 |
Treat these numbers as a starting point, not a verdict. Zyphra designed and ran the benchmark itself, and no independent group has reproduced the results yet. The headline 66.2 score uses the speaker-embedding mode; when cloning from a raw reference clip, ZONOS3-Mini scores 44.7 on the same index, a large drop. On the automated naturalness score, it trails both Fish Audio S2 Pro and Qwen3-TTS 1.7B. Licensing is a real differentiator, though: Fish Audio S2 Pro is listed under a research license that requires a separate deal for commercial use.
Zyphra's own position is that text-to-speech is best judged by ear, and its launch post pairs written prompts with samples from several open and commercial systems so listeners can compare.
How developers can run it
There are four official paths:
- In the browser: a WebGPU build runs entirely on the user's own GPU with no install and no server. The first visit downloads the model and the browser caches it.
- On NVIDIA servers: SGLang-Omni serves the model behind an OpenAI-compatible
/v1/audio/speechendpoint, with streaming and cloning from a reference clip that needs no transcript. Zyphra also mentions vLLM support for servers. - Native local runtimes: an MLX server for Apple Silicon, a CUDA server, and a C++ ggml server and command-line tool that work on CPU, CUDA, Metal or Vulkan, all loading GGUF files.
- Hosted: Zyphra's own cloud service, which runs on NVIDIA H200 GPUs.
At peak, Zyphra reports 457 seconds of audio generated per second on a single H100, which gives teams a rough sense of how many concurrent users one GPU might handle. Your numbers will depend on hardware, batch size and settings.
What it means for you
If you build apps: an Apache 2.0 voice model that runs on-device lowers both cost and privacy risk for features like read-aloud, accessibility narration and offline assistants. The OpenAI-compatible endpoint means an existing integration may need only a model name and URL change to try it.
If you care about privacy: local runtimes keep text and voice samples on your own hardware, which can simplify compliance for teams in the US, UK, Canada and Australia that handle customer recordings.
If you clone voices: get clear, written consent from anyone whose voice you use, and label synthetic audio. The launch post and model card focus on capability and do not describe built-in audio watermarking, so any safeguards against misuse are up to you.
What happens next
The open questions are independent testing and real-world reliability. Expect community comparisons on other benchmarks and listening tests in the coming weeks, along with quantized builds and integrations in popular local AI tools. Zyphra lists 43 supported languages, but text normalization, which handles things like numbers and dates, covers only nine of them, including English, Mandarin, Japanese and German. Developers in other languages should test that carefully.
Takeaways
- Try it locally first: the browser build is the quickest way to judge quality on your own text before writing any code.
- Test both cloning modes: embedding mode scored far higher than reference-clip mode in Zyphra's own tests, so compare them on your voices.
- Check numbers and dates: if you work outside the nine normalized languages, spell out tricky text before sending it to the model.
- Benchmark on your hardware: company latency figures are a guide; measure time to first audio on the devices your users actually have.
- Build consent in: record permission for every cloned voice and disclose synthetic speech to listeners.