VibeVoice: Microsoft's Open Voice AI Family, From 90-Minute Podcasts to On-Device ASR
— by Vivax
VibeVoice is Microsoft's open-source family of frontier voice AI models, spanning text-to-speech and speech recognition.
Over eleven months, Microsoft quietly assembled one of the most complete open voice-AI stacks in existence. It started in August 2025 with VibeVoice-TTS, a frontier open-weights text-to-speech model — and by July 2026 the family spanned expressive long-form synthesis, a realtime variant, a 7B speech-recognition model, and a CPU inference engine for the edge. The through-line: everything is open, from weights to training recipes to the C++ runtime.
The flagship's numbers are still startling. VibeVoice-TTS synthesizes up to 90 minutes of multi-speaker audio in a single pass — up to four distinct voices holding a conversation — thanks to a continuous speech tokenizer running at an ultra-low 7.5 Hz frame rate paired with a next-token diffusion head. That design, accepted as an ICLR 2026 oral, is what makes podcast-length generation coherent rather than a stitched-together patchwork; the model plans the whole conversation like a document, not sentence by sentence.
The project also became a case study in open-source responsibility. Days after release, the community showed the model could be misused for voice cloning at scale, and Microsoft temporarily pulled the repository, re-releasing it with strengthened disclosures, audible-watermark guidance, and clearer intended-use terms. It was a rare public course correction — evidence that 'open' and 'governed' are not opposites, and a preview of the norms frontier audio models will be held to.
Then the family grew legs. VibeVoice-Realtime brought latency low enough for live agents; VibeVoice-ASR added a 7B recognition model that transcribes up to 60 minutes of audio in one pass with speaker diarization and word-level timestamps; and VibeASR.cpp — built on a BitNet-style ternary-quantized variant — runs speech recognition on ordinary CPUs, no GPU required. That last piece matters most: it moves serious speech AI from the datacenter to the device in your pocket.
Voice is the native interface of medicine — consultations, handovers, dictation, the worried call at 2 a.m. For Vivax, the VibeVoice arc confirms two things we build around. First, open voice stacks mean clinical products no longer rent their ears and voice from a closed API: an on-device, BitNet-class ASR engine is exactly what privacy-preserving bedside transcription needs, keeping raw patient audio on the machine it was spoken to. Second, the misuse-and-relaunch episode is a governance lesson we take seriously — in healthcare, watermarking, consent, and provenance aren't add-ons; they're the product. The technology to listen well is now open. The responsibility to listen safely is ours.