MLX-Audio

MLX-Audio runs audio models on Apple Silicon, covering text-to-speech, speech-to-text, speech-to-speech and music generation.

Verified

Core facts

Read by a script from the official API, not rewritten by a model.

License
MIT
Languages
Python
Maintainer
Neywa-Labs
Verified

What it is

MLX-Audio runs audio models on Apple Silicon, covering text-to-speech, speech-to-text, speech-to-speech and music generation. MLX-Audio is built on MLX. MLX-Audio is under the MIT License.

Main features

  1. The project lists fast inference optimized for Apple Silicon (M series chips) in its features list.

  2. MLX-Audio offers multiple model architectures for TTS, STT, STS and music generation.

  3. MLX-Audio provides voice customization and cloning capabilities.

  4. MLX-Audio provides an interactive web interface with 3D audio visualization.

  5. MLX-Audio provides an OpenAI-compatible REST API.

  6. MLX-Audio provides a Swift package for iOS/macOS integration.

Getting started

  1. The requirements list Python 3.10 or newer.

  2. The requirements list an Apple Silicon Mac.

  3. The requirements list the MLX framework.

  4. ffmpeg is required for saving audio in MP3, FLAC, OGG, Opus or Vorbis format.

  5. WAV format works without ffmpeg.

  6. By default, when generation yields multiple segments, mlx-audio saves numbered files.

From the README · bash
pip install -U mlx-audio
From the README · bash
uv tool install --force mlx-audio --prerelease=allow
From the README · bash
uv tool install --force git+https://github.com/Blaizzy/mlx-audio.git --prerelease=allow
From the README · bash
git clone https://github.com/Blaizzy/mlx-audio.git
cd mlx-audio
pip install -e ".[dev, server]"
From the README · bash
# Basic TTS generation
mlx_audio.tts.generate --model mlx-community/Qwen3-TTS-12Hz-0.6B-CustomVoice-8bit --text 'Hello, world!' --voice Vivian

# With a different voice and language hint
mlx_audio.tts.generate --model mlx-community/Qwen3-TTS-12Hz-0.6B-CustomVoice-8bit --text 'Welcome to MLX-Audio!' --voice Ryan --lang_code English

# Play audio immediately
mlx_audio.tts.generate --model mlx-community/Qwen3-TTS-12Hz-0.6B-CustomVoice-8bit --text 'Hello!' --voice Vivian --play

# Save to a specific directory
mlx_audio.tts.generate --model mlx-community/Qwen3-TTS-12Hz-0.6B-CustomVoice-8bit --text 'Hello!' --voice Vivian --output_path ./my_audio

# Stream audio during generation
mlx_audio.tts.generate --model mlx-community/Qwen3-TTS-12Hz-0.6B-CustomVoice-8bit --text 'Hello!' --voice Vivian --stream

# Stream audio during generation and save it to disk
mlx_audio.tts.generate --model mlx-community/Qwen3-TTS-12Hz-0.6B-CustomVoice-8bit --text 'Hello!' --voice Vivian --stream --save

# Join multiple generated segments into one file
mlx_audio.tts.generate --model mlx-community/Qwen3-TTS-12Hz-0.6B-CustomVoice-8bit --text $'Hello!\nHow are you?' --voice Vivian --join_audio
From the README · python
from mlx_audio.tts.utils import load_model

# Load model
model = load_model("mlx-community/Qwen3-TTS-12Hz-0.6B-CustomVoice-8bit")

# Generate speech
for result in model.generate(
    "Hello from MLX-Audio!",
    voice="Vivian",
    lang_code="English",
):
    print(f"Generated {result.audio.shape[0]} samples")
    # result.audio contains the waveform as mx.array

Who it is for

MLX-Audio provides multilingual support across models. MLX-Audio provides adjustable speech speed control. The project lists quantization support, including 3-bit, 4-bit, 6-bit and 8-bit, for optimized performance in its features list.

MLX-Audio includes a web interface and an OpenAI-compatible API. The project says quantization with its convert script reduces model size and improves performance. mlx-audio-swift provides on-device TTS using MLX on macOS and iOS.

The project describes Kokoro as a fast, multilingual TTS model with 54 voice presets. OmniVoice is a zero-shot multilingual TTS model for at least 646 languages, with voice cloning, batch generation, pronunciation controls and nonverbal tags. OmniVoice uses a bidirectional Qwen3 backbone with iterative masked generation and a HiggsAudioV2 acoustic tokenizer.

VibeVoice-ASR is Microsoft's 9B parameter speech-to-text model with speaker diarization and timestamps. VibeVoice-ASR supports long-form audio up to 60 minutes and outputs structured JSON. Parakeet v3 supports 25 European languages.

KugelAudio requires about 17GB memory (7B params in bfloat16). For KugelAudio, pre-encoded voice presets (voice cloning) are not yet available in the upstream model, and the model generates speech with a default voice. Voxtral TTS is Mistral's 4B multilingual text-to-speech model with 20 voice presets across 9 languages.

MedASR is a specialized model for medical terms and dictation. SAM-Audio separates specific sounds from audio using text prompts. MossFormer2 removes noise from speech recordings.

For Phonon-2 and Phonon-2.5, the published checkpoints load directly and no Fermion runtime is needed.

Sources

This page is written from the official sources below. Open them to check the original text.

Verified ·

This page is written by AI from the official sources above and checked automatically. This site has not tested it. Found a mistake? Tell us.