中文

whisper.cpp

whisper.cpp provides inference of OpenAI's Whisper automatic speech recognition (ASR) model.

Verified

Core facts

Read by a script from the official API, not rewritten by a model.

License
MIT
Languages
C++
Maintainer
ggml-org
Verified

What it is

whisper.cpp provides inference of OpenAI's Whisper automatic speech recognition (ASR) model. It is a plain C/C++ implementation without dependencies. The entire high-level implementation of the model is contained in whisper.h and whisper.cpp. The rest of the code is part of the ggml machine learning library.

Main features

  1. Apple Silicon is listed as a first-class citizen, optimized via ARM NEON, Accelerate framework, Metal and Core ML.

  2. whisper.cpp supports AVX intrinsics for x86 architectures.

  3. whisper.cpp uses mixed F16 / F32 precision.

  4. The feature list includes zero memory allocations at runtime.

  5. whisper.cpp supports CPU-only inference.

  6. The feature list includes Vulkan support.

Getting started

  1. First, clone the repository.

  2. Then download one of the Whisper models converted in ggml format.

  3. Next, build the whisper-cli example and transcribe an audio file.

  4. The whisper-cli example currently runs only with 16-bit WAV files.

  5. Pre-built binaries of whisper.cpp can also be installed, or it can be built from source, using Conan.

  6. The original models are converted to a custom binary format.

From the README · bash
git clone https://github.com/ggml-org/whisper.cpp.git
From the README
cd whisper.cpp
From the README · bash
sh ./models/download-ggml-model.sh base.en
From the README · bash
# build the project
cmake -B build
cmake --build build -j --config Release

# transcribe an audio file
./build/bin/whisper-cli -f samples/jfk.wav
From the README · bash
ffmpeg -i input.mp3 -ar 16000 -ac 1 -c:a pcm_s16le output.wav
From the README
conan install --requires="whisper-cpp/[*]" --build=missing

Who it is for

The feature list includes a C-style API. The feature list includes Voice Activity Detection (VAD). whisper.cpp supports integer quantization of the Whisper ggml models.

On Apple Silicon devices, the Encoder inference can be executed on the Apple Neural Engine (ANE) via Core ML. Supported platforms include Mac OS (Intel and Arm). Supported platforms include iOS.

Supported platforms include Android. Supported platforms include WebAssembly. With NVIDIA cards, the processing of the models is done on the GPU via cuBLAS and custom CUDA kernels.

With AMD GPUs, the processing can be accelerated via HIP/ROCm. Encoder processing can be accelerated on the CPU via OpenBLAS. Ascend NPU provides inference acceleration via CANN and AI cores.

With Moore Threads cards, the processing of the models is done on the GPU via muBLAS and custom MUSA kernels. By default, the examples in this repo use the miniaudio library to decode audio files. The limitation the project lists is inference only.

Sources

This page is written from the official sources below. Open them to check the original text.

Verified ·

This page is written by AI from the official sources above and checked automatically. This site has not tested it. Found a mistake? Tell us.