Setting Up WhisperX on macOS (Apple Silicon)

This guide walks you through setting up WhisperX on Apple Silicon macOS. After completing these steps, your machine will be ready to run the LymeScribe server.

Note on acceleration: WhisperX runs on CPU here. Its ASR backend (CTranslate2 / faster-whisper) has no Metal/MPS GPU backend on Apple Silicon, so transcription + diarization are CPU-bound and batch-only (roughly 2× the audio length). This is expected and is the validated configuration.

Prerequisites Checklist

Before starting, confirm you have:

• macOS 13 (Ventura) or later — macOS 14 (Sonoma) or 15 (Sequoia) recommended for best MPS performance
• Apple Silicon chip (M1, M2, M3, or M4)
• 16 GB RAM minimum (24 GB recommended for large-v2 model)
• ~10 GB free disk space for models and dependencies
• A HuggingFace account and token (complete this separately)

1. Install Homebrew

If you do not have Homebrew installed, open Terminal and run:

/bin/bash -c "$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/HEAD/install.sh)"

Follow the on-screen instructions. After installation, verify:

brew --version

Expected output:

Homebrew 4.x.x

Note: If brew is not found after installation, follow the instructions Homebrew prints at the end of the install to add it to your PATH.

2. Install System Dependencies

Install ffmpeg

brew install ffmpeg

Verify:

ffmpeg -version

Expected output (first line):

ffmpeg version 7.x Copyright (c) 2000-2024 the FFmpeg developers

Install Python

brew install python@3.11

Verify:

python3 --version

Expected output:

Python 3.11.x

Note: Python 3.11 is recommended for best compatibility with WhisperX and PyTorch. Python 3.12 also works but some dependencies may produce warnings.

3. Create Python Virtual Environment and Install WhisperX

Create and Activate the Virtual Environment

python3 -m venv ~/whisper-env
source ~/whisper-env/bin/activate

After activation, your terminal prompt will show (whisper-env) at the beginning.

Upgrade pip

pip install --upgrade pip

Install PyTorch

pip install torch==2.8.0 torchvision==0.23.0 torchaudio==2.8.0

Note: The default PyTorch wheel for arm64 macOS is correct here — no CUDA flags. WhisperX uses PyTorch on the CPU (the diarization/alignment stages); the ASR backend is CTranslate2, also CPU.

Install WhisperX

pip install whisperx==3.8.6

Install Speaker Diarization Support

pip install pyannote.audio==4.0.4

Note: These installations may take several minutes as they download PyTorch and model dependencies.

4. HuggingFace Setup

Complete the HuggingFace Account Setup guide to create your account, generate an access token, and accept the required model license terms.

5. Test Your Setup

With the virtual environment still activated, run the following tests.

Verify PyTorch is Installed

python3 -c "import torch; print('torch', torch.__version__)"

Expected output (a 2.8-series build on arm64):

torch 2.8.x

Note: WhisperX runs on CPU on Apple Silicon — its ASR backend (CTranslate2 / faster-whisper) has no Metal/MPS support. torch.backends.mps.is_available() may report True, but WhisperX does not use it; the device is cpu.

Verify WhisperX is Installed

python3 -c "import whisperx; print('WhisperX imported successfully')"

Expected output:

WhisperX imported successfully

Run a Test Transcription

Create a short audio file or use any .mp3/.wav file you have available:

export HF_TOKEN="your_huggingface_token_here"
whisperx "test-audio.mp3" --model large-v2 --device cpu --compute_type int8 --diarize --hf_token $HF_TOKEN --output_format all

If the transcription completes and produces .json, .srt, .txt, .vtt, and .tsv output files, your setup is working correctly.

Important — device is cpu, not mps. WhisperX's ASR (CTranslate2 / faster-whisper) has no Metal/MPS backend on Apple Silicon; --device mps fails at model load. The working device is cpu with --compute_type int8. Diarization is therefore batch-only and CPU-bound (roughly 2× the recording's length) — it is never used for live dictation.

Note: The first run will download the large-v2 model (~3 GB) and the diarization weights. This is a one-time download. The diarization model is pyannote/speaker-diarization-community-1 (CC-BY-4.0), which whisperx 3.8+ uses by default.

Troubleshooting

MPS not available (MPS available: False)

• Check your macOS version: MPS requires macOS 12.3+ but works best on macOS 13+ bash sw_vers
• Verify you are on Apple Silicon (not Intel): bash uname -m Should show arm64. If it shows x86_64, you may be running Terminal under Rosetta — open Terminal natively.
• Reinstall PyTorch: bash pip uninstall torch torchvision torchaudio pip install torch torchvision torchaudio

Memory errors with large models

Apple Silicon uses unified memory shared between CPU and GPU. If you encounter out-of-memory errors:

• Use a smaller model: replace --model large-v2 with --model medium or --model small
• Close memory-intensive applications before running transcription
• 16 GB RAM can handle medium model comfortably; large-v2 benefits from 24 GB+

MPS device errors during transcription

If WhisperX crashes or produces errors when using --device mps:

• Fall back to CPU (slower but reliable): use --device cpu instead of --device mps
• Check for PyTorch updates that may fix MPS bugs: pip install --upgrade torch torchvision torchaudio

Model download issues

• Check your internet connection
• If downloads stall behind a corporate proxy, configure pip: bash pip install --proxy http://proxy:port whisperx==3.8.6
• Models are cached in ~/.cache/huggingface/ — delete this directory and retry if downloads are corrupted

"No module named whisperx" errors

• Verify the virtual environment is activated (prompt shows (whisper-env))
• Reinstall WhisperX: pip install whisperx==3.8.6
• Check pip is using the venv Python: which pip should show ~/whisper-env/bin/pip