Sample rate is one line of a speech data spec you can't fix after recording. Audio recorded at 8 kHz holds nothing above 4 kHz, and no later processing puts it back. The right rate follows from what the data is for, and so does the rest of the audio spec: bit depth, channels and file format.
The short answer: 16 kHz for most speech recognition, 8 kHz when you are building for narrowband phone calls, and 22.05 kHz or more, recorded at 44.1 or 48 kHz, for text-to-speech. Record at the rate you deploy at or higher, never lower. This guide explains why, and ends with a summary table and the lines to put in a request for proposal.
Sample rate sets the bandwidth
The sample rate is the number of samples taken per second. The highest frequency a recording can hold is half its sample rate, known as the Nyquist limit. So 8 kHz audio stops at 4 kHz, 16 kHz audio at 8 kHz, and 48 kHz audio at 24 kHz. Bandwidth, not the number in the file header, decides what a model can learn from the audio.
The Opus codec standard, RFC 6716, names the bands you will meet in telephony and codec documents: narrowband (4 kHz of audio bandwidth at an 8 kHz sample rate), wideband (8 kHz at 16 kHz), super-wideband (12 kHz at 24 kHz) and fullband (20 kHz at 48 kHz). Opus never codes above 20 kHz, which the standard calls the generally accepted upper limit of human hearing.
| Sample rate | Highest frequency | Where you meet it |
|---|---|---|
| 8 kHz | 4 kHz | Narrowband phone calls, older conversational corpora |
| 16 kHz | 8 kHz | Input to speech recognition models, wideband calls |
| 22.05 kHz | 11.025 kHz | TTS training sets such as LJ Speech |
| 24 kHz | 12 kHz | TTS models and neural audio codecs |
| 44.1 kHz | 22.05 kHz | CD audio, studio recording |
| 48 kHz | 24 kHz | Studio and TTS voice recording |
8 kHz: narrowband telephone audio
Classic telephone audio is narrowband. ITU-T G.711, "Pulse code modulation (PCM) of voice frequencies", encodes eight bits per sample after logarithmic scaling, either mu-law or A-law. The RTP audio profile, RFC 3551, carries the two as PCMU and PCMA and clocks both at 8,000 Hz. That makes 64 kbit/s, and nothing above 4 kHz.
Not every call is narrowband any more. G.722, "7 kHz audio-coding within 64 kbit/s", samples at 16,000 Hz, and Opus, built for interactive audio such as VoIP, runs from narrowband to fullband. So "telephone data" in a spec is ambiguous: name the band, and the codec if you know it. Our call center speech data is recorded to order as narrowband telephony or wideband audio, whichever matches the calls your model will handle.
The best-known English conversational corpora, Switchboard and Fisher, are 8 kHz telephone recordings. That suits a phone model, and limits anything that needs wider-band speech.
16 kHz: the standard sample rate for ASR
16 kHz is the usual input rate for speech recognition models. OpenAI's Whisper sets SAMPLE_RATE = 16000 in its audio loader, which decodes every file through ffmpeg to mono 16-bit PCM at that rate, "resampling as necessary". The wav2vec 2.0 base model card tells users to make sure their speech input is sampled at 16 kHz, the rate of the 960 hours of LibriSpeech the model was trained on.
Hugging Face's audio course calls 16 kHz a common rate for training speech models. For speech, it says, a higher rate "will not capture more information" and only raises processing cost, while speech sampled at 8 kHz sounds muffled. That holds for recognition. Generation is different, as the next section shows.
Two consequences follow. A 48 kHz file sent to Whisper is downsampled to 16 kHz before the model hears it, so a higher delivery rate buys headroom for other uses, not recognition accuracy. And a phone call resampled to 16 kHz for the same model is still narrowband: the band from 4 to 8 kHz is empty.
22.05 and 24 kHz: TTS models and audio codecs
Text-to-speech models generate audio that people listen to, so missing high frequencies become audible. TTS training sets usually sit above the ASR rate.
LJ Speech, a single-speaker set, holds 13,100 clips, about 24 hours, of single-channel 16-bit PCM WAV at 22,050 Hz. LibriTTS rebuilt LibriSpeech's original material into 585 hours at 24 kHz from 2,456 speakers, to fix issues that made LibriSpeech less than ideal for TTS. Recent models follow suit: the Kokoro-82M model card's examples save its output at 24,000 Hz.
Neural audio codecs, which turn audio into tokens for speech language models, work in the same range. Kyutai's Mimi, the codec behind the Moshi dialogue model, processes 24 kHz audio down to a 12.5 Hz representation at 1.1 kbps. Meta's EnCodec ships a 24 kHz mono model trained on varied audio and a 48 kHz stereo model trained on music.
44.1 and 48 kHz: studio recording
44.1 kHz is the CD rate, and 48 kHz is the fullband rate in Opus. For a TTS voice, record at one of them or higher, then deliver at the rate the model needs.
The VCTK corpus shows the pattern. Its speech was recorded at 96 kHz and 24 bits in a hemi-anechoic chamber, then converted to 16 bits and downsampled to 48 kHz for release. Recording above the working rate costs storage, and downsampling later loses nothing in the band you keep. Our overview of text-to-speech datasets covers the other TTS requirements, from hours per voice to voice rights.
Upsampling can't restore missing bandwidth
Resampling an 8 kHz file to 16 or 24 kHz multiplies the samples and changes the header, but the frequencies the phone line removed stay missing. The file reports 16 kHz and still carries narrowband audio.
Super-resolution models can estimate the missing band. Kyutai's Moshi works with a 24 kHz codec, so when its authors fine-tuned it on Fisher's 8 kHz conversations, they first upsampled Fisher with AudioSR, a model that predicts the removed frequencies rather than recovering them. Our guide to full-duplex conversational speech data covers that pipeline. Estimated content is not recorded content, so treat it as a workaround, not a source.
Check a sample before you trust a label. Open a few files in a spectrogram viewer: if the energy stops sharply well below half the file's sample rate, such as at 4 kHz in a 16 kHz file or 8 kHz in a 48 kHz file, the audio was recorded or coded at a lower rate than the file claims. Lossy history works the same way. An MP3 decoded to WAV is still the MP3.
Mixed sample rates and matching production
A model trains at one input rate, so everything gets resampled to it. The Hugging Face course warns that models struggle to generalize between sampling rates, and that fine-tuning data should match the rate the model was pretrained on. Mixed bandwidths can still train together: in 2012, Microsoft researchers trained a neural acoustic model on wideband and narrowband speech by treating the missing high band of narrowband audio as missing features, with no bandwidth extension needed.
What decides your mix is production:
- Record at the rate you deploy at, or higher. Downsampling is clean. Upsampling only relabels.
- Match the channel, not just the rate. A phone agent should train and be tested on telephone audio, either native narrowband recordings or wideband recordings passed through the same codec.
- Log the native rate and capture chain for every file, so you can balance the mix and report accuracy per band.
- Keep the test set at production conditions. Score models on the audio they will hear.
Bit depth: 16-bit is enough for most speech
Bit depth sets how finely each sample's amplitude is measured. 16-bit audio has 65,536 levels and 24-bit audio has 16,777,216. The Hugging Face course notes that the quantization noise of 16-bit audio is already small enough to be inaudible in practice, and Whisper's loader converts every input to 16-bit PCM anyway.
Record at 24-bit if you can, for headroom against level mistakes during sessions, and deliver 16-bit, as VCTK did. And remember the phone line: a call that traveled as G.711 carries eight bits per sample after logarithmic scaling, so a 16-bit WAV of it holds no more detail than the line carried.
Lossless or lossy formats
Store training data losslessly: PCM WAV, or FLAC, the Free Lossless Audio Codec, which its maintainers say compresses audio "without any loss in quality". Lossy codecs save space by throwing information away. MP3 uses perceptual coding to discard or reduce the precision of components that are less audible, as the Library of Congress format description puts it. Opus, which spans 6 to 510 kbit/s, is lossy as well.
If production audio is compressed, apply that compression to copies of lossless masters, for augmentation or a test set. Don't make it the only copy you have.
Mono, stereo or dual-channel audio
For a single speaker, such as a TTS voice or voice commands, mono is right, unless the product listens through a microphone array. For conversations, the question is not mono or stereo but whether each speaker has a channel of their own. A stereo recording from two microphones in one room puts both voices on both channels. Dual-channel audio in the conversational sense puts each speaker on a separate channel, typically because each was recorded on their own line or headset.
One channel per speaker keeps overlaps intact, makes each side easier to transcribe and gives you who-spoke-when labels from the channels themselves, as our speaker diarization guide explains. You can always mix two channels down to mono for a model that expects one. Going the other way takes source separation, and it is never as clean.
Best sample rate and audio spec by use case
| Use case | Sample rate | Bit depth | Channels | Format |
|---|---|---|---|---|
| Contact center ASR, phone voice agents | Native rate of your calls: 8 kHz narrowband or 16 kHz wideband | 16-bit | One per speaker | WAV or FLAC, codec named |
| ASR for apps, web and voice agents | 16 kHz or higher | 16-bit | One per speaker for conversations, mono for single speakers | WAV or FLAC |
| ASR test set | Same as production | 16-bit | Same as production | WAV or FLAC masters, plus copies through the production codec |
| TTS voice | Record at 44.1 or 48 kHz, train at the model's rate, often 22.05 or 24 kHz | Record 24-bit, deliver 16 or 24-bit | Mono | WAV or FLAC |
| Speech-to-speech and full-duplex models | Native, at or above the model's codec rate (often 24 kHz) | 16-bit | One per speaker | WAV or FLAC |
| Speaker diarization | Production rate, often 16 kHz | 16-bit | One per speaker, mixed down for model input | WAV or FLAC |
What to put in your RFP
Add these lines to the recording conditions in your speech data RFP template:
- Native sample rate of every recording, with written confirmation that nothing was upsampled.
- Delivered sample rate and bit depth, for example 16 kHz, 16-bit PCM.
- Capture chain: devices, and for calls the band (narrowband or wideband) and the codec.
- Channel layout: one channel per speaker, each mapped to a speaker ID in the metadata.
- Format: lossless WAV or FLAC, plus any lossy step in the chain and its bitrate.
- Per-file metadata: sample rate, bit depth, channel count, codec, duration and recording device.
- A sample you can check in a spectrogram before you sign.
Check the audio in a sample
SpeechData.ai licenses spontaneous two-person conversations in 60 languages and dialects, recorded with one channel per speaker, with time-aligned, human-verified transcripts and recording device metadata for every speaker. Ask for a free sample from the dataset catalog and check its sample rate, bandwidth and channel layout against your spec before you buy.
