Audio annotation adds structured labels to speech recordings. A label can describe what is said, who speaks, when a word or turn occurs, or an event such as overlapping speech. The useful label set depends on the task: training a recognizer, reviewing conversations or preparing data for another speech model.
If you are learning this work, start with our transcription practice guide for contributors. Its exercise is open to read.
18 October 2026 revision: Removed unsupported model-performance and QA threshold claims, qualified annotation requirements, added a fictional review example and updated the annotation enquiry next step.
Choose the labels your task needs
Start with the output your model or team will consume. A transcript, a sequence of speaker turns and a word alignment answer different questions. Writing those outputs down helps you avoid collecting extra layers that the project will not use.
| Label layer | What it describes | A decision for the guideline |
|---|---|---|
| Transcription | The words in a speech segment | Whether to retain fillers, repetitions and false starts; how to write numbers and unclear words |
| Speaker turns | Which anonymous speaker spoke each segment | How to identify speakers within a recording and handle overlap or uncertainty |
| Timestamps | The start and end of a segment, word or other label | Which units need timing and how boundaries will be reviewed |
| Speech events | Events relevant to the task, such as laughter, a cough or overlapping speech | Which events belong in the label set and how to mark their span |
| Intent or emotion categories | A category assigned under a defined rubric | Category definitions, ambiguous cases and when an unknown label is allowed |
| Phonetic or prosodic labels | Pronunciation or features such as stress and tone | The notation, expertise and detail the task requires |
A verbatim transcript and a normalized transcript are different output targets. State the convention explicitly, including punctuation, abbreviations, non-speech sounds and language switches. A speaker label may mean Speaker A within one recording; persistent identities across recordings are a separate requirement.
Check the chosen model's preparation instructions before specifying every layer. For example, Hugging Face's Whisper fine-tuning recipe uses audio and text transcriptions. That example does not require a supplied word-timestamp layer. Another model or downstream application may need different labels or alignment.
Write the guideline and try a pilot
A useful guideline connects a label definition to examples and a review decision. It should say how to handle difficult cases, who resolves disagreements and what the delivered files will contain.
One possible workflow is:
- Define the task. Record the speech type, languages, intended output and label layers. Decide how personally identifying information will be handled within the agreed project scope.
- Specify the files. Define the fields, units, speaker identifiers and text conventions. Include how missing, uncertain and overlapping material is represented.
- Choose a representative pilot. Include examples of the recording conditions and difficult cases the project must handle. Set its size according to the range of cases you need to inspect.
- Compare the labels. Use independent annotation or a reviewed reference where appropriate. Review both the disagreements and the guideline that produced them.
- Revise and version the rules. Keep the examples, decisions and guideline version with the labels. Apply a changed rule consistently to earlier work when it affects delivery.
- Agree acceptance and rework. Specify the sample, checks and response to a failed batch before scaling the process.
Effort depends on the transcription method. Glenn et al. (2010) describe LDC methods with approximate effort ranging from five to fifty work hours per audio hour; the quick five-hour example is for English. Those are historical, defined methods, rather than a current staffing estimate for your recordings. A pilot gives the team evidence for its own scope and workflow.
A fictional annotation and review example
Consider a fictional, untested support-call exercise with the invented line: “Um, I need to change my address.” The team wants both a transcript that retains fillers and a separate readable text version.
- Verbatim layer: “Um, I need to change my address.”
- Normalized layer: “I need to change my address.”
- Speaker rule: use an anonymous identifier within the recording, and the agreed unknown mark when assignment is uncertain.
The review question is whether each layer follows its stated convention. A reviewer would check the recording, the filler rule and any uncertain words, then record the reason for a correction. If a word cannot be resolved, the guideline should provide an uncertainty rule instead of encouraging a guess.
Timing is another decision. A project that needs word boundaries must specify how those boundaries will be produced and checked. This text-only example contains no actual audio and demonstrates no timing or model accuracy. It illustrates how a brief turns into explicit label and review choices.
For your own pilot, keep a short decision record: the task, example identifier, proposed label, relevant guideline rule, reviewer decision and any change required. This makes the next disputed case easier to resolve consistently without treating the example as a universal specification.
Review agreement, accuracy and usability separately
A label-quality score needs a defined comparison. Artstein and Poesio's agreement survey (2008) explains why annotation design and the chosen agreement measure matter. Document the unit, category set, sample and method when reporting agreement.
Three useful checks address different questions:
- Agreement between annotators: compare independent labels for the same sample under the same rules. Investigate the disagreements, including difficult audio, unclear definitions and inconsistent application.
- Accuracy against a reference: compare work with a reviewed reference using the agreed metric and normalization. State how the reference was made and what was sampled.
- Model evaluation: assess the trained model on suitable held-out material using its own evaluation protocol. Keep that result separate from annotation review scores.
Gao et al. (2023) investigate the effect of noisy human transcriptions on two pretrained ASR models in a defined study. For project planning, measure label quality and downstream performance separately rather than translating a transcript error percentage into a guaranteed model result.
Structural checks help too. Check that required fields exist, identifiers resolve, time values fall within the recording and category values match the schema. Overlapping timestamps may be valid for overlapping speakers, so validate them against the label rules rather than rejecting every overlap.
Write acceptance criteria for the actual task. Specify the review sample, reference, metric, treatment of uncertain cases and what happens when work fails review. The appropriate thresholds and review depth need an explicit project decision.
Tools for speech annotation
Choose a tool according to the labels, export format and review workflow you need. The following descriptions use official documentation checked on 18 October 2026.
| Tool | Speech annotation use |
|---|---|
| Label Studio | Templates for transcription, speaker segmentation and audio labels |
| ELAN | Multiple tiers of time-aligned annotations |
| Praat | Phonetic inspection and interval or point labels in TextGrid tiers |
| Audacity | Label tracks for points and regions, with text export |
| Sonic Visualiser | Audio inspection with annotations on time points and segments |
| Prodigy | Developer-oriented recipes for audio regions and transcription |
Check the current license and edition before choosing a workflow. For example, Label Studio's edition comparison distinguishes collaboration, review and agreement features. Confirm that the required functions and export format are available in the edition you plan to use.
Before scaling, export a small labeled example and inspect it with the receiving team. A visually tidy annotation screen does not by itself establish that speaker identifiers, times, text and category values will match the receiving pipeline.
Define the scope before comparing cost
For a project quote, describe the recordings and the work required: languages and varieties, approximate audio volume, recording conditions, speaker count, label layers, output format and review requirements. Share representative examples through the process agreed with the team.
Also agree who can access the recordings, how they will be handled and the relevant retention arrangements. SpeechData.ai agrees storage and access requirements before work begins, with access, storage and retention set in the contract. A tool name or a general service description does not settle them for every recording.
SpeechData.ai quotes annotation per audio hour, item or batch after reviewing a sample. Compare quotes against the same deliverables and acceptance criteria. Extra label layers and a different review scope change what is being commissioned, so a price without its specification is difficult to assess.
Decide which approach fits your recordings
In-house annotation can fit when your team has the required language skills, tooling and time to own the guideline and review. Identify who will resolve difficult cases and inspect exported work before committing to the process.
If you already have recordings and need a team to transcribe or label them, use the annotation brief to describe your audio, required labels and output format. The next step is to review a representative sample and agree the work.
If you need additional recordings rather than labels for your own audio, inspect the language, conditions and included labels in our dataset catalog. Check fit, rights documentation and acceptance requirements for any source you select.
