Automatic Speech Recognition

Automatic speech recognition (ASR) is the technology that converts spoken words in an audio recording into written text. It uses computational models to identify the most likely sequence of words represented by a speech signal. Recognising words does not, by itself, mean that the system understands their meaning.

Capturing speech as digital data

A microphone converts sound waves into an electrical signal. The signal is digitised through sampling, which measures its amplitude at regular intervals, and quantisation, which represents those measurements using discrete numerical values.

• The resulting digital audio contains information about speech, but may also include background noise, silence and sounds from other speakers.
• Preprocessing may reduce noise, adjust signal levels and identify portions containing speech before recognition proceeds.

Extracting useful speech features

Speech changes continuously, so a recogniser commonly analyses overlapping, short segments called frames. It extracts numerical features that describe how sound energy is distributed across frequencies over time.

• A spectrogram represents the changing frequency content of speech and can provide input to a recognition model.
• Mel-frequency cepstral coefficients are another established feature representation, designed using a frequency scale related to human auditory perception.
• These representations help the system identify speech patterns despite differences in pitch, speaking speed and recording conditions.

Predicting and decoding words

In a traditional recognition pipeline, an acoustic model relates audio features to speech sounds, a pronunciation lexicon connects sound sequences to words, and a language model estimates how likely different word sequences are.

• Decoding combines these sources of evidence to select a likely transcription, because similar sounds can correspond to different words.
• Modern end-to-end neural systems learn a more direct mapping from audio features to text units, such as characters or word fragments, without necessarily using separate pronunciation and acoustic components.
• Recognition models are typically trained using speech recordings paired with their correct transcriptions.

Producing text and handling errors

The predicted text units are assembled into machine-readable text. Punctuation and capitalisation may be added through further processing rather than directly recognised from sound.

• Noise, overlapping speakers, unfamiliar accents, specialised vocabulary and switching between languages can reduce accuracy.
• Word error rate measures substitutions, deletions and insertions relative to the number of words in a reference transcription.
• Representative training data and evaluation across languages and speaker groups are important for assessing reliability.