How does a machine know you said “open”?

When you say "open," what leaves your mouth is a wave of air. A microphone turns it into a sequence of numbers. Then, somewhere, something decides that this sequence of numbers means "open." What happens in between is not the computer understanding language — it is something far more modest, and far more ingenious.

First, what is sound?

As you breathe out, air from the lungs sets the vocal folds in the larynx vibrating: roughly a hundred to a hundred and fifty times per second in men, two hundred to two hundred and fifty in women. The raw buzz that results is then shaped by the throat, the nasal cavities and the mouth.

That distinction matters. The source produced by the vocal folds and the filter applied by the mouth are different things. What separates one vowel from another is not the speed of the vibration but the shape the mouth takes at that moment. And that is what the machine has to be taught.

From wave to numbers

To get a continuous wave into a computer you have to sample it at intervals. The classic rule says: if the sampling rate is more than twice the highest frequency in the signal, the original wave can be reconstructed from the samples without loss. Look often enough and nothing disappears.

The signal is then cleaned: cut into short windows and passed through filters. The point is to reduce the noise picked up during recording and leave something steadier to work with.

Where does speech begin?

A microphone records continuously, but what was actually said is a small part of the recording. The rest is silence and room noise. The system has to find the speech first.

Two simple measures suffice: the energy of a segment, and how many times the signal crosses zero. In silence the energy is low. In hissing consonants the energy is low but the zero-crossing rate is high. Looking at both together separates speech from noise surprisingly well.

Squeezing sound into a handful of numbers

A second of audio is thousands of samples. Comparing them directly is both heavy and meaningless. Instead, each short window is reduced to a few coefficients describing the shape of the vocal tract at that instant. The method is called linear predictive coding.

The idea behind it is lovely: try to predict each sample as a weighted sum of the few samples before it. What carries information is not how good the prediction is, but which weights make it good. Those weights encode the geometry of the vocal tract. Thousands of numbers become a handful of coefficients.

Same word, different speed

Now the real problem. The same person says "open" twice, at different speeds. The two recordings have different lengths and their peaks fall in different places. Comparing them point by point does not work.

The solution is dynamic time warping: allow time to stretch. The algorithm looks for the path that joins the start of one sequence to the start of the other, the end to the end, and minimises the total mismatch along the way. A syllable drawn out in one recording can be matched to a clipped one in the other.

The system keeps a stored template for each command. When a new utterance arrives it is compared against every template and the lowest cost wins. If that cost falls below a set threshold the command counts as recognised; if not, the system honestly says it does not know.

Where does it stand today?

This architecture was built to recognise fifteen Turkish commands — the digits zero through nine, plus open, close, delete, cancel and search. For a small, closed vocabulary it remains more than adequate.

Today's voice assistants do not match templates; they use neural networks trained on millions of hours of speech. The difference is not only scale: the newer systems are not bound to a vocabulary and can parse sentences they have never heard.

Yet the first links of the chain have not changed. Finding where speech begins, reducing the signal to meaningful coefficients, tolerating shifts in time — these are still inside every voice system, in one form or another.

Selim Kusursuz

Founder and writer 5 pieces

Studies Computer Network Systems at Paris-Saclay, on the Autonomous Systems track. Works on mathematics and computer science. Speaks French, English and Turkish well.

See all their articles