What we did
Both studies fixed their pass criteria in advance and used recordings with the speech marked word by word, so the right answer is known.
- VAD-1, the first detector. 40 read recordings from LibriSpeech dev-clean with word alignment, each in 17 conditions: clean, and four DEMAND noises at 20, 10, 5 and 0 dB. That is 680 mixtures.
- VAD-2, a detector with hysteresis. 40 new recordings from LibriSpeech test-clean in the same 17 conditions (680 mixtures), and the 1,344 recordings of our scene study.
- The start and end measures were checked against constructed signals with a known answer: a tone at the start reads its level within 1 dB, and a file that starts in the middle of speech is seen.
The result
VAD-1: no criterion was met. On clean speech the F1 for speech frames was 0.92, but the share of speech read 12.5 percentage points too low on average (the bar was 4) and the purity of the pauses was 0.69 (the bar was 0.95): soft beginnings and endings of words fall under the threshold. With steady noise the F1 was 0.90 or better down to 20 dB and 0.79 to 0.90 at 10 dB; at 5 dB and below it is unreliable. At 0 dB with living-room or cafeteria noise it calls almost everything speech, and the pause measures return nothing in 82 to 85% of the recordings.
VAD-2: the hysteresis met all five criteria. On clean speech and at 20 and 10 dB the F1 went from 0.878 to 0.941, the purity of the pauses from 0.608 to 0.756 and the mean error of the speech share from 0.170 to 0.082. At 5 dB the F1 went from 0.637 to 0.773. It is an improvement, not a solution: the original bars (purity 0.95, error 0.04) are still out of reach.
What that means for the measures. All the speech and pause measures stay indicative (Proxy): information only, never a pass or fail check. The improved detector is an opt-in in the API (speechDetector: "k2") and the check on this site uses it; it becomes the default on 15 January 2027. The start and end measures are Validated.
How far this reaches
Read English speech with steady background noise. Speech detection, and everything that leans on it (the noise estimate, the pause measures, the scene measures), gives an indication down to about 10 dB signal-to-noise ratio and is weak below that.
What a cut-off beginning, a late stop or a pause counted as speech costs a recogniser is not published yet. It will be added here when its numbers are in our register.
What we have not tested
- Real conversations and telephone speech.
- Television or music behind the speaker.
- Streaming behaviour and real turn-taking.
- Other languages.
- What these errors cost a recogniser.
Sources and next steps
Studies in our register: VAD-1, VAD-2. The status of every measure is on /lenses, and how we test is on /methodology.
Other evidence
- Level and clipping
- Codecs
- Packet loss
- Echo: in preparation
- Noise and SNR
- A competing voice
- Scene measures and content type
- Cut-off at the start and end
- Room and reverberation
- Do audio measures predict errors?
- A cleaning step, before and after: in preparation
- Where we were wrong