Every measure on this site is tested against something with a known answer, and some tests fail. We do not hide those: we change the status of the measure, we strike the claim, and we keep the numbers. These are the ones that matter for audio on its way to a recogniser.
-
1. A reverb score that did not follow the reverb
- What we claimed.
- A score from speech that was meant to follow how reverberant a room is.
- What the test showed.
- Tested on 27 September 2026 against 24 rooms with measured reverberation times: a rank correlation of 0.06 and an AUC of 0.52, and it rises with noise.
- What changed.
- Withdrawn everywhere. It stays on /lenses as withdrawn, so nobody finds an old number and trusts it.
-
2. A speech-based measure we said measures the room itself
- What we claimed.
- That the decay at the tail of a phrase measures the room itself.
- What the test showed.
- Against 24 real rooms its rank correlation with the measured reverberation time was 0.56 (the bar was 0.70); the other measure had 0.303 and showed no relationship.
- What changed.
- Both measures are indicative only, and the sentence that said they measure the room itself was struck.
-
3. Our first speech detector
- What we claimed.
- That we find where the speech is, and so the share of speech and the pauses.
- What the test showed.
- No criterion was met. The share of speech read 12.5 percentage points too low on average, and the purity of the pauses was 0.69 against a bar of 0.95.
- What changed.
- Every speech and pause measure became indicative. A detector with hysteresis is better and is offered as an opt-in, but it does not reach the original bars either.
-
4. A signal-to-noise estimate that read too high, and a fix we did not adopt
- What we claimed.
- The signal-to-noise ratio of speech.
- What the test showed.
- At 0 dB the estimate read 10.7 dB too high on average with kitchen noise and 12.5 dB with hallway noise. A hybrid estimator that fixed 5 dB overcorrected at 10 dB and missed its bar (90th percentile 6.21 dB against 6.0).
- What changed.
- The hybrid was not adopted. A correction on the improved detector was, as an opt-in. Below 5 dB the estimate still reads too high, and the margin says so.
-
5. Content type and the label "mixed"
- What we claimed.
- That we can tell speech, music and noise apart, and that speech over music is called mixed.
- What the test showed.
- The first model called noise noise in 3 of 200 fragments. "Mixed" was used in 29 of 100 recordings of speech over music 6 dB softer, where the bar was 80.
- What changed.
- Both are indicative only. A second model is better in every class and is offered as an opt-in, but it still misses more than half of the stationary noise.
-
6. A stationarity score that worked backwards
- What we claimed.
- That a score separates steady machine noise from changing babble.
- What the test showed.
- It scored dense babble as more stationary than a washing machine: AUC 0.23 at 20 dB and 0.27 at 10 dB against a bar of 0.80. The measure for the kind of background failed too.
- What changed.
- Both are indicative only, and the score is now called "Constant background noise", not "machine".
-
7. Jitter and shimmer
- What we claimed.
- Voice roughness measures on running speech.
- What the test showed.
- Both just missed their bars (for example a rank correlation of 0.74 against 0.75 for jitter). On real running speech they read two to three times the clinical norm for a held vowel, where no truth exists.
- What changed.
- Beta: not offered through the API or the MCP server.
Sources and next steps
Other evidence
- Level and clipping
- Codecs
- Packet loss
- Echo: in preparation
- Noise and SNR
- A competing voice
- Scene measures and content type
- Speech detection
- Cut-off at the start and end
- Room and reverberation
- Do audio measures predict errors?
- A cleaning step, before and after: in preparation