Compare voice-agent and TTS vendors, and check conversational audio

Who this is for

You have recordings from two or more voice-agent, text-to-speech or voice-isolation vendors, or from one vendor over time, and you want to compare them with numbers: how loud, how noisy, whether anything clips, how wide the band is, how many pauses.

It is for teams that set their own pass and fail rules on top. It is not for judging how natural a voice sounds, and it does not read the words. See what AudioLab does not do.

What you get

For each recording, about ten numbers. Each one has a status that says how far to trust it (ask for measureStatus and the answer carries it), and, where we know it, a margin that says how far it can be off. We give you starting ranges. The decision to pass or fail a recording stays with you.

Prefer not to write code? The workspace compares two sets of recordings in your browser: sign in on /account, open the Voice tab and switch on “Compare two sets” in the batch. You get the same ten numbers with the same statuses and margins, your audio is not uploaded, and the report exports as one HTML file.

Before you start

1. Get a key. It is free and needs no card: sign in on /account and generate one. The free tier is 1,000 calls a month, 2 GB a month and 60 requests a minute. The ten numbers cost three calls per recording, and the file counts against your monthly data once per call, so three times.
2. Use the same sentences with every vendor. A difference between vendors only means something on the same text, at the same background noise.
3. Send the audio the way your users hear it. Same sample rate and codec as in production. A vendor that sounds fine as a studio WAV can look different after the phone line.
4. Ask for the extras. Add "round": true and "measureStatus": true to the request. Both are off by default, so nothing in an existing integration changes. round gives fewer decimals (46.7 instead of 46.69999999999999). measureStatus adds one key with the status of every measure and, where we know it, its margin. With a raw file as the body, put the two options in the query string instead: ?round=true&measureStatus=true.

The three calls

One recording takes three POST requests with the same body: mixlab/analyze for loudness, voicelab/qa for the voice and the background, and signallab for the bandwidth. Send a public URL (up to 150 MB), or a small file as the raw body (about 4.5 MB). For a local file over 4.5 MB, upload it through a signed URL first and send its objectPath, as described in the OpenAPI spec.

export AUDIOLAB_KEY=al_live_...
BODY='{"url":"https://example.com/vendor-a/sentence-01.wav","round":true,"measureStatus":true}'

for route in mixlab/analyze voicelab/qa signallab; do
  curl -s "https://audiolab.tools/v1/$route" \
    -H "Authorization: Bearer $AUDIOLAB_KEY" \
    -H "Content-Type: application/json" \
    -d "$BODY" > "vendor-a_sentence-01_${route//\//_}.json"
done

pip install audiolabtools

from audiolabtools import AudioLab

numbers = AudioLab().ten_numbers("https://example.com/vendor-a/sentence-01.wav")
for field, m in numbers.items():
    print(field, m.value, m.unit, m.status, m.margin)

import os, requests

KEY = os.environ["AUDIOLAB_KEY"]

def analyze(route, url):
    r = requests.post(
        f"https://audiolab.tools/v1/{route}",
        headers={"Authorization": f"Bearer {KEY}"},
        json={"url": url, "round": True, "measureStatus": True},
        timeout=120,
    )
    if not r.ok:
        # error.code says what happened, see https://audiolab.tools/docs/errors
        raise RuntimeError(r.json()["error"]["code"] + ": " + r.json()["error"]["message"])
    return r.json()

def ten_numbers(url):
    mix, voice, sig = (analyze(r, url) for r in ("mixlab/analyze", "voicelab/qa", "signallab"))
    return {
        "integratedLufs": mix["integratedLufs"],
        "truePeakDb": mix["truePeakDb"],
        "peakDb": mix["peakDb"],
        "speechLevelDb": voice["averageSpeechRmsDb"],
        "noiseFloorDb": voice["noiseFloorDb"],
        "signalToNoiseDb": voice["signalToNoiseDb"],
        "clipping": voice["clipping"]["severity"],
        "speechRatio": voice["speechRatio"],
        "longSilences": voice["longSilences"],
        "bandwidthHz": sig["bandwidthHz"],
    }

const KEY = process.env.AUDIOLAB_KEY!;

async function analyze(route: string, url: string) {
  const r = await fetch(`https://audiolab.tools/v1/${route}`, {
    method: 'POST',
    headers: { Authorization: `Bearer ${KEY}`, 'Content-Type': 'application/json' },
    body: JSON.stringify({ url, round: true, measureStatus: true }),
  });
  const body = await r.json();
  // On an error, body.error.code says what happened: https://audiolab.tools/docs/errors
  if (!r.ok) throw new Error(`${body.error.code}: ${body.error.message}`);
  return body;
}

export async function tenNumbers(url: string) {
  const [mix, voice, sig] = await Promise.all(['mixlab/analyze', 'voicelab/qa', 'signallab'].map((r) => analyze(r, url)));
  return {
    integratedLufs: mix.integratedLufs, truePeakDb: mix.truePeakDb, peakDb: mix.peakDb,
    speechLevelDb: voice.averageSpeechRmsDb, noiseFloorDb: voice.noiseFloorDb, signalToNoiseDb: voice.signalToNoiseDb,
    clipping: voice.clipping.severity, speechRatio: voice.speechRatio, longSilences: voice.longSilences,
    bandwidthHz: sig.bandwidthHz,
  };
}

The ten numbers

1. integratedLufs POST /v1/mixlab/analyze · LUFS Validated

   How loud the whole recording is.

   A level, not a quality. Compare vendors on the same sentences. Conversational audio has no standard target (broadcast uses -23, podcasts -16).

   Typical range: none. Set by your own gain staging, not by the vendor. Compare vendors at the same level instead.

   Margin

   ±0.1 LU. The tolerance of EBU Tech 3341, met by the loudness benchmark on every build.

   Tested against, and what it is not: loudness.integrated

2. truePeakDb POST /v1/mixlab/analyze · dBTP Validated

   The highest level the signal reaches once it is played back, including between samples.

   Information for conversational audio, not a pass or fail. Above 0 means a decoder or an encoder further down the line can clip it.

   Typical range: none. Set by your own gain staging, not by the vendor. Compare vendors at the same level instead.

   Margin

   +0.2 / -0.4 dB. The tolerance for a true-peak meter in ITU-R BS.1770, met by the loudness benchmark on every build.

   Tested against, and what it is not: loudness.truePeak

3. peakDb POST /v1/mixlab/analyze · dBFS Validated

   The highest single sample.

   0 is full scale. Within 1 dB of 0, look at clipping.

   Typical range: none. Set by your own gain staging, not by the vendor. Compare vendors at the same level instead.

   Tested against, and what it is not: level.peak

4. averageSpeechRmsDb POST /v1/voicelab/qa · dBFS Validated

   How loud the voice is while it speaks.

   Measured over the speech only, in dBFS, so it is not on the same scale as LUFS. Compare vendors on the same text.

   Typical range: none. Set by your own gain staging, not by the vendor. Compare vendors at the same level instead.

   Tested against, and what it is not: voice.speechLevel

5. noiseFloorDb POST /v1/voicelab/qa · dBFS Validated

   The level of the quietest tenth of the recording: the background.

   Lower is cleaner. On real noise it sits 1 to 8 dB below the average level of that noise, because it reads the quietest moments. Synthetic voices often have digital silence between the words: the engine then reads its floor, -120. It depends on how loud the file is: compare vendors at the same speech level, or use the signal-to-noise ratio, which does not.

   Synthetic voice, clean

   -120 to -65.4

   Human voice, clean

   -56.1 to -54.2

   Human voice, noise 10 dB below

   -38.8 to -37

   Synthetic voice, phone band

   -120 to -76.1

   Tested against, and what it is not: voice.noiseFloor

6. signalToNoiseDb POST /v1/voicelab/qa · dB Validated

   How far the voice stands above that background.

   An estimate. Within 6 dB when the real ratio is 10 to 30 dB with steady noise. Below 10 dB it reads too high. Above about 50 dB it only says that the pauses are very quiet, often digital silence: do not compare numbers up there.

   Synthetic voice, clean

   46.8 to 101.3

   Human voice, clean

   26.6 to 29.5

   Human voice, noise 10 dB below

   11.9 to 13.8

   Synthetic voice, phone band

   55.4 to 99.5

   Margin

   ±6 dB. Holds for read speech with steady background noise at a true ratio of 10 to 30 dB (90 percent of cases). Below 10 dB it reads too high, by 3 dB or more.

   Tested against, and what it is not: voice.snr

7. clipping POST /v1/voicelab/qa · severity Validated

   Whether samples hit the ceiling and were flattened.

   None, Low, Medium or High, with the count. None on every clean recording we measured. Fix this first: nothing else compares well on a clipped recording.

   Synthetic voice, clean

   None 48 of 48

   Human voice, clean

   None 40 of 40

   Human voice, noise 10 dB below

   None 40 of 40

   Synthetic voice, phone band

   None 48 of 48

   Tested against, and what it is not: voice.clipping

8. speechRatio POST /v1/voicelab/qa · share Proxy

   The share of the recording where the voice is loud enough to count as speech.

   Reads about 12 points low on clean speech: soft word starts and endings count as pause. Use it to compare vendors on the same text, not as a true share.

   Synthetic voice, clean

   0.63 to 0.72

   Human voice, clean

   0.54 to 0.73

   Human voice, noise 10 dB below

   0.36 to 0.58

   Synthetic voice, phone band

   0.57 to 0.68

   Tested against, and what it is not: voice.speechRatio

9. longSilences POST /v1/voicelab/qa · count Proxy

   How many pauses longer than 1.5 seconds sit between stretches of speech.

   Dead air. Exact on 7 or 8 of 10 clean recordings, and within one on almost all. Noise makes it less reliable.

   Typical range: none. It depends on your text: a single sentence has none, a script with pauses has several. Count them on the same sentences for every vendor.

   Tested against, and what it is not: voice.longSilences

10. bandwidthHz POST /v1/signallab · Hz Validated

    The highest frequency that still carries real content.

    About 4,000 is a phone line. Wideband voice reaches about 8,000, and a recording sampled at 44.1 or 48 kHz can show more. A drop when the background is noisy can mean the vendor is cutting highs to hide it.

    Synthetic voice, clean

    8,000

    Human voice, clean

    8,000

    Human voice, noise 10 dB below

    8,000

    Synthetic voice, phone band

    4,100 to 4,200

    Tested against, and what it is not: provenance.bandwidthHz

Ranges are the 5th to the 95th percentile of what the engine measured: 48 synthetic Dutch voices, 40 English read recordings with a very quiet floor, the same recordings with living-room noise added 10 dB below the voice, and the synthetic voices through a phone band. They show what to expect, not what is good.

The ranges are what we measured on clean and on noisy recordings. They are starting points, not standards. For your own limits, see the last step below.

How to read them

1. Look at clipping first. If it is not None, fix the audio. Nothing else compares well on a clipped recording.
2. Treat true peak as information. There is no standard ceiling for conversational audio. Our own broadcast and streaming presets use -1 or -2 dBTP, and they exist for delivery, not for a voice agent. If your pipeline re-encodes the audio, look at the peak at the point where it is re-encoded.
3. Read noise with two numbers. The noise floor says how quiet the background is. The signal-to-noise ratio says how far the voice stands above it. When they disagree, look at the noise floor first: it is the simpler measurement. It reads the quietest tenth of the recording, so on real noise it sits 1 to 8 dB below the average level of that noise.
4. Look at the bandwidth next to the noise. A vendor whose bandwidth drops when the background is noisy may be cutting the high frequencies to hide the noise. That is a trade-off to see, not necessarily a fault.
5. Use the proxies to compare, not to state facts. Speech ratio and long silences are proxies: they reliably tell you which vendor leaves more pauses on the same text, but the speech ratio is not the true share of speech.
6. Do not read a gap smaller than the margin as a result. The margin says how far one number can be from the truth. We have not measured how well the difference between two numbers holds. If two vendors are 3 dB apart on signal-to-noise and the margin is 6 dB, treat it as a hint and listen to the recordings.
7. Set your own limits from your own good recordings. Take twenty recordings you consider good. Use their 5th and 95th percentile as your range, and keep the starting ranges above as a sanity check for the measurement itself.
8. Compare vendors over many sentences, not one. Use the same sentences with every vendor. For each number, look at the median and the spread across the sentences, and at how many sentences go the same way when you put two vendors side by side. We have not measured how well the difference between two vendors holds, so a gap that shows in only a few sentences is a hint.

What it cannot do

• It does not transcribe, and it does not predict transcription errors from audio quality. In our own test of 1,248 transcriptions, signal-to-noise did not predict recognition errors.
• It does not say whether a voice sounds natural or robotic, and it does not tell voices apart.
• It does not measure latency, interruptions or turn-taking.
• Its background hints (sceneBackground and the other scene fields) are heuristics on the pauses, not a recognition of what is making the noise.

The full list is on what AudioLab does not do.

Privacy and limits

• Your audio is not kept. It is analysed in memory in Frankfurt. A compressed file (MP3, FLAC, M4A) passes through a temporary file that is deleted right after decoding. A file you upload through the signed-upload route is removed within the hour. We keep the numbers and, if you let us, the file name. See /privacy.
• Limits. Free: 1,000 calls and 2 GB a month, 60 requests a minute. Plus: 10,000 calls, 20 GB, 120 a minute. A file can be 150 MB when you send a public URL or an objectPath. Every accepted request counts, including one that then fails. An error tells you what to do: /docs/errors.
• Where you stand. GET /v1/usage with your key returns the plan, the calls and data used this month against the limits, and the requests per minute: the same numbers as /account. Reading it is free and does not count as a call. Check it before a big batch instead of finding out from a 402.
• Questions about a number. Write to support@audiolab.tools with the field name and the file, and we will say what we know.