作者构建 VoxEntity-EU 基准：Whisper 德语和法语口述电话号仅 34% 完全正确

I benchmarked whether ASR captures a dictated phone number exactly, in German and French. Whisper: 34% exact. Same text written down: 91% [P]

AI 导读

作者构建了 VoxEntity-EU 基准，用于评估语音系统在德语和法语中能否逐字符准确转录口述的姓名、电话号、邮箱、日期、邮编地址和参考码，一个字符错误即判定该槽位失败。

正文

I built a benchmark around that failure. VoxEntity-EU scores whether a speech system captures a dictated name, phone number, email, date, postal address or reference code exactly, in German and French. One wrong character fails the slot.

The two ASR systems are statistically indistinguishable (difference −2.9 pp [−7.1, +1.3]). It is published as a null result, because that is what it is.

Dataset (audio included): https://huggingface.co/datasets/Gjusev/voxentity-eu

Listen to the failures: https://huggingface.co/spaces/Gjusev/voxentity-eu-results

Code and reports: https://github.com/Gjusev/voxentity-eu

submitted by /u/Gjuesev
[link] [留言]