TTS Arena’s public listening tests come with an important qualification: not every comparison counts toward its leaderboard. The text-to-speech evaluation project, introduced on February 27, 2024, currently documents ranked voting only for qualifying comparisons using unchanged, first-use Random prompts—not text that users type themselves. 1 2
That distinction shapes what benchmarking speech in the wild means. The arena offers evidence about listener preferences under specified conditions, rather than an unrestricted sample of real-world requests. Its provider agreement explicitly describes ratings as crowd-preference estimates, not guarantees that a system is suitable for a particular purpose. 3
September 2026 announcements of Voice Acting Arena and the Open TTS Leaderboard broaden the evaluation picture, addressing acting performance and multilingual, objective measurements respectively. They are distinct projects, not a new launch of the original TTS Arena. 4 5
From anonymous comparisons to a revised ranking system
The original TTS Arena announcement described a straightforward process: users supplied text, listened to two synthesized versions and selected the more natural result before learning which models produced them. It proposed Elo-like rankings and listed six launch entries: ElevenLabs, MetaVoice, OpenVoice, Pheme, WhisperSpeech and XTTS. Those names describe the historical launch lineup, not a verified current roster. 1
Creator mrfakename publicly announced V2 on Hacker News on May 2, 2025, describing conversational comparisons, randomized voices, optional personal leaderboards and a rebuilt backend. That establishes an announcement date, not necessarily the first deployment date. A preserved project FAQ says V1 became static and its votes did not carry into V2, so the two versions should not be treated as one continuous ranking history. 6 7
Despite its hosting on Hugging Face Spaces, TTS Arena is not an official Hugging Face ranking. Its current provider agreement describes an independent, volunteer-maintained community project that is not operated, sponsored or endorsed by Hugging Face. 3
Which votes count—and what rank means
Current documentation requires Hugging Face sign-in with an account at least 30 days old. Prompts are English-only and limited to 1,000 characters. 8
The detailed voting rules distinguish between experimenting and contributing to the public ranking:
- Only qualifying votes on first-use, unchanged Random prompts affect the leaderboard.
- Typed custom prompts allow comparisons but do not change ratings.
- Playback requirements must be satisfied before voting unlocks. 2
That is more restrictive than introductory copy suggesting that users can type text and have each vote feed the leaderboard. The specific voting policy is the more precise basis for describing ranked participation. 8 2
The ranking documentation also differs from historical Elo-style descriptions. It specifies a Bradley–Terry model fitted across eligible vote history, with ratings centered around 1500. Crucially, display order uses the lower confidence bound rather than the highest point estimate. Position therefore incorporates uncertainty, not just estimated preference. The documented publication threshold is 100 votes; entries below 300 votes are marked Preliminary, and users can optionally reveal low-vote models. 9
The source review did not establish when all these rules took effect or whether earlier V2 votes were retrospectively filtered or recalculated. Historical documentation should therefore not be used as a description of current operation. 7 2 9
What listener preference cannot isolate
Providers supply fixed voice pools for rotation, according to the model-submission documentation. TTS Arena does not use zero-shot voice cloning as its comparison protocol. A reasonable interpretation is that listeners are evaluating model-and-voice configurations: liking a particular voice may contribute to preferring its model’s output. The results do not isolate model architecture alone. 10
Other operators make different choices. Artificial Analysis says its Controlled Voice Arena uses shared cloned reference voices, while its Provider Voice Arena evaluates provider voices. It also reports that most published ratings come from screened paid panels. Those differences in voice controls and recruitment are reasons not to treat scores from separate arenas as interchangeable. 11
English-only prompts and account-age requirements further define the scope of TTS Arena’s evidence. The research package found no demographic or listening-environment audit establishing how well participants represent other languages, customer groups or usage settings. That is an evidence gap, not proof that the preferences are invalid. 8
New projects test different speech qualities
LAION and TTS Arena announced Voice Acting Arena on September 16, 2026. Its organizers describe anonymous comparisons of performances using the same scene and script, with judgments covering overall acting preference, instruction following, authenticity and requested vocal bursts such as laughter or gasps. The premise separates natural-sounding speech from convincing acting. 4
Hugging Face’s September 30 Open TTS Leaderboard announcement describes multilingual and voice-cloning evaluations using word and character error rates, inference speed, time to first audio and speaker similarity. Its authors present these measurements as complements to human preference—not replacements for it. 5
For developers, the implication is to use preference rankings alongside task-specific tests. A favored voice does not, by itself, establish intelligibility across languages, low streaming latency or faithful voice cloning. 5 3
Safeguards and unresolved questions
TTS Arena’s integrity policy prohibits provider-affiliated voting, purchased or coordinated votes, and attempts to identify hidden models through endpoint logs or other signals. It provides for vote exclusion, rating recomputation and publicly labeled suspension. These are published safeguards; the source review found no independent validation or false-positive rates establishing their effectiveness. 12 2
Participation also involves data collection. The operator’s privacy page discloses collection of prompts, generated audio, votes, listening duration, IP addresses and device fingerprints. It says vote, session and anti-abuse records are retained indefinitely and that anonymized research datasets may be published. These are operator disclosures, not privacy-audit findings. 13
One policy conflict remains unresolved: the submission page permits permanent anonymity for genuinely unreleased models, while the provider agreement prohibits indefinite codenames and says anonymity ends after a specified period or vote count. Maintainer clarification is needed. 10 3
Further questions concern the meaning of first-use, Random-prompt sampling and refresh policies, voice-pool selection, and audio-level normalization. The reviewed material did not settle those details. 2 10
The source review also did not obtain a usable current primary-source leaderboard table. This draft therefore names no present winner and reports no current scores or aggregate vote count. A timestamped export with model versions, vote counts and uncertainty values would be needed to support those claims. 14 9
Research scope: This draft draws on a source review dated October 12, 2026. No hands-on model testing or independent audit was conducted; documented rules should not be mistaken for independently verified runtime behavior.