# ElevenLabs voice (C) vs Gemini Orus (B) and edge-tts (A), Dr Aman's sample shorts

Nothing published. All numbers are MEASURED (no listening possible here); warmth, accent and Gujarati/Hindi naturalness need his ears.
Scripts: SCRIPTS-V3.md verbatim (S02 / S16 / S30). `scripts-v3/SCRIPTS-FINAL.md` does not exist yet, so the opening line is hook A from SCRIPTS-V3 in all three.

## Setup
- Model: `eleven_v4` (accepted by the API, no fallback to v3 needed). Endpoint `/v1/text-to-speech/{voice}/with-timestamps`, `mp3_44100_192`, stability 0.5, similarity 0.75, one request per clip (prosody carries across sentences), character-level alignment mapped back to the script words for caption timing.
- Text formatting per ELEVENLABS-NOTES.md: no `<break>` tags, `[short pause]` between beats, one CAPITALS emphasis in S02 ("Not ALWAYS"), numbers/abbreviations spelled out (S16: "X-ray" spoken as "एक्स-रे", caption unchanged). Nothing else added to the verbatim words.
- Voices (shared Voice Library, English, Indian accent, male, middle-aged; none was in the account, the Library voices were used directly by ID):
  - C1 Aaditya, "Calm, Soft and Soothing", `xMagNCpMgZ83QOEsHNre`
  - C2 Maneesh, "Clear, Warm and Calm", `LolxzR74HCt7Un4IvoxI`

## S02 (English, 73 words), same measurement method as VOICE-COMPARISON.md
Pauses: ffmpeg silencedetect -38 dB / 70 ms at punctuation; pitch: Praat F0; accuracy: faster-whisper small.en. Orus row re-measured in this folder and reproduces the earlier numbers.

| Variant | Narration s | Full-stop pause mean (min) ms | Comma pause mean (min) ms | Pauses at punctuation | Pitch SD st (whole / per word) | Loudness SD dB | Sentence-final fall st | Word error % | Cost |
|---|---|---|---|---|---|---|---|---|---|
| R0 edge-tts Prabhat (current, +5%) | 31.53 | 534 (393) | 232 (165) | 8/8, 5/5 | 4.66 / 3.66 | 1.96 | n/a | 1.4 | Rs 0 |
| A1 edge-tts Prabhat tuned | 33.64 | 535 (413) | 393 (330) | 8/8, 5/5 | 4.66 / 3.68 | 1.87 | n/a | 1.4 | Rs 0 |
| B2 Gemini 2.5 Orus (his pick) | 29.20 | 404 (292) | 140 (81) | 8/8, 4/5 | 3.85 / 3.28 | 1.76 | -7.6 | 0.0 | ~Rs 0.65 |
| C1 ElevenLabs v4, Aaditya | 31.52 | 571 (243) | 139 (112) | 8/8, 5/5 | 5.58 / 4.91 | 2.97 | +1.3 | 0.0 | 519 credits |
| **C2 ElevenLabs v4, Maneesh (chosen)** | 29.84 | 666 (400) | 188 (140) | 8/8, 5/5 | 4.90 / 4.26 | 3.43 | -5.0 | 0.0 | 519 credits |

Choice of C2 over C1: both hit every full stop and comma with 0 % word errors and clearly more pitch and loudness movement than Orus (per-word pitch SD 4.3 to 4.9 vs 3.3; loudness SD 3.0 to 3.4 dB vs 1.8). C1 has the most pitch spread but its sentence endings do not fall (+1.3 st, reads as rising or hanging) and it is 1.7 s slower; C2 falls at sentence ends like Orus (-5.0 vs -7.6), is closest to Orus in pace (29.8 vs 29.2 s) and has the longest, most deliberate full-stop pauses (min 400 ms, none clipped). Caveat: Orus 0.65 Rs vs ElevenLabs is a plan-credit cost, not marginal rupees.

## Clips rendered (all in `out/` of each `-eleven` folder; the originals were not touched)
| Clip | Voice / model | Narration s | Video s | Credits (characters sent) |
|---|---|---|---|---|
| S02-v3-voiceC.mp4 | C2 Maneesh, v4, English | 29.84 | 32.83 | 519 (plus 519 for the C1 audition) |
| S16-v3-voiceC.mp4 | C2 Maneesh, v4, `language_code=hi` | 28.04 | 31.50 | 440 |
| S30-v3-voiceC.mp4 | C2 Maneesh, v4, `language_code=gu` | 35.72 as laid down (38.28 raw from the API) | 38.00 | 550 (plus 452 for a no-tags test, discarded) |

Captions are re-timed to the ElevenLabs word timings (character alignment grouped to script words) in all three; frame spot checks show captions, schematic and end card intact.

### S30 length (Gujarati)
The v4 render is slow and carries long pauses: 38.28 s raw (the edge-tts build had a sped-up voice and a 37.5 s video). Speech is NOT sped up. The ~4.3 s of inter-sentence silence was cut to a fixed 0.12 s gap (sentences cut from the same single render, 8 ms fades), giving 35.72 s of narration; the tail after the last word was shortened 0.30 to 0.20 s and the end card to 2.07 s (normally 2.55 s) so the video lands exactly at the 38.0 s cap. Consequence: Gujarati sentence gaps are tighter than the English/Hindi ones. If he likes the voice and wants natural pauses back, the clip needs either a ~40 s limit or a script trimmed by about 8 words.

## Word accuracy (faster-whisper medium, script vs transcript, punctuation stripped, vowel signs kept)
- Hindi S16, ElevenLabs: WER 26.3 %, CER 8.9 % (76 words). Same test on the current edge-tts Madhur Hindi build: WER 27.6 %, CER 11.6 %. Most differences are Whisper spelling variants (है/हैं, ऑपरेशन/औपरेशन, घुटना/घुट्ना), not missed or wrong words, so Hindi is at least as intelligible as the current voice by this ruler. With small (vs medium) the same clip scores 27.6 % word error.
- Gujarati S30: NOT reliably measurable. Whisper transcribes Gujarati speech into Devanagari-script phonetics; script-vs-transcript word error is 85 % (meaningless), while after folding the Gujarati script into Devanagari the character error is 15 %, which only says the sounds are in the right neighbourhood. Treat Gujarati correctness as UNVERIFIED until a Gujarati speaker listens.

## Credits
- Before: `GET /v1/user/subscription` = character_count 0 / 386,845, tier creator (the v4 launch promo may be keeping the counter at 0 for app use; it was 0 after the first calls too).
- Characters sent to the API this task: 2,480 (S02 519 + 519, S16 440, S30 550 + 452), budget 8,000. Ledger: `voices/credits-ledger.jsonl`.
- After: character_count 301 / 386,845 at the time of the final check (the counter lags or v4 API usage is partly discounted; the sent total above is the safe upper bound).
- Only the chosen clips use the plan's credits at 1 credit per character in the worst case: about 500 per 30 s short.

## Recommendation
- If he prefers expressiveness: ElevenLabs v4 Maneesh beats Orus on every measurable expressiveness signal (pitch movement, loudness movement, deliberate full-stop pauses, all punctuation honoured, 0 % word errors) at similar pace; it is Indian-accented English, which Orus is not. Use it for English and Hindi (Hindi verified as intelligible as the existing voice).
- Gujarati is the open risk: v4 speaks it, but it runs about 2 s longer than the old build and cannot be machine-verified. Have a Gujarati speaker listen before committing; keep Orus or edge-tts gu-IN-NiranjanNeural for Gujarati if it sounds off.
- Orus stays the cheaper fallback (about Rs 0.65 per clip, commercial licence), ElevenLabs costs about 500 plan credits per short (Creator: 386,845 this month, so cost is immaterial at 36 shorts).
- Not measurable here and needs his ear: warmth, whether the accent sounds like a Surat doctor, and any "AI" artefacts. Disclose AI voice in captions as per the compliance note in ELEVENLABS-NOTES.md.
