VaaniMesh is a live voice-translation mesh for multilingual rooms. A Telugu speaker talks; the Bangla listener hears Bangla, the Hindi listener hears Hindi — each with a spoken voice and a written transcript, a few seconds behind the speaker.
Try the live demo Read the docsIndia runs its meetings, market negotiations, panchayat sessions, and family calls across language lines every day. The usual fixes all tax the conversation: someone volunteers as interpreter and stops participating, everyone falls back to a second language nobody loves, or the group splits into side conversations. Text chat translators exist — but the moment the medium is voice, in real time, with more than two languages in the room, the tooling disappears.
VaaniMesh treats the many-language room as the normal case, not the edge case: up to five participants, five languages, one conversation.
Each participant opens a browser page — a speaker console — picks their language, and talks. Utterances travel to an independent relay that transcribes them, identifies the language actually spoken, translates the meaning (never a phonetic transliteration) into each listener's language, synthesizes a voice, and routes every listener exactly their own version. An operator dashboard watches the whole mesh live.
New rooms start in manual mode: speak, review the captured clip, press Send. When the relay has proven the chain — five clean deliveries, latency in budget — each speaker may flip to automatic.
The relay stamps a mesh header on everything it forwards: source speaker, languages, per-stage latency, sequence number, audio format. What you hear is auditable.
The dashboard shows each connected speaker's mic status, loudness, settings, round-trip time, and the pipeline's rolling latencies — the same numbers we use to debug it.
Turn-based (default), streaming-simultaneous (translation begins mid-sentence), and shared-mic (one phone on the table, speakers identified by the language they speak).
Translation runs on a frontier language model prompted as a live interpreter — colloquial register, native script, fragments handled gracefully.
Recognition, translation, and speech synthesis are independent, swappable stages with a benchmark harness that scores any provider on real reference recordings.
Three wired nodes. Consoles capture and play audio; the relay — a separate server process — owns the entire AI pipeline; the dashboard observes. Nothing about a conversation lives in the browser.
Every requirement, with its real status. A matrix that shows everything green is a matrix nobody believes.
| Requirement | What exists | Status |
|---|---|---|
| Five-language voice translation (te, bn, mr, hi, en) | Full chain live-verified per language, including the Hindi/Marathi shared-script disambiguation | Built |
| Three-node mesh (consoles · relay · dashboard) | Independent relay server, per-speaker settings, live telemetry | Built |
| Mesh header on every forwarded message | Source, mode, per-stage latencies, sequence, audio format | Built |
| Manual send with earned automatic mode | Pending-tray review; auto unlocks on relay-verified chain health | Built |
| Operator dashboard | Speaker cards, loudness, audio rate, RTT, rolling latencies, message log | Built |
| PIN access gate for public demos | Join and dashboard both gated | Built |
| Shared-mic room (one phone on the table) | Language-identification routing verified with two real recorded speakers through one mic; voice-fingerprint tie-break implemented but not yet validated with live humans | Partial |
| Streaming-simultaneous mode (translation mid-sentence) | Full machinery implemented and test-verified against protocol mocks; live operation requires a Sarvam realtime credential we don't yet hold | Partial |
| Provider benchmark harness | Scores any configured provider on real reference clips; live rows recorded for the current stack only — Bhashini/Deepgram rows await keys | Partial |
| Cloud-twin deployment (no-install, 24×7) | Deployment scaffold drafted; demo currently runs from a secure tunnel to our hardware | Roadmap |
| Narrated demo video | Not yet recorded | Roadmap |
| Two speakers sharing a language in shared-mic rooms | Requires acoustic diarization (design selected: streaming Sortformer) | Roadmap |
VaaniMesh is a small Python service (FastAPI + a Pipecat pipeline) plus
static browser consoles — it runs on a single modest machine under a process
manager. The demo you can try below runs on our hardware, exposed through a
secure tunnel; AI stages call cloud models through one metered API key held
server-side. An on-premises install needs only Python 3.12, one API key, and
a browser; a no-install cloud twin is on the roadmap (a deployment scaffold
ships in the repository under deploy/twin/).
Live demo, gated by PIN. Open the app on two devices (earphones recommended), enter the PIN from your invite, pick different languages, and talk. Keep the dashboard open on a third screen if you want to watch the mesh work.
Open the live app Demo guide & PIN
Demo sessions run on our hardware; the link redirects to the current session address. If it doesn't respond, the session is offline — ask us to bring it up.