AI voice receptionist · 04 / 09
Sawti
A voice receptionist that speaks Libyan Arabic and books real clinic appointments through LLM tool calls.
- Role
- Solo developer · capstone
- Year
- 2026
- Status
- Demo build
- Platform
- Web (local demo)
Overview
The project.
Sawti (صوتي, “my voice”) is a voice receptionist for small clinics that miss calls, especially after hours. A caller speaks in Libyan Arabic, and the agent answers, checks the calendar and books, cancels or reschedules a real appointment.
It was built as a live demo that had to run from a network in Libya, where some AI endpoints are sometimes geo-blocked, using free tiers only. So every stage has a fallback, and a live calendar on screen proves each booking really happened.
The problem
Small clinics lose patients to missed calls. A voice agent is only useful if it understands the local dialect, takes a real action rather than just chatting, and never invents a time slot.
My solution
A single-endpoint pipeline: speech-to-text, an LLM that can act only through validated booking tools, then text-to-speech, with keyless fallbacks at every stage and a dashboard that shows the calendar change live.
What it does
Key features.
- 01
Push-to-talk voice
Record in the browser and send one turn to the server, with a typed-text fallback on the same endpoint.
- 02
Arabic speech recognition
Groq Whisper transcribes Arabic speech without a separate audio conversion step.
- 03
A receptionist in dialect
A Libyan Arabic system prompt with example dialogues, today's date and the recent turns of the conversation.
- 04
Real bookings
Book, cancel and reschedule through function calling. Existing bookings are found by the caller's phone number.
- 05
Hand-off and call logs
An escalate-to-human tool, plus an end-of-call summary written to a call history panel with outcome badges.
- 06
Three voice engines
ElevenLabs or Gemini voices first, then edge-tts with a Libyan neural voice, then a keyless fallback.
- 07
Live dashboard
Calendar tabs where booked slots turn green with the patient's name, a per-turn latency readout and business counters.
- 08
Never a dead end
If every model fails, the caller hears a friendly Arabic message instead of an error.
Under the hood
How it's built.
Architecture and the engineering decisions behind it.
01
One turn, one request
The browser posts audio (or text) to POST /api/turn. The FastAPI pipeline runs speech-to-text, the LLM with tools, then text-to-speech, and returns transcript, reply audio, calendar and latency in one payload.
02
Tools as the only way to act
The LLM loops through tool calls (check availability, book, cancel or reschedule, escalate) for a few hops, then must answer in speech. It can change the calendar only through these tools.
03
Availability is computed, never trusted
Free slots are generated from clinic hours minus bookings. Every write re-checks format, working hours and occupancy, and a partial unique index blocks double bookings in SQLite.
04
Async server, blocking SDKs
The synchronous Groq and Gemini SDKs run in worker threads so the event loop stays responsive, and the async edge-tts client gets its own loop inside its thread.
Problems solved
Technical challenges.
AI endpoints blocked from Libya
A preflight script checks every engine from the demo network beforehand, and at runtime each engine falls back to the next, so a turn never returns a server error.
The model inventing appointment times
Writes are validated server-side and failures go back to the model as reasons like “slot taken”. The prompt forbids confirming any booking the tools didn't make.
Raw audio from the TTS stream
Gemini returns raw PCM. The backend reads its rate and bit depth and writes a WAV header itself, then passes the real MIME type so the browser plays it correctly.
Development status · Demo build
What's next.
Everything above is built and working. These parts are planned and not implemented yet:
- Real phone lines instead of the browser
- Calendar and WhatsApp integrations
- Multiple clinics with accounts
- Hosted deployment over HTTPS