AI-Powered Call Handling Engine for Taxi Bookings
Empower Solutions has developed a standalone AI-powered call handling engine tailored specifically for a taxi company to automate and manage incoming customer calls for booking taxis.
The solution leverages state-of-the-art speech recognition, natural language processing, and speech synthesis technologies, integrated within a Dockerized environment for portability and scalability.
The client required an automated system capable of handling incoming calls efficiently without human intervention, especially during off-hours.
Outcome Snapshot
24/7 Availability
Scalable
Improved Customer
Experience
Increased Revenue
Privacy and Control
Fully local deployment option ensures data privacy, with no external API calls (beyond VoIP and optional validation).
Challenge
During non-office hours, staff availability was limited, leading to missed booking opportunities and reduced customer satisfaction.
Objectives
Empower Solutions implemented an AI-driven call handling system that enables seamless, automated taxi bookings 24/7.
The call flow is designed to mimic a human-like conversation:
- Incoming voice is streamed from Asterisk to the application.
- Speech is converted to text using a high-performance transcription model.
- The transcribed text is analyzed by a Large Language Model (LLM) to extract the address and formulate a natural response.
- The response is then converted back into speech and played to the caller via Asterisk.
Impact
The result was a fully automated, intelligent voice interaction loop with minimal latency and high accuracy.
Multilingual Support
Expand support for additional languages based on client demographics.
CRM Integration
Analytics Dashboard
Fallback to Human Agent
What We Built
Call Reception: Asterisk receives the incoming call on a configured VoIP number.
Audio Streaming: Voice is streamed to a Python-based socket server using the Asterisk Audiosocket module.
Speech-to-Text: Audio is transcribed using the faster-whisper model for rapid and accurate conversion.
LLM Processing: Transcribed text is analyzed by Mistral LLM to identify the pickup address and generate a conversational response.
Text-to-Speech: Response is converted to speech using Piper TTS and played back to the caller via Asterisk.
Address Validation: If an address is detected, it is validated using Google’s API. If valid, a taxi is booked. If not, the system prompts for clarification and repeats the process until a valid address is confirmed.
Architecture & Deployment
The system integrates with Asterisk to manage incoming calls and uses a Python-based socket server to stream and process audio in real time.
The system can be configured to use any of the following LLM sources:
- Cloud-based Mistral
- AWS Bedrock
- Local Ollama instance (for offline/GPU-accelerated processing)
Supports both English and Italian.
Modular design allows for replacement of any component (STT, LLM, TTS) based on evolving business or technical needs.
Its containerized architecture ensures quick deployment on any infrastructure – cloud or on-premise with full control over data and performance.
Technology Stack
| Technology | Purpose |
|---|---|
| Docker + Ubuntu 22.04 | Containerized deployment for easy hosting and portability |
| Asterisk 20.12.0 | Manages incoming SIP calls and audio routing |
| Faster-Whisper | Fast and accurate speech-to-text transcription |
| Mistral LLM (via Cloud, AWS Bedrock, or Ollama) | Extracts intent and key information (e.g., pickup address) and generates human-like responses |
| Piper TTS | Converts the LLM-generated responses into natural-sounding voice |
| Google Address Validation API | Ensures pickup addresses are valid and serviceable |
Conclusion
This AI-powered call handling engine represents a significant step forward in automating customer communication. While it has been customized for a taxi service, the same system can be easily adapted to other industries such as healthcare, logistics, or customer support.
The ability to toggle between LLM sources gives businesses flexibility in balancing cost, latency, and privacy needs.