All work

Voice AI / Automation

Nova Voice Call Assistant

A real-time conversational assistant that understands spoken requests, accesses backend tools and data, manages appointment workflows, and responds naturally through voice.

Nova Voice Call Assistant dashboard previewVoice AI / AutomationNova Voice Call Assistant

Overview

Nova is a voice-first AI assistant designed to act like a conversational receptionist for businesses. It brings speech recognition, real-time reasoning, backend tools, business logic, and speech synthesis into one conversational system.

01 problem → system → outcome

The problem

Call handling often means answering repetitive questions, checking schedules, looking up information, booking or changing appointments, recording caller details, summarizing conversations, and handing off complex cases to staff.

Goals

  • Make business calls feel conversational
  • Turn spoken intent into structured actions
  • Keep human escalation available when automation is not appropriate

The solution

The initial version was designed as a browser-based voice assistant using local microphone input, allowing the core conversational workflow to be built and tested without depending on paid telephony infrastructure.

Workflow

  1. 01Browser microphone / caller audio
  2. 02Speech-to-text
  3. 03Intent and entity understanding
  4. 04AI reasoning
  5. 05Tool and backend call
  6. 06Database and business logic
  7. 07Response generation
  8. 08Text-to-speech
  9. 09Voice reply

System design

The frontend and backend communicate through structured WebSocket events so transcripts, assistant state, tool actions, and responses can be updated in real time during a conversation. The architecture uses React, TypeScript, Python, FastAPI, PostgreSQL, SQLite for local or development use where applicable, Docker, speech-to-text, LLM reasoning, tool calling, and TTS audio responses.

Capabilities

  • Real-time voice interaction
  • Microphone audio input and live transcript
  • Intent detection and entity extraction
  • Contextual conversation handling
  • Appointment booking, cancellation, and rescheduling
  • Doctor and service information lookup
  • FAQ retrieval and backend tool calling
  • Conversation memory, history, summaries, and human handoff

Engineering focus

  • Reducing delay across audio, transcription, reasoning, tool action, response, and TTS
  • Maintaining conversation state for follow-up requests
  • Keeping business actions behind explicit backend tools
  • Requesting missing date, time, service, or doctor details
  • Recognizing when a request should be handed to staff

Outcome

Nova demonstrates how a voice system can understand a caller's goal and carry out structured business actions. The project brings voice interaction, business data, and workflow automation into one real-time system.

Next opportunities include Real telephone calls and SIP or telephony-provider integrations, Multi-business and CRM integrations, Clinic or hospital system integrations, WhatsApp follow-up, Multilingual Urdu, Pashto, and English workflows, Provider switching and deeper analytics.

Discuss a similar project