Build an AI Voice Assistant for Ticket Booking

Step-by-step guide to building an AI voice assistant for ticket booking using VideoSDK.

Introduction to AI Voice Agents in How to Build AI Voice Assistant for Ticket Booking

What is an AI

Voice Agent

?

An AI

Voice Agent

 is a software application that uses artificial intelligence to interact with users through voice commands. These agents can understand spoken language, process the information, and respond in a human-like manner. They are commonly used in various industries to automate customer service, provide information, and perform tasks based on user requests.

Why are they important for the Ticket Booking Industry?

In the ticket booking industry, AI Voice Agents can streamline the booking process, reduce wait times, and enhance customer satisfaction by providing instant responses to inquiries. They can assist users in finding available events, checking seat availability, and guiding them through the booking process, making the experience more efficient and user-friendly.

Core Components of a

Voice Agent

The core components of a

voice agent

 include:
  • Speech-to-Text (STT): Converts spoken language into text.
  • Language Model (LLM): Processes the text to understand the user's intent.
  • Text-to-Speech (TTS): Converts the agent's response text back into spoken language.

What You'll Build in This Tutorial

In this tutorial, you will learn how to build an AI Voice Assistant for ticket booking using the VideoSDK framework. We will guide you through setting up the environment, creating a custom agent, defining the processing pipeline, and running the agent for testing.

Architecture and Core Concepts

High-Level Architecture Overview

The architecture of an AI Voice Agent involves several components that work together to process user input and generate responses. The data flow begins with the user's speech, which is captured and processed through a series of steps:
  1. Voice

    Activity Detection

     (VAD):     Detects when the user is speaking.
  2. Speech-to-Text (STT): Transcribes the spoken words into text.
  3. Language Model (LLM): Analyzes the text to determine the user's intent.
  4. Text-to-Speech (TTS): Converts the response text into speech.

  5. Turn Detector

    :     Manages the conversation flow by determining when the agent should respond.
Diagram

Understanding Key Concepts in the VideoSDK Framework

  • Agent: The core class representing your bot, responsible for handling interactions.
  • CascadingPipeline: Manages the flow of audio processing from STT to LLM to TTS.
  • VAD & TurnDetector: Ensure the agent knows when to listen and respond appropriately.

Setting Up the Development Environment

Prerequisites

Before you begin, ensure you have Python 3.11+ installed and a VideoSDK account. You can sign up at the VideoSDK website.

Step 1: Create a Virtual Environment

To manage dependencies, create a virtual environment using the following command:
1python -m venv venv
2source venv/bin/activate  # On Windows use `venv\\Scripts\\activate`
3

Step 2: Install Required Packages

Install the necessary Python packages using pip:
1pip install videosdk
2pip install python-dotenv
3

Step 3: Configure API Keys in a .env file

Create a .env file in your project directory and add your VideoSDK API keys:
1VIDEOSDK_API_KEY=your_api_key_here
2VIDEOSDK_SECRET_KEY=your_secret_key_here
3

Building the AI Voice Agent: A Step-by-Step Guide

Here's the complete code for the AI Voice Agent:
1import asyncio, os
2from videosdk.agents import Agent, AgentSession, CascadingPipeline, JobContext, RoomOptions, WorkerJob, ConversationFlow
3from videosdk.plugins.silero import SileroVAD
4from videosdk.plugins.turn_detector import TurnDetector, pre_download_model
5from videosdk.plugins.deepgram import DeepgramSTT
6from videosdk.plugins.openai import OpenAILLM
7from videosdk.plugins.elevenlabs import ElevenLabsTTS
8from typing import AsyncIterator
9
10pre_download_model()
11
12agent_instructions = "You are a friendly and efficient AI Voice Assistant specialized in ticket booking. Your primary role is to assist users in booking tickets for various events, including concerts, movies, and flights. You can provide information about available events, check seat availability, and guide users through the booking process. However, you cannot process payments or handle any financial transactions. Always remind users to verify event details and booking confirmations through official channels. You must ensure user privacy and data security at all times, and you are not authorized to store any personal information. If users have questions beyond your capabilities, direct them to customer support for further assistance."
13
14class MyVoiceAgent(Agent):
15    def __init__(self):
16        super().__init__(instructions=agent_instructions)
17    async def on_enter(self): await self.session.say("Hello! How can I help?")
18    async def on_exit(self): await self.session.say("Goodbye!")
19
20async def start_session(context: JobContext):
21    agent = MyVoiceAgent()
22    conversation_flow = ConversationFlow(agent)
23
24    pipeline = CascadingPipeline(
25        stt=DeepgramSTT(model="nova-2", language="en"),
26        llm=OpenAILLM(model="gpt-4o"),
27        tts=ElevenLabsTTS(model="eleven_flash_v2_5"),
28        vad=SileroVAD(threshold=0.35),
29        turn_detector=TurnDetector(threshold=0.8)
30    )
31
32    session = AgentSession(
33        agent=agent,
34        pipeline=pipeline,
35        conversation_flow=conversation_flow
36    )
37
38    try:
39        await context.connect()
40        await session.start()
41        await asyncio.Event().wait()
42    finally:
43        await session.close()
44        await context.shutdown()
45
46def make_context() -> JobContext:
47    room_options = RoomOptions(
48        name="VideoSDK Cascaded Agent",
49        playground=True
50    )
51
52    return JobContext(room_options=room_options)
53
54if __name__ == "__main__":
55    job = WorkerJob(entrypoint=start_session, jobctx=make_context)
56    job.start()
57

Step 4.1: Generating a VideoSDK Meeting ID

To generate a meeting ID, use the VideoSDK API. Here is an example using curl:
1curl -X POST \
2  https://api.videosdk.live/v1/meetings \
3  -H "Authorization: YOUR_API_KEY" \
4  -H "Content-Type: application/json" \
5  -d '{"region": "us-west"}'
6

Step 4.2: Creating the Custom Agent Class

The MyVoiceAgent class is a custom agent that inherits from the Agent class. It defines the behavior of the voice agent, including the welcome and goodbye messages. For a comprehensive understanding of the components involved, refer to the

AI voice Agent core components overview

.

Step 4.3: Defining the Core Pipeline

The

Cascading pipeline in AI voice Agents

 is the heart of the voice agent, connecting the STT, LLM, and TTS components. Each plugin plays a specific role:
  • DeepgramSTT: Transcribes speech to text.
  • OpenAILLM: Processes the text to understand user intent.
  • ElevenLabsTTS: Converts the response text back to speech.
  • SileroVAD: Detects when the user is speaking.
  • TurnDetector: Manages the conversation flow.

Step 4.4: Managing the Session and Startup Logic

The start_session function initializes the

AI voice Agent Sessions

 and manages the lifecycle of the conversation. The make_context function sets up the room options for the agent. The if __name__ == "__main__": block starts the agent job.

Running and Testing the Agent

Step 5.1: Running the Python Script

To run the agent, execute the following command:
1python main.py
2

Step 5.2: Interacting with the Agent in the Playground

After running the script, a

playground

 link will be provided in the console. Use this link to join the session and interact with the agent.

Advanced Features and Customizations

Extending Functionality with Custom Tools

You can extend the agent's functionality by integrating custom tools. This allows the agent to perform additional tasks based on specific requirements.

Exploring Other Plugins

The VideoSDK framework supports various plugins for STT, LLM, and TTS. Explore other options to enhance the agent's capabilities.

Troubleshooting Common Issues

API Key and Authentication Errors

Ensure that your API keys are correctly configured in the .env file. Double-check the keys for any typos.

Audio Input/Output Problems

Verify that your microphone and speakers are correctly set up and functioning. Check system settings if issues persist.

Dependency and Version Conflicts

Ensure that all dependencies are installed with compatible versions. Use a virtual environment to manage packages effectively.

Conclusion

Summary of What You've Built

In this tutorial, you've built a functional AI Voice Assistant for ticket booking using the VideoSDK framework. You learned how to set up the environment, create a custom agent, and define the processing pipeline.

Next Steps and Further Learning

Continue exploring the VideoSDK documentation to learn more about advanced features and customization options. Consider integrating additional plugins to enhance your agent's capabilities.

Start Building With Free $20 Balance

No credit card required to start.

Want to level-up your learning? Subscribe now

Subscribe to our newsletter for more tech based insights

FAQ