Build an AI Voice Assistant for Travel

Step-by-step guide to build an AI voice assistant for the travel industry using VideoSDK.

Introduction to AI Voice Agents in the Travel Industry

What is an AI

Voice Agent

?

An AI

Voice Agent

 is an artificial intelligence system designed to interact with users through voice commands. It processes spoken language, understands the intent, and responds appropriately, making it a powerful tool for automating tasks and providing information.

Why are they important for the Travel Industry?

In the travel industry, AI Voice Agents can enhance customer service by providing instant information about destinations, itineraries, and travel tips. They can handle inquiries about flight availability, hotel bookings, and more, offering a seamless experience for travelers.

Core Components of a

Voice Agent

  • Speech-to-Text (STT): Converts spoken language into text.
  • Large Language Model (LLM): Understands and processes the text to generate responses.
  • Text-to-Speech (TTS): Converts the generated text back into speech.

What You'll Build in This Tutorial

In this tutorial, you'll learn how to build a travel-focused AI Voice Assistant using the VideoSDK framework. This assistant will interact with users, providing travel-related information and suggestions.

Architecture and Core Concepts

High-Level Architecture Overview

The AI

Voice Agent

 operates by converting user speech into text using STT, processing the text with an LLM to understand and generate responses, and then converting the response back into speech using TTS.
Diagram

Understanding Key Concepts in the VideoSDK Framework

  • Agent: The core class representing your bot, handling interactions.

  • Cascading Pipeline in AI voice Agents

    :     Manages the flow of audio processing from STT to LLM to TTS.
  • VAD & TurnDetector: These components help the agent determine when to listen and when to speak.

Setting Up the Development Environment

Prerequisites

Before starting, ensure you have Python 3.11+ installed and a VideoSDK account. You can sign up at app.videosdk.live.

Step 1: Create a Virtual Environment

Create a virtual environment to manage dependencies:
1python -m venv travel-voice-agent
2source travel-voice-agent/bin/activate  # On Windows use `travel-voice-agent\Scripts\activate`
3

Step 2: Install Required Packages

Install the necessary packages using pip:
1pip install videosdk
2

Step 3: Configure API Keys in a .env file

Create a .env file in your project directory and add your VideoSDK API keys:
1VIDEOSDK_API_KEY=your_api_key_here
2

Building the AI Voice Agent: A Step-by-Step Guide

First, let's present the complete code block for the AI Voice Agent:
1import asyncio, os
2from videosdk.agents import Agent, AgentSession, CascadingPipeline, JobContext, RoomOptions, WorkerJob, ConversationFlow
3from videosdk.plugins.silero import [Silero Voice Activity Detection](https://docs.videosdk.live/ai_agents/plugins/silero-vad)
4from videosdk.plugins.turn_detector import [Turn detector for AI voice Agents](https://docs.videosdk.live/ai_agents/plugins/turn-detector), pre_download_model
5from videosdk.plugins.deepgram import DeepgramSTT
6from videosdk.plugins.openai import [OpenAI LLM Plugin for voice agent](https://docs.videosdk.live/ai_agents/plugins/llm/openai)
7from videosdk.plugins.elevenlabs import ElevenLabsTTS
8from typing import AsyncIterator
9
10# Pre-downloading the Turn Detector model
11pre_download_model()
12
13agent_instructions = "You are a knowledgeable and friendly AI Voice Assistant specialized in the travel industry. Your primary role is to assist users with travel-related inquiries and tasks. You can provide information about destinations, suggest travel itineraries, check flight and hotel availability, and offer travel tips and advice. However, you are not a travel agent and cannot book flights or accommodations directly. Always remind users to verify travel details with official sources and consult with travel agents for bookings. Your responses should be concise, informative, and engaging, ensuring a pleasant user experience."
14
15class MyVoiceAgent(Agent):
16    def __init__(self):
17        super().__init__(instructions=agent_instructions)
18    async def on_enter(self): await self.session.say("Hello! How can I help?")
19    async def on_exit(self): await self.session.say("Goodbye!")
20
21async def start_session(context: JobContext):
22    # Create agent and conversation flow
23    agent = MyVoiceAgent()
24    conversation_flow = ConversationFlow(agent)
25
26    # Create pipeline
27    pipeline = CascadingPipeline(
28        stt=DeepgramSTT(model="nova-2", language="en"),
29        llm=OpenAILLM(model="gpt-4o"),
30        tts=ElevenLabsTTS(model="eleven_flash_v2_5"),
31        vad=SileroVAD(threshold=0.35),
32        turn_detector=TurnDetector(threshold=0.8)
33    )
34
35    session = AgentSession(
36        agent=agent,
37        pipeline=pipeline,
38        conversation_flow=conversation_flow
39    )
40
41    try:
42        await context.connect()
43        await session.start()
44        # Keep the session running until manually terminated
45        await asyncio.Event().wait()
46    finally:
47        # Clean up resources when done
48        await session.close()
49        await context.shutdown()
50
51def make_context() -> JobContext:
52    room_options = RoomOptions(
53    #  room_id="YOUR_MEETING_ID",  # Set to join a pre-created room; omit to auto-create
54        name="VideoSDK Cascaded Agent",
55        playground=True
56    )
57
58    return JobContext(room_options=room_options)
59
60if __name__ == "__main__":
61    job = WorkerJob(entrypoint=start_session, jobctx=make_context)
62    job.start()
63

Step 4.1: Generating a VideoSDK Meeting ID

To generate a meeting ID, use the following curl command:
1curl -X POST "https://api.videosdk.live/v1/meetings" \
2-H "Authorization: Bearer YOUR_API_KEY" \
3-H "Content-Type: application/json" \
4-d '{}'
5

Step 4.2: Creating the Custom Agent Class

The MyVoiceAgent class is a custom implementation of the Agent class. It defines how the agent interacts with users. The on_enter and on_exit methods manage greetings and farewells:
1class MyVoiceAgent(Agent):
2    def __init__(self):
3        super().__init__(instructions=agent_instructions)
4    async def on_enter(self): await self.session.say("Hello! How can I help?")
5    async def on_exit(self): await self.session.say("Goodbye!")
6

Step 4.3: Defining the Core Pipeline

The CascadingPipeline is crucial as it defines the flow from STT to LLM to TTS. Each plugin plays a specific role:
1pipeline = CascadingPipeline(
2    stt=DeepgramSTT(model="nova-2", language="en"),
3    llm=OpenAILLM(model="gpt-4o"),
4    tts=ElevenLabsTTS(model="eleven_flash_v2_5"),
5    vad=SileroVAD(threshold=0.35),
6    turn_detector=TurnDetector(threshold=0.8)
7)
8

Step 4.4: Managing the Session and Startup Logic

The start_session function initializes the session and manages the lifecycle of the agent:
1async def start_session(context: JobContext):
2    agent = MyVoiceAgent()
3    conversation_flow = ConversationFlow(agent)
4    pipeline = CascadingPipeline(
5        stt=DeepgramSTT(model="nova-2", language="en"),
6        llm=OpenAILLM(model="gpt-4o"),
7        tts=ElevenLabsTTS(model="eleven_flash_v2_5"),
8        vad=SileroVAD(threshold=0.35),
9        turn_detector=TurnDetector(threshold=0.8)
10    )
11    session = AgentSession(
12        agent=agent,
13        pipeline=pipeline,
14        conversation_flow=conversation_flow
15    )
16    try:
17        await context.connect()
18        await session.start()
19        await asyncio.Event().wait()
20    finally:
21        await session.close()
22        await context.shutdown()
23
The make_context function creates a job context with room options:
1def make_context() -> JobContext:
2    room_options = RoomOptions(
3        name="VideoSDK Cascaded Agent",
4        playground=True
5    )
6    return JobContext(room_options=room_options)
7

Running and Testing the Agent

Step 5.1: Running the Python Script

To run the agent, execute the script:
1python main.py
2

Step 5.2: Interacting with the Agent in the Playground

Once running, find the

AI Agent playground

 link in the console. Join the session and interact with your AI Voice Assistant.

Advanced Features and Customizations

Extending Functionality with Custom Tools

You can extend the agent's functionality by implementing custom tools using the function_tool interface.

Exploring Other Plugins

Explore other plugins for STT, LLM, and TTS to enhance the agent's capabilities.

Troubleshooting Common Issues

API Key and Authentication Errors

Ensure your API keys are correctly configured in the .env file.

Audio Input/Output Problems

Check your microphone and speaker settings if you encounter audio issues.

Dependency and Version Conflicts

Ensure all dependencies are compatible with Python 3.11+.

Conclusion

Summary of What You've Built

You've successfully built a travel-focused AI Voice Assistant using VideoSDK, capable of handling travel-related inquiries.

Next Steps and Further Learning

Explore additional features and plugins to enhance your AI Voice Assistant further.

Start Building With Free $20 Balance

No credit card required to start.

Want to level-up your learning? Subscribe now

Subscribe to our newsletter for more tech based insights

FAQ