Building AI Voice Agents for Tourism

Step-by-step guide to building AI voice agents for tourism using VideoSDK.

Introduction to AI Voice Agents in Tourism

What is an AI

Voice Agent

?

An AI

Voice Agent

 is a software application that uses artificial intelligence to interact with users through voice commands. These agents can understand spoken language, process the information, and respond accordingly, providing a seamless conversational experience. They are built using various technologies like Speech-to-Text (STT), Text-to-Speech (TTS), and Language Learning Models (LLM) to interpret and generate human-like responses.

Why are they important for the Tourism Industry?

In the tourism industry, AI Voice Agents can significantly enhance the customer experience by providing instant, 24/7 assistance. They can offer information about tourist attractions, suggest itineraries, recommend dining and accommodation options, and provide travel tips. This level of service can improve customer satisfaction and streamline operations for travel agencies and tourism boards.

Core Components of a

Voice Agent

  • Speech-to-Text (STT): Converts spoken language into text.
  • Language Learning Model (LLM): Processes the text to understand and generate responses.
  • Text-to-Speech (TTS): Converts text responses back into spoken language.

What You'll Build in This Tutorial

In this tutorial, we will guide you through building an AI

Voice Agent

 tailored for the tourism industry using the VideoSDK framework. You will learn how to set up the environment, create an agent, and test it using a

playground environment

.

Architecture and Core Concepts

High-Level Architecture Overview

The AI

Voice Agent

 architecture involves several components working together to process user input and generate responses. The user's speech is first captured and converted to text using STT. This text is then processed by the LLM to understand the intent and generate a suitable response. Finally, the response is converted back to speech using TTS.
Diagram

Understanding Key Concepts in the VideoSDK Framework

  • Agent: The core class representing your bot, responsible for handling interactions.

  • Cascading Pipeline

    :     Manages the flow of audio processing from STT to LLM to TTS.
  • VAD &

    Turn Detector

    :     Tools that help the agent know when to listen and when to speak.

Setting Up the Development Environment

Prerequisites

  • Python 3.11+
  • VideoSDK Account: Sign up at app.videosdk.live to get your API keys.

Step 1: Create a Virtual Environment

To keep dependencies organized, create a virtual environment:
1python -m venv tourism_env
2source tourism_env/bin/activate  # On Windows use `tourism_env\\Scripts\\activate`
3

Step 2: Install Required Packages

Install the necessary packages using pip:
1pip install videosdk-agents videosdk-plugins
2

Step 3: Configure API Keys in a .env file

Create a .env file in your project directory and add your VideoSDK API keys:
1VIDEOSDK_API_KEY=your_api_key_here
2

Building the AI Voice Agent: A Step-by-Step Guide

Complete Code

Here is the complete code for our AI Voice Agent:
1import asyncio, os
2from videosdk.agents import Agent, AgentSession, CascadingPipeline, JobContext, RoomOptions, WorkerJob, ConversationFlow
3from videosdk.plugins.silero import SileroVAD
4from videosdk.plugins.turn_detector import TurnDetector, pre_download_model
5from videosdk.plugins.deepgram import DeepgramSTT
6from videosdk.plugins.openai import OpenAILLM
7from videosdk.plugins.elevenlabs import ElevenLabsTTS
8from typing import AsyncIterator
9
10pre_download_model()
11
12agent_instructions = "You are a friendly and knowledgeable AI Voice Agent specializing in tourism. Your primary role is to assist travelers by providing information about tourist attractions, local culture, and travel tips. You can answer questions about popular destinations, suggest itineraries, and offer recommendations for dining and accommodation. However, you are not a travel agent and cannot book flights or accommodations. Always remind users to verify travel details with official sources and consult local guidelines for the latest travel advisories. Your goal is to enhance the travel experience by offering helpful and accurate information."
13
14class MyVoiceAgent(Agent):
15    def __init__(self):
16        super().__init__(instructions=agent_instructions)
17    async def on_enter(self): await self.session.say("Hello! How can I help?")
18    async def on_exit(self): await self.session.say("Goodbye!")
19
20async def start_session(context: JobContext):
21    agent = MyVoiceAgent()
22    conversation_flow = ConversationFlow(agent)
23
24    pipeline = CascadingPipeline(
25        stt=DeepgramSTT(model="nova-2", language="en"),
26        llm=OpenAILLM(model="gpt-4o"),
27        tts=ElevenLabsTTS(model="eleven_flash_v2_5"),
28        vad=SileroVAD(threshold=0.35),
29        turn_detector=TurnDetector(threshold=0.8)
30    )
31
32    session = [AgentSession](https://docs.videosdk.live/ai_agents/core-components/agent-session)(
33        agent=agent,
34        pipeline=pipeline,
35        conversation_flow=conversation_flow
36    )
37
38    try:
39        await context.connect()
40        await session.start()
41        await asyncio.Event().wait()
42    finally:
43        await session.close()
44        await context.shutdown()
45
46def make_context() -> JobContext:
47    room_options = RoomOptions(
48    #  room_id="YOUR_MEETING_ID",  # Set to join a pre-created room; omit to auto-create
49        name="VideoSDK Cascaded Agent",
50        playground=True
51    )
52
53    return JobContext(room_options=room_options)
54
55if __name__ == "__main__":
56    job = WorkerJob(entrypoint=start_session, jobctx=make_context)
57    job.start()
58

Step 4.1: Generating a VideoSDK Meeting ID

To interact with the agent, you need a meeting ID. You can generate one using the following curl command:
1curl -X POST https://api.videosdk.live/v1/meetings \
2  -H "Authorization: Bearer YOUR_API_KEY" \
3  -H "Content-Type: application/json"
4

Step 4.2: Creating the Custom Agent Class

The MyVoiceAgent class extends the Agent class, providing custom behavior for entering and exiting sessions. This is where you define how the agent greets users and says goodbye.
1class MyVoiceAgent(Agent):
2    def __init__(self):
3        super().__init__(instructions=agent_instructions)
4    async def on_enter(self): await self.session.say("Hello! How can I help?")
5    async def on_exit(self): await self.session.say("Goodbye!")
6

Step 4.3: Defining the Core Pipeline

The

Cascading Pipeline

 is the backbone of the agent's audio processing capabilities. It chains together STT, LLM, and TTS plugins to handle user interactions.
1pipeline = CascadingPipeline(
2    stt=DeepgramSTT(model="nova-2", language="en"),
3    llm=OpenAILLM(model="gpt-4o"),
4    tts=ElevenLabsTTS(model="eleven_flash_v2_5"),
5    vad=SileroVAD(threshold=0.35),
6    turn_detector=TurnDetector(threshold=0.8)
7)
8

Step 4.4: Managing the Session and Startup Logic

The start_session function initializes the agent session and starts the conversation flow. The make_context function sets up the room options for the session.
1def make_context() -> JobContext:
2    room_options = RoomOptions(
3    #  room_id="YOUR_MEETING_ID",  # Set to join a pre-created room; omit to auto-create
4        name="VideoSDK Cascaded Agent",
5        playground=True
6    )
7
8    return JobContext(room_options=room_options)
9
10if __name__ == "__main__":
11    job = WorkerJob(entrypoint=start_session, jobctx=make_context)
12    job.start()
13

Running and Testing the Agent

Step 5.1: Running the Python Script

Run the script using the command:
1python main.py
2

Step 5.2: Interacting with the Agent in the Playground

After running the script, you will receive a playground link in the console. Open this link in your browser to interact with the agent. You can speak to the agent and receive responses in real-time.

Advanced Features and Customizations

Extending Functionality with Custom Tools

You can extend the agent's capabilities by integrating custom tools using the function_tool feature, allowing for specialized processing or data retrieval.

Exploring Other Plugins

The VideoSDK framework supports various plugins for STT, LLM, and TTS. Consider experimenting with different plugins to optimize performance and cost.

Troubleshooting Common Issues

API Key and Authentication Errors

Ensure that your API keys are correctly set in the .env file and that you have the necessary permissions.

Audio Input/Output Problems

Check your microphone and speaker settings, and ensure that your browser has the necessary permissions to access them.

Dependency and Version Conflicts

Make sure all dependencies are up-to-date and compatible with your Python version.

Conclusion

Summary of What You've Built

In this tutorial, you built an AI Voice Agent for the tourism industry using the VideoSDK framework. You learned how to set up the environment, create and configure the agent, and test it in a playground environment.

Next Steps and Further Learning

Explore more advanced features of the

AI voice Agent core components overview

 and consider integrating additional plugins or custom tools to enhance your agent's capabilities.

Start Building With Free $20 Balance

No credit card required to start.

Want to level-up your learning? Subscribe now

Subscribe to our newsletter for more tech based insights

FAQ