AI Voice Assistants for Tourism

Step-by-step guide to building AI Voice Assistants for tourism with VideoSDK. Includes code examples and testing.

Introduction to AI Voice Agents in AI Voice Assistants for Tourism

What is an AI

Voice Agent

?

An AI

Voice Agent

 is a software program designed to interact with users through voice commands. It leverages technologies such as Speech-to-Text (STT), Natural Language Processing (NLP), and Text-to-Speech (TTS) to understand and respond to user queries. These agents are becoming increasingly popular in various industries, including tourism, where they enhance customer experience by providing instant information and assistance.

Why are they important for the AI Voice Assistants for Tourism industry?

In the tourism industry, AI Voice Agents can significantly enhance the travel experience by offering real-time information on tourist attractions, local culture, and travel tips. They can suggest itineraries, provide updates on local events, and offer advice on customs and etiquette. This instant access to information helps travelers make informed decisions and enriches their overall experience.

Core Components of a

Voice Agent

  • Speech-to-Text (STT): Converts spoken language into text.
  • Large Language Model (LLM): Processes the text to understand the intent and generate responses.
  • Text-to-Speech (TTS): Converts text responses back into spoken language.

What You'll Build in This Tutorial

In this tutorial, you'll learn to build an AI Voice Assistant tailored for the tourism industry using the VideoSDK framework. We'll guide you through setting up the environment, building the agent, and testing it in a real-world scenario.

Architecture and Core Concepts

High-Level Architecture Overview

The AI

Voice Agent

 processes user speech through a series of steps: capturing audio, converting it to text, processing the text to generate a response, and converting the response back to audio. This seamless flow ensures a natural and intuitive interaction.
Diagram

Understanding Key Concepts in the VideoSDK Framework

  • Agent: The core class representing your bot, responsible for managing interactions.

  • Cascading Pipeline in AI voice Agents

    :     Defines the flow of audio processing from STT to LLM to TTS.
  • VAD & TurnDetector: These components help the agent determine when to listen and when to speak, ensuring smooth conversations.

Setting Up the Development Environment

Prerequisites

To get started, ensure you have Python 3.11+ installed and a VideoSDK account at app.videosdk.live.

Step 1: Create a Virtual Environment

Create a virtual environment to manage dependencies:
1python -m venv myenv
2source myenv/bin/activate  # On Windows use `myenv\\Scripts\\activate`
3

Step 2: Install Required Packages

Install the necessary packages using pip:
1pip install videosdk
2

Step 3: Configure API Keys in a .env file

Create a .env file in your project directory and add your VideoSDK API keys:
1VIDEOSDK_API_KEY=your_api_key_here
2

Building the AI Voice Agent: A Step-by-Step Guide

Here is the complete code block that we will break down:
1import asyncio, os
2from videosdk.agents import Agent, AgentSession, CascadingPipeline, JobContext, RoomOptions, WorkerJob, ConversationFlow
3from videosdk.plugins.silero import SileroVAD
4from videosdk.plugins.turn_detector import TurnDetector, pre_download_model
5from videosdk.plugins.deepgram import DeepgramSTT
6from videosdk.plugins.openai import OpenAILLM
7from videosdk.plugins.elevenlabs import ElevenLabsTTS
8from typing import AsyncIterator
9
10pre_download_model()
11
12agent_instructions = "You are a friendly and knowledgeable AI Voice Assistant specialized in tourism. Your primary role is to assist travelers by providing information about tourist attractions, local culture, and travel tips. You can answer questions about popular destinations, suggest itineraries, and offer advice on local customs and etiquette. However, you are not a travel agent and cannot book flights or accommodations. Always remind users to verify information with official sources and consult local authorities for travel advisories. Your goal is to enhance the travel experience by offering helpful and accurate information."
13
14class MyVoiceAgent(Agent):
15    def __init__(self):
16        super().__init__(instructions=agent_instructions)
17    async def on_enter(self): await self.session.say("Hello! How can I help?")
18    async def on_exit(self): await self.session.say("Goodbye!")
19
20async def start_session(context: JobContext):
21    agent = MyVoiceAgent()
22    conversation_flow = ConversationFlow(agent)
23
24    pipeline = CascadingPipeline(
25        stt=DeepgramSTT(model="nova-2", language="en"),
26        llm=OpenAILLM(model="gpt-4o"),
27        tts=ElevenLabsTTS(model="eleven_flash_v2_5"),
28        vad=[Silero Voice Activity Detection](https://docs.videosdk.live/ai_agents/plugins/silero-vad)(threshold=0.35),
29        turn_detector=[Turn detector for AI voice Agents](https://docs.videosdk.live/ai_agents/plugins/turn-detector)(threshold=0.8)
30    )
31
32    session = [AI voice Agent Sessions](https://docs.videosdk.live/ai_agents/core-components/agent-session)(
33        agent=agent,
34        pipeline=pipeline,
35        conversation_flow=conversation_flow
36    )
37
38    try:
39        await context.connect()
40        await session.start()
41        await asyncio.Event().wait()
42    finally:
43        await session.close()
44        await context.shutdown()
45
46def make_context() -> JobContext:
47    room_options = RoomOptions(
48        name="VideoSDK Cascaded Agent",
49        playground=True
50    )
51
52    return JobContext(room_options=room_options)
53
54if __name__ == "__main__":
55    job = WorkerJob(entrypoint=start_session, jobctx=make_context)
56    job.start()
57

Step 4.1: Generating a VideoSDK Meeting ID

To generate a meeting ID, use the following curl command:
1curl -X POST "https://api.videosdk.live/v1/meetings" \
2-H "Authorization: Bearer your_api_key_here" \
3-H "Content-Type: application/json"
4

Step 4.2: Creating the Custom Agent Class

The MyVoiceAgent class inherits from the Agent class. It initializes with specific instructions tailored for tourism and defines behavior when entering or exiting a session.
1class MyVoiceAgent(Agent):
2    def __init__(self):
3        super().__init__(instructions=agent_instructions)
4    async def on_enter(self): await self.session.say("Hello! How can I help?")
5    async def on_exit(self): await self.session.say("Goodbye!")
6

Step 4.3: Defining the Core Pipeline

The CascadingPipeline defines the sequence of processing stages: STT, LLM, TTS, VAD, and Turn Detection. Each plugin is configured with specific models and thresholds.
1pipeline = CascadingPipeline(
2    stt=DeepgramSTT(model="nova-2", language="en"),
3    llm=OpenAILLM(model="gpt-4o"),
4    tts=ElevenLabsTTS(model="eleven_flash_v2_5"),
5    vad=SileroVAD(threshold=0.35),
6    turn_detector=TurnDetector(threshold=0.8)
7)
8

Step 4.4: Managing the Session and Startup Logic

The start_session function manages the session lifecycle, connecting the agent and pipeline. The make_context function sets up the room options, and the main block starts the job.
1async def start_session(context: JobContext):
2    agent = MyVoiceAgent()
3    conversation_flow = ConversationFlow(agent)
4
5    pipeline = CascadingPipeline(
6        stt=DeepgramSTT(model="nova-2", language="en"),
7        llm=OpenAILLM(model="gpt-4o"),
8        tts=ElevenLabsTTS(model="eleven_flash_v2_5"),
9        vad=SileroVAD(threshold=0.35),
10        turn_detector=TurnDetector(threshold=0.8)
11    )
12
13    session = AgentSession(
14        agent=agent,
15        pipeline=pipeline,
16        conversation_flow=conversation_flow
17    )
18
19    try:
20        await context.connect()
21        await session.start()
22        await asyncio.Event().wait()
23    finally:
24        await session.close()
25        await context.shutdown()
26
27def make_context() -> JobContext:
28    room_options = RoomOptions(
29        name="VideoSDK Cascaded Agent",
30        playground=True
31    )
32
33    return JobContext(room_options=room_options)
34
35if __name__ == "__main__":
36    job = WorkerJob(entrypoint=start_session, jobctx=make_context)
37    job.start()
38

Running and Testing the Agent

Step 5.1: Running the Python Script

To run your agent, execute the script:
1python main.py
2

Step 5.2: Interacting with the Agent in the Playground

After starting the agent, find the playground link in the console. Join the session and interact with your AI Voice Assistant. Use Ctrl+C to gracefully shut down the agent.

Advanced Features and Customizations

Extending Functionality with Custom Tools

You can extend your agent's capabilities by integrating custom tools using the function_tool mechanism. This allows you to add specialized functions tailored to specific needs.

Exploring Other Plugins

Explore other STT, LLM, and TTS plugins to enhance your agent's performance. Options include Cartesia for STT, Google Gemini for LLM, and Deepgram for TTS.

Troubleshooting Common Issues

API Key and Authentication Errors

Ensure your API keys are correctly set in the .env file and that they have the necessary permissions.

Audio Input/Output Problems

Verify your microphone and speaker settings to ensure audio is correctly captured and played back.

Dependency and Version Conflicts

Check for any version conflicts in your dependencies and ensure all packages are up to date.

Conclusion

Summary of What You've Built

In this tutorial, you've built a fully functional AI Voice Assistant for the tourism industry using the VideoSDK framework. This agent can provide valuable information and enhance the travel experience.

Next Steps and Further Learning

Explore additional features and plugins to further customize your agent. Consider integrating more complex NLP capabilities or expanding the agent's knowledge base to cover more topics.

Start Building With Free $20 Balance

No credit card required to start.

Want to level-up your learning? Subscribe now

Subscribe to our newsletter for more tech based insights

FAQ