Learn to build a HIPAA-compliant healthcare AI voice agent using VideoSDK. Full code, architecture, and testing included.

## 1. Introduction to AI Voice Agents in healthcare ai voice agent

### What is an AI Voice Agent?
AI voice agents are intelligent software systems capable of understanding and responding to human speech in real time. Powered by advances in speech recognition, natural language processing, and speech synthesis, these agents can interpret spoken language, process it through AI models, and reply in natural-sounding voices.

In practice, a user speaks into a microphone, the agent transcribes the speech into text, analyzes the meaning, generates a response, and then converts that response back into speech. This seamless interaction enables hands-free, conversational experiences.

### Why are they important for the healthcare ai voice agent industry?
AI voice agents are transforming healthcare by streamlining communication and reducing administrative burdens. Key use cases include:
- **Appointment scheduling:** Patients can book, reschedule, or cancel appointments via voice.
- **Patient triage:** The agent can gather symptom information and direct patients to appropriate care.
- **Medication reminders:** Automated reminders help patients adhere to treatment plans.
- **Telehealth support:** Agents can answer general questions and guide patients through telemedicine workflows.

These capabilities improve accessibility, efficiency, and patient satisfaction, while freeing up staff for higher-value tasks.

### Core Components of a Voice Agent
Every AI voice agent relies on three core technologies:
- **STT (Speech-to-Text):** Converts spoken audio into text.
- **LLM (Large Language Model):** Analyzes text and generates intelligent responses.
- **TTS (Text-to-Speech):** Converts text responses back into natural-sounding audio.

For a detailed breakdown of these essential building blocks, see the [AI voice Agent core components overview](https://docs.videosdk.live/ai_agents/core-components/overview).

### What You'll Build in This Tutorial
In this guide, you'll build a healthcare-focused AI voice agent using the Videosdk AI Agents framework. Your agent will:
- Answer general healthcare questions
- Assist with appointment and medication reminders
- Provide empathetic, privacy-conscious support
- Adhere to healthcare privacy guidelines (HIPAA)

By the end, you'll have a fully functional, testable agent ready for further customization.

## 2. Architecture and Core Concepts

### High-Level Architecture Overview
Let's visualize how your healthcare AI voice agent processes a conversation:

```mermaid
sequenceDiagram
  participant User
  participant Microphone
  participant Agent
  participant STT
  participant LLM
  participant TTS
  participant Speaker
  User->>Microphone: Speaks
  Microphone->>Agent: Audio Stream
  Agent->>STT: Audio for Transcription
  STT->>Agent: Text
  Agent->>LLM: Text for Response
  LLM->>Agent: Generated Reply
  Agent->>TTS: Text for Synthesis
  TTS->>Agent: Audio
  Agent->>Speaker: Plays Response
  Speaker->>User: Hears Reply
```

This sequence shows the data flow from user speech to the agent's spoken reply. Each component plays a crucial role in the pipeline. To get started quickly, you can follow the [Voice Agent Quick Start Guide](https://docs.videosdk.live/ai_agents/voice-agent-quick-start) for step-by-step instructions.

### Understanding Key Concepts in the VideoSDK Framework
- **Agent:** The core class representing your voice bot's persona and behavior. You customize its instructions and responses.
- **CascadingPipeline:** Orchestrates the flow of audio and text through STT, LLM, and TTS plugins. For more on this, check out the [Cascading pipeline in AI voice Agents](https://docs.videosdk.live/ai_agents/core-components/cascading-pipeline).
- **VAD (Voice Activity Detection) & TurnDetector:** These plugins help the agent determine when the user is speaking and when it's their turn to respond, enabling smooth, natural conversations.

With these building blocks, you can create robust, real-time conversational agents tailored for healthcare.

## 3. Setting Up the Development Environment

### Prerequisites
Before you begin, ensure you have:
- **Python 3.11+**
- **VideoSDK account** (sign up at app.videosdk.live)
- **API keys** for Deepgram (STT), OpenAI (LLM), and ElevenLabs (TTS)

If you want to experiment and test your agent in a browser, the [AI Agent playground](https://docs.videosdk.live/ai_agents/playground) provides an interactive environment for rapid prototyping.

### Step 1: Create a Virtual Environment
It's best to isolate your dependencies. Run:

```bash
python -m venv venv
# On Windows:
venv\Scripts\activate
# On macOS/Linux:
source venv/bin/activate
```

### Step 2: Install Required Packages
Install the Videosdk AI Agents framework and plugins:

```bash
pip install videosdk-agents videosdk-plugin-deepgram videosdk-plugin-openai videosdk-plugin-elevenlabs videosdk-plugin-silero videosdk-plugin-turn-detector
```

### Step 3: Configure API Keys in a `.env` File
Create a `.env` file in your project root with the following content (replace placeholders with your actual keys):

```env
DEEPGRAM_API_KEY=your_deepgram_api_key
OPENAI_API_KEY=your_openai_api_key
ELEVENLABS_API_KEY=your_elevenlabs_api_key
VIDEOSDK_API_KEY=your_videosdk_api_key
```

Now you're ready to build your agent!

## 4. Building the AI Voice Agent: A Step-by-Step Guide

### Complete, Runnable Code Example
Below is the full Python script for your healthcare AI voice agent. We'll break down each section in detail afterward.

```python
import asyncio, os
from videosdk.agents import Agent, AgentSession, CascadingPipeline, JobContext, RoomOptions, WorkerJob, ConversationFlow
from videosdk.plugins.silero import SileroVAD
from videosdk.plugins.turn_detector import TurnDetector, pre_download_model
from videosdk.plugins.deepgram import DeepgramSTT
from videosdk.plugins.openai import OpenAILLM
from videosdk.plugins.elevenlabs import ElevenLabsTTS
from typing import AsyncIterator

# Pre-downloading the Turn Detector model
pre_download_model()

agent_instructions = "You are a helpful healthcare AI voice agent designed to assist users with general healthcare inquiries and administrative tasks. Your persona is empathetic, patient, and professional, aiming to provide clear and supportive guidance. Your capabilities include: answering general questions about common symptoms, providing information on healthcare services, assisting with appointment scheduling, offering reminders for medications or check-ups, and helping users navigate healthcare resources. Constraints: You are not a licensed medical professional and cannot diagnose, prescribe, or provide medical advice. Always include a disclaimer that users should consult a qualified healthcare provider for medical concerns. Do not collect or store any sensitive personal health information. Maintain user privacy and adhere to HIPAA guidelines where applicable. If a user requests urgent medical help, instruct them to contact emergency services immediately."

class MyVoiceAgent(Agent):
    def __init__(self):
        super().__init__(instructions=agent_instructions)
    async def on_enter(self): await self.session.say("Hello! How can I help?")
    async def on_exit(self): await self.session.say("Goodbye!")

async def start_session(context: JobContext):
    # Create agent and conversation flow
    agent = MyVoiceAgent()
    conversation_flow = ConversationFlow(agent)

    # Create pipeline
    pipeline = CascadingPipeline(
        stt=DeepgramSTT(model="nova-2", language="en"),
        llm=OpenAILLM(model="gpt-4o"),
        tts=ElevenLabsTTS(model="eleven_flash_v2_5"),
        vad=SileroVAD(threshold=0.35),
        turn_detector=TurnDetector(threshold=0.8)
    )

    session = AgentSession(
        agent=agent,
        pipeline=pipeline,
        conversation_flow=conversation_flow
    )

    try:
        await context.connect()
        await session.start()
        # Keep the session running until manually terminated
        await asyncio.Event().wait()
    finally:
        # Clean up resources when done
        await session.close()
        await context.shutdown()

def make_context() -> JobContext:
    room_options = RoomOptions(
    #  room_id="YOUR_MEETING_ID",  # Set to join a pre-created room; omit to auto-create
        name="VideoSDK Cascaded Agent",
        playground=True
    )

    return JobContext(room_options=room_options)

if __name__ == "__main__":
    job = WorkerJob(entrypoint=start_session, jobctx=make_context)
    job.start()
```

Let's break down how each part works.

### Step 4.1: Generating a VideoSDK Meeting ID
Before running your agent, you'll need a meeting room for users to join. You can create one via the VideoSDK API:

```bash
curl -X POST -H "Authorization: YOUR_VIDEOSDK_API_KEY" -H "Content-Type: application/json" -d '{}' https://api.videosdk.live/v2/rooms
```

The response will include a `roomId`. You can use this in your agent by setting `room_id` in `RoomOptions`. For playground testing, you can omit `room_id` and let the agent auto-create a room.

### Step 4.2: Creating the Custom Agent Class
The heart of your agent is the `MyVoiceAgent` class. Let's examine how it's defined and customized for healthcare:

```python
agent_instructions = "You are a helpful healthcare AI voice agent designed to assist users with general healthcare inquiries and administrative tasks. ..."

class MyVoiceAgent(Agent):
    def __init__(self):
        super().__init__(instructions=agent_instructions)
    async def on_enter(self):
        await self.session.say("Hello! How can I help?")
    async def on_exit(self):
        await self.session.say("Goodbye!")
```

- The `agent_instructions` string defines your agent's persona, capabilities, and important constraints (such as not providing medical advice and maintaining privacy).
- `on_enter` and `on_exit` provide polite greetings and farewells.

This ensures your agent is empathetic, professional, and HIPAA-aware.

### Step 4.3: Defining the Core Pipeline
The `CascadingPipeline` connects all the audio and language processing components:

```python
pipeline = CascadingPipeline(
    stt=DeepgramSTT(model="nova-2", language="en"),
    llm=OpenAILLM(model="gpt-4o"),
    tts=ElevenLabsTTS(model="eleven_flash_v2_5"),
    vad=SileroVAD(threshold=0.35),
    turn_detector=TurnDetector(threshold=0.8)
)
```

- **DeepgramSTT:** Converts user speech to text quickly and accurately. Learn more about integrating this plugin in the [Deepgram STT Plugin for voice agent](https://docs.videosdk.live/ai_agents/plugins/stt/deepgram).
- **OpenAILLM:** Uses GPT-4o to generate natural, context-aware responses. For setup and configuration, see the [OpenAI LLM Plugin for voice agent](https://docs.videosdk.live/ai_agents/plugins/llm/openai).
- **ElevenLabsTTS:** Produces high-quality, expressive voice output. Explore the [ElevenLabs TTS Plugin for voice agent](https://docs.videosdk.live/ai_agents/plugins/tts/eleven-labs) for more details.
- **SileroVAD:** Detects when the user is speaking, reducing false triggers. For implementation guidance, refer to [Silero Voice Activity Detection](https://docs.videosdk.live/ai_agents/plugins/silero-vad).
- **TurnDetector:** Determines when it's the agent's turn to reply, for smooth conversations. See the [Turn detector for AI voice Agents](https://docs.videosdk.live/ai_agents/plugins/turn-detector) documentation for more information.

You can swap out these plugins for others (like Cartesia for STT or Google Gemini for LLM) as needed.

### Step 4.4: Managing the Session and Startup Logic
The session management code handles connecting to VideoSDK, starting the agent, and cleaning up resources:

```python
async def start_session(context: JobContext):
    agent = MyVoiceAgent()
    conversation_flow = ConversationFlow(agent)
    pipeline = CascadingPipeline(...)
    session = AgentSession(agent=agent, pipeline=pipeline, conversation_flow=conversation_flow)
    try:
        await context.connect()
        await session.start()
        await asyncio.Event().wait()
    finally:
        await session.close()
        await context.shutdown()

def make_context() -> JobContext:
    room_options = RoomOptions(
        name="VideoSDK Cascaded Agent",
        playground=True
    )
    return JobContext(room_options=room_options)

if __name__ == "__main__":
    job = WorkerJob(entrypoint=start_session, jobctx=make_context)
    job.start()
```

- `start_session` creates the agent, pipeline, and session, then connects and starts everything.
- `make_context` sets up the meeting room (with `playground=True` for easy testing).
- The main block starts the agent worker.

For more on managing and maintaining agent sessions, visit the [AI voice Agent Sessions](https://docs.videosdk.live/ai_agents/core-components/agent-session) documentation.

## 5. Running and Testing the Agent

### Step 5.1: Running the Python Script
Activate your virtual environment and run:

```bash
python main.py
```

In the console, you'll see output with a "Playground" link. This link lets you join the agent's meeting room as a user for live testing.

### Step 5.2: Interacting with the Agent in the Playground
1. Open the provided Playground link in your browser.
2. Join as a participant (with microphone enabled).
3. Speak your healthcare question or request.
4. The agent will respond with synthesized speech.

To gracefully shut down the agent, press `Ctrl+C` in your terminal. This ensures all resources are cleaned up.

## 6. Advanced Features and Customizations

### Extending Functionality with Custom Tools
The Videosdk framework supports `function_tool`, allowing you to add custom actions (like appointment booking or medication reminders). You can define Python functions and expose them to the LLM for dynamic, healthcare-specific workflows.

### Exploring Other Plugins
You can experiment with different plugins for each pipeline stage:
- **STT:** Cartesia (best), Deepgram (cost-effective), Rime (low-cost)
- **TTS:** ElevenLabs (best quality), Deepgram (cost-effective)
- **LLM:** OpenAI GPT-4, Google Gemini

This flexibility lets you optimize for cost, quality, or compliance as needed.

## 7. Troubleshooting Common Issues

### API Key and Authentication Errors
- Double-check your `.env` file for typos or missing keys.
- Ensure your VideoSDK, Deepgram, OpenAI, and ElevenLabs accounts are active.

### Audio Input/Output Problems
- Make sure your microphone and speakers are working.
- Test in the VideoSDK Playground to isolate issues.

### Dependency and Version Conflicts
- Use a fresh virtual environment.
- Run `pip list` to check for conflicting packages.

## 8. Conclusion

You've built a fully functional, HIPAA-aware healthcare AI voice agent using the Videosdk AI Agents framework. This agent can answer general healthcare questions, assist with administrative tasks, and deliver a natural, conversational experience.

Next steps:
- Add custom tools for appointment booking or reminders.
- Explore other plugins for improved quality or cost savings.
- Integrate with healthcare systems for real-world deployments.

With this foundation, you're ready to bring intelligent voice automation to healthcare workflows.

How to Build a Conversational AI Voice Agent with VideoSDK

Integrating Custom Tools with VideoSDK AI Agents

Choosing the Best STT and TTS Plugins for AI Voice Agents

Step-by-step guide to create a healthcare AI voice agent using VideoSDK. Includes code, architecture, and testing instructions.

Build a Healthcare AI Voice Agent with VideoSDK

Healthcare AI Voice Agent 

Free 10,000 minutes for video calls

A comprehensive tutorial for building and testing an AI voice call agent with Python and VideoSDK. Includes code, setup, and troubleshooting.

## 1. Introduction to AI Voice Agents in ai voice call

### What is an AI [Voice Agent](https://docs.videosdk.live/ai_agents/voice-agent-quick-start)?

An AI [Voice Agent](https://docs.videosdk.live/ai_agents/plugins/tts/eleven-labs) is an intelligent software system designed to interact with users through spoken language, simulating natural conversations over voice calls. These agents use advanced speech recognition, language understanding, and speech synthesis technologies to understand user intent, generate appropriate responses, and deliver them in a human-like voice.

### Why are they important for the ai voice call industry?

AI Voice Agents are transforming the ai voice call industry by automating customer support, appointment scheduling, information delivery, and more. They offer 24/7 availability, consistent service quality, and scalability, reducing operational costs and improving user experience. By leveraging the latest advances in AI, businesses can handle high call volumes, personalize interactions, and free up human agents for complex tasks.

### Core Components of a [Voice Agent](https://docs.videosdk.live/ai_agents/plugins/stt/deepgram)

- **Speech-to-Text (STT):** Converts user speech into text.
- **Language Model (LLM):** Understands user intent and generates responses.
- **Text-to-Speech (TTS):** Synthesizes natural-sounding voice responses.
- **Voice [Activity Detection](https://docs.videosdk.live/ai_agents/plugins/silero-vad) (VAD):** Detects when the user starts and stops speaking.
- **Turn Detection:** Determines when the agent should listen or speak.

### What You'll Build in This Tutorial

In this tutorial, you will build a fully functional AI [Voice Agent](https://docs.videosdk.live/ai_agents/plugins/llm/openai) for ai voice call using Python and the VideoSDK AI Agents framework. The agent will be able to:
- Hold a natural conversation with a user over a voice call.
- Answer general questions, guide users, and handle basic tasks.
- Run in a test playground for easy experimentation.

By the end, you'll have a ready-to-test voice agent you can extend for your own use cases.

## 2. Architecture and Core Concepts

### High-Level Architecture Overview

The AI Voice Agent follows a modular pipeline architecture. Here's how the data flows during a voice call:

1. **User speaks:** Audio is streamed to the agent.
2. **Voice Activity Detection (VAD):** Detects when the user is speaking.
3. **Turn Detection:** Determines when the user has finished speaking.
4. **Speech-to-Text (STT):** Converts audio to text.
5. **Language Model (LLM):** Processes the text and generates a response.
6. **Text-to-Speech (TTS):** Converts the response to audio.
7. **Agent speaks:** The synthesized response is played to the user.

For a more detailed [AI voice Agent core components overview](https://docs.videosdk.live/ai_agents/core-components/overview), you can refer to the VideoSDK documentation, which breaks down each essential building block and their roles in the agent's workflow.

#### Mermaid UML Sequence Diagram

![Diagram](https://strapi.videosdk.live/uploads/2831d95b391e912da33a3be5b9a7c90c_acdf85bec3.png)

### Understanding Key Concepts in the VideoSDK Framework

- **Agent:** The core class that encapsulates your bot's behavior and instructions.
- **CascadingPipeline:** Orchestrates the flow of audio and text through STT, LLM, TTS, VAD, and Turn Detection components. To learn more about how this works, see the [Cascading pipeline in AI voice Agents](https://docs.videosdk.live/ai_agents/core-components/cascading-pipeline) documentation.
- **VAD & TurnDetector:** Ensure the agent listens and responds at the right moments, creating natural conversational turns. The [Turn detector for AI voice Agents](https://docs.videosdk.live/ai_agents/plugins/turn-detector) plugin is especially useful for managing conversational flow and detecting when a speaker has finished their turn.

By leveraging these abstractions, you can focus on your agent's logic while the framework handles the complexities of real-time audio processing.

## 3. Setting Up the Development Environment

### Prerequisites

- **Python 3.11+** (Recommended: Python 3.11 for best compatibility)
- **A VideoSDK account** (Sign up for free to access API keys and the dashboard)
- **API keys for Deepgram (STT), OpenAI (LLM), and ElevenLabs (TTS)**

### Step 1: Create a Virtual Environment

It's best practice to use a virtual environment to manage dependencies.

```bash
python3 -m venv venv
source venv/bin/activate  # On Windows: venv\Scripts\activate
```

### Step 2: Install Required Packages

Install the VideoSDK AI Agents framework and required plugins:

```bash
pip install videosdk-agents videosdk-plugins-silero videosdk-plugins-turn-detector videosdk-plugins-deepgram videosdk-plugins-openai videosdk-plugins-elevenlabs
```

### Step 3: Configure API Keys in a .env File

Create a `.env` file in your project directory and add your API keys:

```env
VIDEOSDK_API_KEY=your_videosdk_api_key
DEEPGRAM_API_KEY=your_deepgram_api_key
OPENAI_API_KEY=your_openai_api_key
ELEVENLABS_API_KEY=your_elevenlabs_api_key
```

Replace the placeholder values with your actual credentials from each provider.

## 4. Building the AI Voice Agent: A Step-by-Step Guide

Let's dive into building your AI Voice Agent. First, here is the complete, runnable code.

```python
import asyncio, os
from videosdk.agents import Agent, AgentSession, CascadingPipeline, JobContext, RoomOptions, WorkerJob, ConversationFlow
from videosdk.plugins.silero import SileroVAD
from videosdk.plugins.turn_detector import TurnDetector, pre_download_model
from videosdk.plugins.deepgram import DeepgramSTT
from videosdk.plugins.openai import OpenAILLM
from videosdk.plugins.elevenlabs import ElevenLabsTTS
from typing import AsyncIterator

# Pre-downloading the Turn Detector model
pre_download_model()

agent_instructions = "You are a professional and friendly AI Voice Call Assistant. Your primary role is to facilitate seamless, natural-sounding voice conversations with users over the phone. You can answer general questions, provide information, assist with scheduling, and guide users through automated processes. Always speak clearly, use polite language, and confirm understanding when necessary. You must never collect sensitive personal information (such as credit card numbers or social security numbers), and you cannot provide legal, medical, or financial advice. If a user requests such information or advice, politely inform them that you are unable to assist and recommend contacting a qualified professional. Always respect user privacy and comply with data protection guidelines. If you are unsure about a request, escalate or suggest speaking with a human representative."

class MyVoiceAgent(Agent):
    def __init__(self):
        super().__init__(instructions=agent_instructions)
    async def on_enter(self): await self.session.say("Hello! How can I help?")
    async def on_exit(self): await self.session.say("Goodbye!")

async def start_session(context: JobContext):
    # Create agent and conversation flow
    agent = MyVoiceAgent()
    conversation_flow = ConversationFlow(agent)

    # Create pipeline
    pipeline = CascadingPipeline(
        stt=DeepgramSTT(model="nova-2", language="en"),
        llm=OpenAILLM(model="gpt-4o"),
        tts=ElevenLabsTTS(model="eleven_flash_v2_5"),
        vad=SileroVAD(threshold=0.35),
        turn_detector=TurnDetector(threshold=0.8)
    )

    session = AgentSession(
        agent=agent,
        pipeline=pipeline,
        conversation_flow=conversation_flow
    )

    try:
        await context.connect()
        await session.start()
        # Keep the session running until manually terminated
        await asyncio.Event().wait()
    finally:
        # Clean up resources when done
        await session.close()
        await context.shutdown()

def make_context() -> JobContext:
    room_options = RoomOptions(
    #  room_id="YOUR_MEETING_ID",  # Set to join a pre-created room; omit to auto-create
        name="VideoSDK Cascaded Agent",
        playground=True
    )

    return JobContext(room_options=room_options)

if __name__ == "__main__":
    job = WorkerJob(entrypoint=start_session, jobctx=make_context)
    job.start()
```

Now, let's break down each part of the code to understand how it works.

### Step 4.1: Generating a VideoSDK Meeting ID

Before starting the agent, you need a meeting (room) for the voice call. You can auto-generate a meeting ID or create one manually via the VideoSDK API.

To create a meeting ID using curl:

```bash
curl -X POST "https://api.videosdk.live/v2/rooms" \
  -H "Authorization: YOUR_VIDEOSDK_API_KEY"
```

Replace `YOUR_VIDEOSDK_API_KEY` with your actual VideoSDK API key. The response will include a `roomId` you can use.

In the code above, the `RoomOptions` uses `playground=True` to auto-create a room and provide a test URL. For production, you'd set `room_id` to your generated meeting ID.

### Step 4.2: Creating the Custom Agent Class (MyVoiceAgent)

Let's look at how to define your agent's persona and behavior.

```python
agent_instructions = "You are a professional and friendly AI Voice Call Assistant. Your primary role is to facilitate seamless, natural-sounding voice conversations with users over the phone. You can answer general questions, provide information, assist with scheduling, and guide users through automated processes. Always speak clearly, use polite language, and confirm understanding when necessary. You must never collect sensitive personal information (such as credit card numbers or social security numbers), and you cannot provide legal, medical, or financial advice. If a user requests such information or advice, politely inform them that you are unable to assist and recommend contacting a qualified professional. Always respect user privacy and comply with data protection guidelines. If you are unsure about a request, escalate or suggest speaking with a human representative."

class MyVoiceAgent(Agent):
    def __init__(self):
        super().__init__(instructions=agent_instructions)
    async def on_enter(self): await self.session.say("Hello! How can I help?")
    async def on_exit(self): await self.session.say("Goodbye!")
```

- The `agent_instructions` string defines the agent's persona, capabilities, and constraints.
- The `MyVoiceAgent` class inherits from `Agent` and customizes the greeting and farewell behavior with `on_enter` and `on_exit`.

### Step 4.3: Defining the Core Pipeline (CascadingPipeline and Plugins)

The pipeline handles all audio and language processing.

```python
pipeline = CascadingPipeline(
    stt=DeepgramSTT(model="nova-2", language="en"),
    llm=OpenAILLM(model="gpt-4o"),
    tts=ElevenLabsTTS(model="eleven_flash_v2_5"),
    vad=SileroVAD(threshold=0.35),
    turn_detector=TurnDetector(threshold=0.8)
)
```

- **DeepgramSTT:** Converts user speech to text.
- **OpenAILLM:** Processes the text and generates a response.
- **ElevenLabsTTS:** Converts the response text to natural-sounding speech.
- **SileroVAD:** Detects when the user is speaking.
- **TurnDetector:** Determines when the user has finished speaking.

This modular pipeline makes it easy to swap out components as needed. If you want to experiment with your agent in a safe and interactive environment, the [AI Agent playground](https://docs.videosdk.live/ai_agents/playground) is a great resource for testing and refining your setup.

### Step 4.4: Managing the Session and Startup Logic

The session orchestrates the conversation and manages resources.

```python
async def start_session(context: JobContext):
    agent = MyVoiceAgent()
    conversation_flow = ConversationFlow(agent)
    pipeline = CascadingPipeline(
        stt=DeepgramSTT(model="nova-2", language="en"),
        llm=OpenAILLM(model="gpt-4o"),
        tts=ElevenLabsTTS(model="eleven_flash_v2_5"),
        vad=SileroVAD(threshold=0.35),
        turn_detector=TurnDetector(threshold=0.8)
    )
    session = AgentSession(agent=agent, pipeline=pipeline, conversation_flow=conversation_flow)
    try:
        await context.connect()
        await session.start()
        await asyncio.Event().wait()
    finally:
        await session.close()
        await context.shutdown()

def make_context() -> JobContext:
    room_options = RoomOptions(
        name="VideoSDK Cascaded Agent",
        playground=True
    )
    return JobContext(room_options=room_options)

if __name__ == "__main__":
    job = WorkerJob(entrypoint=start_session, jobctx=make_context)
    job.start()
```

- `start_session` sets up the agent, pipeline, and session, then starts the call.
- `make_context` configures the room and enables the test playground.
- The `__main__` block starts the agent as a worker job.

## 5. Running and Testing the Agent

### Step 5.1: Running the Python Script

Make sure your `.env` file is configured and your virtual environment is activated. Then, run:

```bash
python main.py
```

The script will print a test playground URL in the console.

### Step 5.2: Interacting with the Agent in the Playground

1. **Find the playground link** in your terminal output (it will look like a VideoSDK meeting URL).
2. **Open the link in your browser.**
3. **Join as a participant** and use your microphone to speak with the agent.
4. The agent will greet you and respond to your questions in real time.

To gracefully shut down the agent, press `Ctrl+C` in your terminal. This ensures all resources are cleaned up properly.

## 6. Advanced Features and Customizations

### Extending Functionality with Custom Tools

You can add custom tools (function_tool) to extend your agent's capabilities, such as booking appointments, accessing databases, or integrating with external APIs.

### Exploring Other Plugins

The VideoSDK AI Agents framework supports a variety of plugins:
- **STT:** Cartesia (best), Deepgram (cost-effective), Rime (low-cost)
- **TTS:** ElevenLabs (best quality), Deepgram (cost-effective)
- **LLM:** OpenAI GPT-4, Google Gemini
- **VAD:** SileroVAD
- **Turn Detection:** TurnDetector

Experiment with different combinations to optimize for quality, speed, or cost.

## 7. Troubleshooting Common Issues

### API Key and Authentication Errors

- Double-check your `.env` file for typos or missing keys.
- Ensure your API keys are active and have the correct permissions.

### Audio Input/Output Problems

- Make sure your microphone and speakers are working.
- Test in the playground environment for browser compatibility.

### Dependency and Version Conflicts

- Use Python 3.11+ and a fresh virtual environment.
- Run `pip install --upgrade` to ensure all packages are up to date.

## 8. Conclusion

In this tutorial, you've built a production-ready AI Voice Agent for ai voice call using Python and the VideoSDK AI Agents framework. You learned how to set up the environment, configure plugins, and deploy your agent in a test playground.

From here, you can extend your agent with custom tools, integrate with CRMs, or deploy to production. For guidance on [AI voice Agent deployment](https://docs.videosdk.live/ai_agents/deploy-your-agents), check out the VideoSDK documentation, which covers best practices for launching your agent in real-world scenarios. Explore the VideoSDK documentation and experiment with different plugin combinations to create powerful, conversational voice applications.

Learn to build a production-ready AI voice call agent using Python and VideoSDK. Step-by-step guide, code walkthrough, and testing instructions.

How to Build an AI Voice Call Agent with Python & VideoSDK

Build an AI Voice Call Agent

A practical tutorial for developers to build and test a fully functional Voice AI Call Center Agent using Videosdk's Python framework, with code walkthrough and testing tips.

## 1. Introduction to AI Voice Agents in Voice AI Call Center

### What is an AI [Voice Agent](https://docs.videosdk.live/ai_agents/voice-agent-quick-start)?
AI Voice Agents are intelligent software systems designed to interact with humans over voice calls, simulating the experience of speaking with a live call center representative. Powered by advances in speech recognition, natural language processing, and text-to-speech synthesis, these agents can understand caller queries, provide information, and even perform basic tasks or escalate issues to humans.

### Why Are They Important for the Voice AI Call Center Industry?
Voice AI is transforming call centers by automating routine customer interactions, reducing wait times, and ensuring 24/7 availability. AI Voice Agents enable businesses to scale support operations, improve customer satisfaction, and reduce operational costs. They handle repetitive inquiries, freeing up human agents for complex cases, and ensure consistent, professional communication.

### Core Components of a [Voice Agent](https://docs.videosdk.live/ai_agents/plugins/stt/deepgram)
- **Speech-to-Text (STT):** Converts caller speech to text.
- **Large Language Model (LLM):** Processes the text, understands intent, and generates responses.
- **Text-to-Speech (TTS):** Converts agent responses back to natural-sounding speech.
- **Voice [Activity Detection](https://docs.videosdk.live/ai_agents/plugins/silero-vad) (VAD) & Turn Detection:** Determines when the caller is speaking and when the agent should respond.

### What You'll Build in This Tutorial
In this guide, you'll build a fully functional AI [Voice Agent](https://docs.videosdk.live/ai_agents/plugins/llm/openai) for a call center using Python and the Videosdk AI Agents framework. You'll learn how to set up the environment, implement the agent, and test it interactively in a browser-based playground.

## 2. Architecture and Core Concepts

### High-Level Architecture Overview
At a high level, our Voice AI Call Center Agent works as follows:

- The caller speaks into their microphone.
- The agent uses Voice Activity Detection (VAD) and Turn Detection to determine when the caller has finished speaking.
- The audio is transcribed to text using Speech-to-Text (STT).
- The transcribed text is processed by a Large Language Model (LLM), which generates an appropriate response.
- The response is converted to speech using Text-to-Speech (TTS) and played back to the caller.
- The cycle repeats, with the agent managing the flow of conversation.

For a more detailed [AI voice Agent core components overview](https://docs.videosdk.live/ai_agents/core-components/overview), you can explore how each part of the system interacts and contributes to the overall workflow.

#### Mermaid UML Sequence Diagram
![Diagram](https://strapi.videosdk.live/uploads/f22db8087caada641a787e7bdc3b49e3_885016b889.png)

### Understanding Key Concepts in the VideoSDK Framework
- **Agent:** The core class representing your AI bot. You define its persona, instructions, and behaviors.
- **CascadingPipeline:** Orchestrates the audio processing flow (STT → LLM → TTS), plus VAD and Turn Detection. To learn more about how this works, see the [Cascading pipeline in AI voice Agents](https://docs.videosdk.live/ai_agents/core-components/cascading-pipeline).
- **VAD & TurnDetector:** VAD detects when the caller is speaking; TurnDetector signals when it's the agent's turn to reply.

## 3. Setting Up the Development Environment

### Prerequisites
- **Python 3.11+**
- **A VideoSDK account** (for authentication and agent deployment)

### Step 1: Create a Virtual Environment
It's best practice to use a virtual environment to manage dependencies.

```bash
python3 -m venv venv
source venv/bin/activate  # On Windows: venv\Scripts\activate
```

### Step 2: Install Required Packages
Install the Videosdk AI Agents framework and plugin dependencies:

```bash
pip install videosdk-agents videosdk-plugins-silero videosdk-plugins-turn-detector videosdk-plugins-deepgram videosdk-plugins-openai videosdk-plugins-elevenlabs
```

For high-quality voice output, you can leverage the [ElevenLabs TTS Plugin for voice agent](https://docs.videosdk.live/ai_agents/plugins/tts/eleven-labs) to synthesize natural-sounding speech.

### Step 3: Configure API Keys in a .env File
Create a `.env` file in your project directory and add your API keys:

```env
VIDEOSDK_API_KEY=your_videosdk_api_key
OPENAI_API_KEY=your_openai_api_key
DEEPGRAM_API_KEY=your_deepgram_api_key
ELEVENLABS_API_KEY=your_elevenlabs_api_key
```

You can obtain these keys from the respective service dashboards.

## 4. Building the AI Voice Agent: A Step-by-Step Guide

Let's dive into the code! Here's the complete, runnable Python script for your Voice AI Call Center Agent:

```python
import asyncio, os
from videosdk.agents import Agent, AgentSession, CascadingPipeline, JobContext, RoomOptions, WorkerJob, ConversationFlow
from videosdk.plugins.silero import SileroVAD
from videosdk.plugins.turn_detector import TurnDetector, pre_download_model
from videosdk.plugins.deepgram import DeepgramSTT
from videosdk.plugins.openai import OpenAILLM
from videosdk.plugins.elevenlabs import ElevenLabsTTS
from typing import AsyncIterator

# Pre-downloading the Turn Detector model
pre_download_model()

agent_instructions = "You are a professional and courteous AI Voice Agent designed specifically for a voice AI call center environment. Your persona is that of a knowledgeable, efficient, and empathetic virtual call center representative. 

Capabilities:
- Answer customer inquiries about products, services, account information, and general support topics relevant to the business.
- Route calls or escalate issues to human agents when necessary, following predefined escalation protocols.
- Collect and verify customer information securely and accurately.
- Provide clear, concise, and accurate information based on the company's knowledge base and guidelines.
- Handle multiple types of customer requests, including troubleshooting, order status, billing questions, and appointment scheduling.
- Maintain a friendly, patient, and professional tone at all times.

Constraints and Limitations:
- Do not provide personal opinions, make promises on behalf of the company, or offer information outside the approved knowledge base.
- Never collect sensitive information such as full credit card numbers, passwords, or social security numbers.
- If a request is outside your scope or requires human intervention, politely inform the caller and offer to transfer or escalate the call.
- Always adhere to privacy, security, and compliance guidelines as outlined by the company.
- Clearly state that you are an AI voice agent at the beginning of each call.
- Do not attempt to diagnose or resolve issues that require specialized human expertise; escalate such cases promptly.

Your goal is to provide efficient, accurate, and friendly support, ensuring a seamless customer experience while respecting all privacy and security protocols."

class MyVoiceAgent(Agent):
    def __init__(self):
        super().__init__(instructions=agent_instructions)
    async def on_enter(self): await self.session.say("Hello! How can I help?")
    async def on_exit(self): await self.session.say("Goodbye!")

async def start_session(context: JobContext):
    # Create agent and conversation flow
    agent = MyVoiceAgent()
    conversation_flow = ConversationFlow(agent)

    # Create pipeline
    pipeline = CascadingPipeline(
        stt=DeepgramSTT(model="nova-2", language="en"),
        llm=OpenAILLM(model="gpt-4o"),
        tts=ElevenLabsTTS(model="eleven_flash_v2_5"),
        vad=SileroVAD(threshold=0.35),
        turn_detector=TurnDetector(threshold=0.8)
    )

    session = AgentSession(
        agent=agent,
        pipeline=pipeline,
        conversation_flow=conversation_flow
    )

    try:
        await context.connect()
        await session.start()
        # Keep the session running until manually terminated
        await asyncio.Event().wait()
    finally:
        # Clean up resources when done
        await session.close()
        await context.shutdown()

def make_context() -> JobContext:
    room_options = RoomOptions(
    #  room_id="YOUR_MEETING_ID",  # Set to join a pre-created room; omit to auto-create
        name="VideoSDK Cascaded Agent",
        playground=True
    )

    return JobContext(room_options=room_options)

if __name__ == "__main__":
    job = WorkerJob(entrypoint=start_session, jobctx=make_context)
    job.start()
```

Now, let's break down and explain each part of the code.

### Step 4.1: Generating a VideoSDK Meeting ID
While the agent can auto-create a room, you can generate a meeting ID via the VideoSDK API if you want to join a specific room.

```bash
curl -X POST \
  -H "Authorization: YOUR_VIDEOSDK_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{}' \
  https://api.videosdk.live/v2/rooms
```

Copy the returned `roomId` and use it in the `RoomOptions` if you want your agent to join a pre-created room. For most testing, leaving `room_id` commented will auto-create a room and provide a playground link.

### Step 4.2: Creating the Custom Agent Class (MyVoiceAgent)
The `Agent` class is where you define your agent's persona and behaviors.

```python
agent_instructions = "You are a professional and courteous AI Voice Agent designed specifically for a voice AI call center environment. ..."

class MyVoiceAgent(Agent):
    def __init__(self):
        super().__init__(instructions=agent_instructions)
    async def on_enter(self):
        await self.session.say("Hello! How can I help?")
    async def on_exit(self):
        await self.session.say("Goodbye!")
```

- **agent_instructions:** This string guides the agent's behavior, tone, and limitations.
- **on_enter:** Runs when a new session starts; greets the caller.
- **on_exit:** Runs when the session ends; says goodbye.

### Step 4.3: Defining the Core Pipeline (CascadingPipeline and Plugins)
The pipeline orchestrates all audio and language processing for the agent.

```python
pipeline = CascadingPipeline(
    stt=DeepgramSTT(model="nova-2", language="en"),
    llm=OpenAILLM(model="gpt-4o"),
    tts=ElevenLabsTTS(model="eleven_flash_v2_5"),
    vad=SileroVAD(threshold=0.35),
    turn_detector=TurnDetector(threshold=0.8)
)
```

- **STT:** Deepgram's Nova-2 model for fast, accurate transcription.
- **LLM:** OpenAI's GPT-4o for advanced conversational intelligence.
- **TTS:** ElevenLabs' Flash model for natural-sounding voice output.
- **VAD:** SileroVAD detects when the caller is speaking.
- **TurnDetector:** Determines when the agent should respond. For more details on this feature, check out the [Turn detector for AI voice Agents](https://docs.videosdk.live/ai_agents/plugins/turn-detector).

You can swap these plugins for others (e.g., Cartesia STT, Google Gemini LLM) as needed.

### Step 4.4: Managing the Session and Startup Logic
The session manages the lifecycle of the conversation, connecting everything together.

```python
async def start_session(context: JobContext):
    agent = MyVoiceAgent()
    conversation_flow = ConversationFlow(agent)
    pipeline = CascadingPipeline(
        stt=DeepgramSTT(model="nova-2", language="en"),
        llm=OpenAILLM(model="gpt-4o"),
        tts=ElevenLabsTTS(model="eleven_flash_v2_5"),
        vad=SileroVAD(threshold=0.35),
        turn_detector=TurnDetector(threshold=0.8)
    )
    session = AgentSession(agent=agent, pipeline=pipeline, conversation_flow=conversation_flow)
    try:
        await context.connect()
        await session.start()
        await asyncio.Event().wait()
    finally:
        await session.close()
        await context.shutdown()

def make_context() -> JobContext:
    room_options = RoomOptions(
        # room_id="YOUR_MEETING_ID",
        name="VideoSDK Cascaded Agent",
        playground=True
    )
    return JobContext(room_options=room_options)

if __name__ == "__main__":
    job = WorkerJob(entrypoint=start_session, jobctx=make_context)
    job.start()
```

- **start_session:** Initializes the agent, pipeline, and session. Connects to the room and starts the conversation.
- **make_context:** Sets up the room (with `playground=True` to enable browser testing).
- **WorkerJob:** Runs the session as a job.

## 5. Running and Testing the Agent

### Step 5.1: Running the Python Script
1. Make sure your `.env` file is set up with all API keys.
2. In your terminal, run:

```bash
python main.py
```

3. The script will print a "Playground" link to the console.

### Step 5.2: Interacting with the Agent in the Playground
- Open the provided Playground link in your browser.
- Join the call using your microphone.
- Speak naturally; the agent will greet you and respond to your questions.
- To end the session, press Ctrl+C in your terminal for a graceful shutdown.

This setup allows you to test the agent's behavior as if you were a real caller. For hands-on experimentation, try using the [AI Agent playground](https://docs.videosdk.live/ai_agents/playground) to interact with your agent in a browser environment.

## 6. Advanced Features and Customizations

### Extending Functionality with Custom Tools
You can add custom function tools to enable the agent to perform actions like checking order status, scheduling appointments, or integrating with external APIs.

- Define a Python function for your custom action.
- Register it as a tool in your agent class.

### Exploring Other Plugins
- **STT:** Try Cartesia for best accuracy, or Rime for low-cost alternatives.
- **TTS:** ElevenLabs for best quality; Deepgram for cost-effective TTS.
- **LLM:** Swap in Google Gemini or other supported models.

Experiment with different plugins to optimize for your use case.

## 7. Troubleshooting Common Issues

### API Key and Authentication Errors
- Double-check your `.env` file for typos.
- Ensure your API keys are active and have sufficient quota.

### Audio Input/Output Problems
- Verify your microphone permissions in the browser.
- Check your internet connection and browser compatibility.

### Dependency and Version Conflicts
- Ensure all required packages are installed in your virtual environment.
- Use compatible versions (Python 3.11+ is recommended).

## 8. Conclusion

You've now built a fully functional Voice AI Call Center Agent using Python and Videosdk! This agent can handle customer queries, provide information, and escalate as needed—all with a professional, friendly tone.

Next steps:
- Add custom tools for business-specific actions.
- Integrate with CRM or ticketing systems.
- Deploy your agent for real-world customer support scenarios. For more information on how to take your solution live, see the [AI voice Agent deployment](https://docs.videosdk.live/ai_agents/deploy-your-agents) guide.

Keep exploring the Videosdk AI Agents framework to unlock even more possibilities for conversational AI in the call center industry.

Learn to build a professional Voice AI Call Center Agent using Python and Videosdk. Step-by-step guide with runnable code and testing instructions.

How to Build a Voice AI Call Center Agent (Step-by-Step Guide)

Build a Voice AI Call Center Agent

Step-by-step tutorial to build a gaming AI voice agent using VideoSDK. Includes code, setup, and testing for real-time in-game support.

## Introduction to AI Voice Agents in ai voice agent for gaming

### What is an AI Voice Agent?
An AI Voice Agent is an intelligent, conversational assistant that interacts with users through natural spoken language. Powered by speech recognition, natural language processing, and text-to-speech, these agents can understand, process, and respond to spoken queries in real time.

### Why are they important for the ai voice agent for gaming industry?
In gaming, AI Voice Agents elevate the player experience by providing hands-free, real-time assistance. They can offer game tips, track stats, moderate chat, and help troubleshoot issues—all without disrupting gameplay. As games become more complex and immersive, voice agents bridge the gap between players and in-game information, making gaming more accessible and engaging.

### Core Components of a Voice Agent
A typical AI Voice Agent for gaming consists of:
- **Speech-to-Text (STT):** Converts player speech into text.
- **Large Language Model (LLM):** Processes and understands the text.
- **Text-to-Speech (TTS):** Responds with natural-sounding speech.
- **Voice Activity Detection (VAD):** Detects when a player is speaking.
- **Turn Detection:** Identifies when a conversation turn ends.

For a more detailed breakdown of these elements, refer to the [AI voice Agent core components overview](https://docs.videosdk.live/ai_agents/core-components/overview).

### What You'll Build in This Tutorial
In this guide, you'll build a fully functional AI Voice Agent tailored for gaming using the VideoSDK AI Agents framework. Your agent will provide real-time game support, strategy suggestions, and more—all accessible via voice.

## Architecture and Core Concepts

### High-Level Architecture Overview
Before diving into code, let’s visualize how the AI Voice Agent operates within the VideoSDK ecosystem.

```mermaid
sequenceDiagram
    participant Player
    participant Microphone
    participant VAD
    participant STT
    participant LLM
    participant TTS
    participant Agent
    participant Speaker

    Player->>Microphone: Speaks a query
    Microphone->>VAD: Audio stream
    VAD->>STT: Detected speech
    STT->>LLM: Transcribed text
    LLM->>Agent: Processed intent
    Agent->>LLM: Generates response
    LLM->>TTS: Response text
    TTS->>Speaker: Synthesized speech
    Speaker->>Player: Agent speaks
```

This sequence shows how player speech flows through the system: from detection and transcription, to agent reasoning, and finally to spoken response. To get started quickly with your own implementation, check out the [Voice Agent Quick Start Guide](https://docs.videosdk.live/ai_agents/voice-agent-quick-start).

### Understanding Key Concepts in the VideoSDK Framework
- **Agent:** The core logic and persona of your voice assistant. It handles conversation flow and responses.
- **CascadingPipeline:** Orchestrates the flow of audio and text between STT, LLM, TTS, VAD, and Turn Detection plugins. For an in-depth explanation, see [Cascading pipeline in AI voice Agents](https://docs.videosdk.live/ai_agents/core-components/cascading-pipeline).
- **VAD & TurnDetector:** VAD (Voice Activity Detection) identifies when the player is speaking; TurnDetector determines when a conversational turn is complete, ensuring smooth back-and-forth dialog.

## Setting Up the Development Environment

### Prerequisites (Python 3.11+, VideoSDK Account)
To follow this tutorial, you'll need:
- Python 3.11 or newer
- A VideoSDK account (sign up for free)
- Access to the VideoSDK dashboard for API keys

### Step 1: Create a Virtual Environment
Isolate your dependencies by creating a virtual environment:

```bash
python3.11 -m venv venv
source venv/bin/activate  # On Windows: venv\\Scripts\\activate
```

### Step 2: Install Required Packages
Install the VideoSDK AI Agents framework and plugin dependencies:

```bash
pip install videosdk-agents videosdk-plugins-silero videosdk-plugins-turn-detector videosdk-plugins-deepgram videosdk-plugins-openai videosdk-plugins-elevenlabs
```

### Step 3: Configure API Keys in a .env file
Create a `.env` file in your project root and add your VideoSDK API key as well as any plugin-specific keys (e.g., OpenAI, Deepgram, ElevenLabs):

```env
VIDEOSDK_API_KEY=your_videosdk_api_key
OPENAI_API_KEY=your_openai_api_key
DEEPGRAM_API_KEY=your_deepgram_api_key
ELEVENLABS_API_KEY=your_elevenlabs_api_key
```

## Building the AI Voice Agent: A Step-by-Step Guide

Let’s build the agent, starting with the full code and then breaking it down.

### Full Working Code

```python
import asyncio, os
from videosdk.agents import Agent, AgentSession, CascadingPipeline, JobContext, RoomOptions, WorkerJob, ConversationFlow
from videosdk.plugins.silero import SileroVAD
from videosdk.plugins.turn_detector import TurnDetector, pre_download_model
from videosdk.plugins.deepgram import DeepgramSTT
from videosdk.plugins.openai import OpenAILLM
from videosdk.plugins.elevenlabs import ElevenLabsTTS
from typing import AsyncIterator

# Pre-downloading the Turn Detector model
pre_download_model()

agent_instructions = "You are a friendly and knowledgeable AI Voice Agent designed specifically for gaming environments. Your persona is that of an enthusiastic gaming companion who assists players with in-game information, strategy tips, real-time updates, and general gaming support. Your capabilities include: answering questions about game mechanics, providing walkthroughs and hints, tracking player stats, suggesting strategies, and offering reminders for in-game events or objectives. You can also moderate basic chat interactions and help troubleshoot common technical issues related to gaming platforms. Constraints: You must not provide cheats, hacks, or any content that violates game terms of service. You are not a replacement for official game support or moderators. Always encourage fair play and positive gaming behavior. If a query is outside your scope or requires human intervention, politely inform the user and suggest contacting official support channels."

class MyVoiceAgent(Agent):
    def __init__(self):
        super().__init__(instructions=agent_instructions)
    async def on_enter(self): await self.session.say("Hello! How can I help?")
    async def on_exit(self): await self.session.say("Goodbye!")

async def start_session(context: JobContext):
    # Create agent and conversation flow
    agent = MyVoiceAgent()
    conversation_flow = ConversationFlow(agent)

    # Create pipeline
    pipeline = CascadingPipeline(
        stt=DeepgramSTT(model="nova-2", language="en"),
        llm=OpenAILLM(model="gpt-4o"),
        tts=ElevenLabsTTS(model="eleven_flash_v2_5"),
        vad=SileroVAD(threshold=0.35),
        turn_detector=TurnDetector(threshold=0.8)
    )

    session = AgentSession(
        agent=agent,
        pipeline=pipeline,
        conversation_flow=conversation_flow
    )

    try:
        await context.connect()
        await session.start()
        # Keep the session running until manually terminated
        await asyncio.Event().wait()
    finally:
        # Clean up resources when done
        await session.close()
        await context.shutdown()

def make_context() -> JobContext:
    room_options = RoomOptions(
    #  room_id="YOUR_MEETING_ID",  # Set to join a pre-created room; omit to auto-create
        name="VideoSDK Cascaded Agent",
        playground=True
    )

    return JobContext(room_options=room_options)

if __name__ == "__main__":
    job = WorkerJob(entrypoint=start_session, jobctx=make_context)
    job.start()
```

Now, let’s break down each part of the code.

### Step 4.1: Generating a VideoSDK Meeting ID

Before launching your agent, you need a meeting (room) for it to join. You can generate a meeting ID via the VideoSDK API:

```bash
curl -X POST \
  -H "Authorization: your_videosdk_api_key" \
  -H "Content-Type: application/json" \
  -d '{"region": "sg001"}' \
  https://api.videosdk.live/v2/rooms
```

You can also let the framework auto-create a room by omitting the `room_id` in `RoomOptions`.

### Step 4.2: Creating the Custom Agent Class

The agent’s behavior and persona are defined in a custom class.

```python
agent_instructions = "You are a friendly and knowledgeable AI Voice Agent designed specifically for gaming environments. Your persona is that of an enthusiastic gaming companion who assists players with in-game information, strategy tips, real-time updates, and general gaming support. Your capabilities include: answering questions about game mechanics, providing walkthroughs and hints, tracking player stats, suggesting strategies, and offering reminders for in-game events or objectives. You can also moderate basic chat interactions and help troubleshoot common technical issues related to gaming platforms. Constraints: You must not provide cheats, hacks, or any content that violates game terms of service. You are not a replacement for official game support or moderators. Always encourage fair play and positive gaming behavior. If a query is outside your scope or requires human intervention, politely inform the user and suggest contacting official support channels."

class MyVoiceAgent(Agent):
    def __init__(self):
        super().__init__(instructions=agent_instructions)
    async def on_enter(self):
        await self.session.say("Hello! How can I help?")
    async def on_exit(self):
        await self.session.say("Goodbye!")
```

- **Persona:** The agent is designed as a friendly gaming companion.
- **on_enter/on_exit:** These methods define greetings and farewells when users join or leave.

### Step 4.3: Defining the Core Pipeline

The `CascadingPipeline` connects all the plugins: STT, LLM, TTS, VAD, and Turn Detector.

```python
pipeline = CascadingPipeline(
    stt=DeepgramSTT(model="nova-2", language="en"),
    llm=OpenAILLM(model="gpt-4o"),
    tts=ElevenLabsTTS(model="eleven_flash_v2_5"),
    vad=SileroVAD(threshold=0.35),
    turn_detector=TurnDetector(threshold=0.8)
)
```

- **DeepgramSTT:** Fast, accurate speech-to-text. Learn more about the [Deepgram STT Plugin for voice agent](https://docs.videosdk.live/ai_agents/plugins/stt/deepgram).
- **OpenAILLM:** GPT-4o for natural conversation. See the [OpenAI LLM Plugin for voice agent](https://docs.videosdk.live/ai_agents/plugins/llm/openai) for more details.
- **ElevenLabsTTS:** High-quality, expressive speech. Explore the [ElevenLabs TTS Plugin for voice agent](https://docs.videosdk.live/ai_agents/plugins/tts/eleven-labs).
- **SileroVAD:** Detects when the player is speaking. For implementation specifics, refer to [Silero Voice Activity Detection](https://docs.videosdk.live/ai_agents/plugins/silero-vad).
- **TurnDetector:** Determines end of a conversational turn. For more on this, check out the [Turn detector for AI voice Agents](https://docs.videosdk.live/ai_agents/plugins/turn-detector).

### Step 4.4: Managing the Session and Startup Logic

The session ties everything together and manages the agent’s lifecycle.

```python
async def start_session(context: JobContext):
    agent = MyVoiceAgent()
    conversation_flow = ConversationFlow(agent)
    pipeline = CascadingPipeline(
        stt=DeepgramSTT(model="nova-2", language="en"),
        llm=OpenAILLM(model="gpt-4o"),
        tts=ElevenLabsTTS(model="eleven_flash_v2_5"),
        vad=SileroVAD(threshold=0.35),
        turn_detector=TurnDetector(threshold=0.8)
    )
    session = AgentSession(agent=agent, pipeline=pipeline, conversation_flow=conversation_flow)
    try:
        await context.connect()
        await session.start()
        await asyncio.Event().wait()
    finally:
        await session.close()
        await context.shutdown()

def make_context() -> JobContext:
    room_options = RoomOptions(
        name="VideoSDK Cascaded Agent",
        playground=True
    )
    return JobContext(room_options=room_options)

if __name__ == "__main__":
    job = WorkerJob(entrypoint=start_session, jobctx=make_context)
    job.start()
```

- **JobContext:** Sets up the meeting room and enables the playground for easy testing.
- **WorkerJob:** Launches the agent as a background job.
- **Graceful Shutdown:** Ensures resources are cleaned up on exit.

## Running and Testing the Agent

### Step 5.1: Running the Python Script

To start your agent, run:

```bash
python main.py
```

When the script starts, it will print a VideoSDK Playground URL in the console.

### Step 5.2: Interacting with the Agent in the Playground

1. **Copy the Playground URL** from your terminal.
2. **Open it in your browser**.
3. **Join as a participant**—you’ll see your AI agent in the room.
4. **Use your microphone** to ask questions or request in-game help.
5. **End the session** with Ctrl+C in your terminal for a graceful shutdown.

The playground provides a real-time, browser-based environment to test your AI Voice Agent without extra setup. You can also experiment interactively in the [AI Agent playground](https://docs.videosdk.live/ai_agents/playground) for rapid prototyping and testing.

## Advanced Features and Customizations

### Extending Functionality with Custom Tools
You can add custom tools to your agent for deeper game integration, such as fetching live stats, tracking achievements, or connecting to game APIs.

### Exploring Other Plugins
VideoSDK supports a wide range of plugins. Try alternatives like CartesiaSTT, RimeSTT, or DeepgramTTS, or experiment with Google Gemini for your LLM.

## Troubleshooting Common Issues

### API Key and Authentication Errors
- Double-check your `.env` for correct API keys.
- Ensure your VideoSDK account is active and has the necessary permissions.

### Audio Input/Output Problems
- Verify your microphone and speaker settings in the playground.
- Make sure your browser has granted the necessary permissions.

### Dependency and Version Conflicts
- Use a clean virtual environment.
- Ensure all packages are compatible with Python 3.11+.

## Conclusion

### Summary of What You've Built
You've built a fully functional AI Voice Agent for gaming using VideoSDK. Your agent can understand player queries, provide real-time support, and interact naturally—enhancing the gaming experience.

### Next Steps and Further Learning
Explore custom tools, try different plugins, or integrate your agent with specific games and platforms. Check out the VideoSDK documentation for more advanced features and [AI voice Agent deployment](https://docs.videosdk.live/ai_agents/deploy-your-agents) options.

How to Build a Custom AI Voice Agent with VideoSDK

Integrating Real-Time Voice AI into Your App with VideoSDK

Choosing the Right Speech-to-Text Plugin for AI Voice Agents

Learn to build a gaming AI voice agent using VideoSDK. Step-by-step guide with code, setup, and testing instructions.

Build an AI Voice Agent for Gaming with VideoSDK

AI Voice Agent for Gaming Guide

Step-by-step guide to building a production-ready AI voice agent for customer support using Python and VideoSDK. Includes code, setup, and testing.

## 1. Introduction to AI Voice Agents in ai voice agent for customer support

### What is an AI Voice Agent?
An AI Voice Agent is an intelligent, automated system that interacts with humans via natural spoken language. It leverages speech-to-text (STT), large language models (LLMs), and text-to-speech (TTS) to understand, process, and respond to user queries in real time.

### Why are they important for the ai voice agent for customer support industry?
In customer support, AI Voice Agents provide 24/7 assistance, reduce wait times, and handle routine queries efficiently. They free up human agents to focus on complex issues, improve customer satisfaction, and scale support operations without proportional increases in cost.

### Core Components of a Voice Agent
- **Speech-to-Text (STT):** Converts user speech to text.
- **Large Language Model (LLM):** Understands and generates human-like responses.
- **Text-to-Speech (TTS):** Converts text responses back to natural-sounding speech.
- **Voice Activity Detection (VAD):** Detects when the user is speaking.
- **Turn Detection:** Determines when it's the agent's turn to respond.

### What You'll Build in This Tutorial
You'll build a production-ready AI Voice Agent for customer support using Python and the VideoSDK AI Agents framework. The agent will handle common support tasks, escalate complex issues, and can be tested in a web-based playground.

## 2. Architecture and Core Concepts

### High-Level Architecture Overview
The AI voice agent system consists of several modular components working together to enable real-time, conversational support. Here's a high-level overview:

```mermaid
sequenceDiagram
    participant User
    participant Mic
    participant STT
    participant LLM
    participant TTS
    participant Agent
    participant Speaker
    User->>Mic: Speaks question
    Mic->>STT: Audio stream
    STT->>LLM: Transcribed text
    LLM->>Agent: Generates response
    Agent->>TTS: Response text
    TTS->>Speaker: Synthesized speech
    Speaker->>User: Plays response
```

### Understanding Key Concepts in the VideoSDK Framework
- **Agent:** The core class that defines the agent's persona and logic.
- **CascadingPipeline:** Orchestrates the flow between STT, LLM, TTS, VAD, and turn detection plugins. For a detailed breakdown of these elements, see the [AI voice Agent core components overview](https://docs.videosdk.live/ai_agents/core-components/overview).
- **VAD & TurnDetector:** VAD (Voice Activity Detection) identifies when the user is speaking, while TurnDetector determines when the agent should respond, ensuring natural conversations.

If you're looking for a practical walkthrough to get started, check out the [Voice Agent Quick Start Guide](https://docs.videosdk.live/ai_agents/voice-agent-quick-start) for step-by-step instructions.

The [Cascading pipeline in AI voice Agents](https://docs.videosdk.live/ai_agents/core-components/cascading-pipeline) is a crucial architectural pattern that enables seamless integration of STT, LLM, TTS, and other plugins, ensuring efficient and natural conversational flow.

## 3. Setting Up the Development Environment

### Prerequisites
- Python 3.11+
- A VideoSDK account (for API keys and testing)

### Step 1: Create a Virtual Environment
Open your terminal and run:

```bash
python3 -m venv venv
source venv/bin/activate  # On Windows: venv\Scripts\activate
```

### Step 2: Install Required Packages
Install the VideoSDK AI Agents framework and plugin dependencies:

```bash
pip install videosdk-agents videosdk-plugins-silero videosdk-plugins-turn-detector videosdk-plugins-deepgram videosdk-plugins-openai videosdk-plugins-elevenlabs
```

### Step 3: Configure API Keys in a .env file
Create a `.env` file in your project directory and add your VideoSDK, Deepgram, OpenAI, and ElevenLabs API keys:

```env
VIDEOSDK_API_KEY=your_videosdk_api_key
DEEPGRAM_API_KEY=your_deepgram_api_key
OPENAI_API_KEY=your_openai_api_key
ELEVENLABS_API_KEY=your_elevenlabs_api_key
```

## 4. Building the AI Voice Agent: A Step-by-Step Guide

Let's dive into the code! Here's the complete, runnable implementation for our customer support AI Voice Agent:

```python
import asyncio, os
from videosdk.agents import Agent, AgentSession, CascadingPipeline, JobContext, RoomOptions, WorkerJob, ConversationFlow
from videosdk.plugins.silero import SileroVAD
from videosdk.plugins.turn_detector import TurnDetector, pre_download_model
from videosdk.plugins.deepgram import DeepgramSTT
from videosdk.plugins.openai import OpenAILLM
from videosdk.plugins.elevenlabs import ElevenLabsTTS
from typing import AsyncIterator

# Pre-downloading the Turn Detector model
pre_download_model()

agent_instructions = "You are an AI Voice Agent specializing in customer support. Your persona is that of a friendly, patient, and knowledgeable customer service representative. Your primary goal is to assist customers by answering their questions, resolving common issues, providing information about products or services, and guiding users through troubleshooting steps. You can handle inquiries related to order status, account information, product details, returns, and basic troubleshooting. Always communicate clearly, empathetically, and professionally. If a request is outside your scope or requires human intervention (such as handling sensitive personal data, processing payments, or making policy decisions), politely inform the customer and offer to escalate the issue to a human representative. Never provide legal, financial, or medical advice. Do not collect or store sensitive personal information. Always prioritize customer privacy and data security. If you are unsure about an answer, admit it and suggest connecting with a human agent for further assistance."

class MyVoiceAgent(Agent):
    def __init__(self):
        super().__init__(instructions=agent_instructions)
    async def on_enter(self): await self.session.say("Hello! How can I help?")
    async def on_exit(self): await self.session.say("Goodbye!")

async def start_session(context: JobContext):
    # Create agent and conversation flow
    agent = MyVoiceAgent()
    conversation_flow = ConversationFlow(agent)

    # Create pipeline
    pipeline = CascadingPipeline(
        stt=DeepgramSTT(model="nova-2", language="en"),
        llm=OpenAILLM(model="gpt-4o"),
        tts=ElevenLabsTTS(model="eleven_flash_v2_5"),
        vad=SileroVAD(threshold=0.35),
        turn_detector=TurnDetector(threshold=0.8)
    )

    session = AgentSession(
        agent=agent,
        pipeline=pipeline,
        conversation_flow=conversation_flow
    )

    try:
        await context.connect()
        await session.start()
        # Keep the session running until manually terminated
        await asyncio.Event().wait()
    finally:
        # Clean up resources when done
        await session.close()
        await context.shutdown()

def make_context() -> JobContext:
    room_options = RoomOptions(
    #  room_id="YOUR_MEETING_ID",  # Set to join a pre-created room; omit to auto-create
        name="VideoSDK Cascaded Agent",
        playground=True
    )

    return JobContext(room_options=room_options)

if __name__ == "__main__":
    job = WorkerJob(entrypoint=start_session, jobctx=make_context)
    job.start()
```

Now, let's break down the code step by step.

### Step 4.1: Generating a VideoSDK Meeting ID
To test your agent, you'll need a VideoSDK meeting room. You can generate a meeting ID using the VideoSDK API or Dashboard. Here's a quick way using `curl`:

```bash
curl -X POST \
  -H "Authorization: YOUR_VIDEOSDK_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{}' \
  https://api.videosdk.live/v2/rooms
```

The response will include a `roomId` you can use for testing. For playground testing, you can omit the `room_id` and let the agent create a room automatically.

### Step 4.2: Creating the Custom Agent Class (MyVoiceAgent)
The `MyVoiceAgent` class defines your agent's persona and greeting/exit logic.

```python
agent_instructions = "You are an AI Voice Agent specializing in customer support. Your persona is that of a friendly, patient, and knowledgeable customer service representative. ..."

class MyVoiceAgent(Agent):
    def __init__(self):
        super().__init__(instructions=agent_instructions)
    async def on_enter(self):
        await self.session.say("Hello! How can I help?")
    async def on_exit(self):
        await self.session.say("Goodbye!")
```

- The `instructions` string sets the agent's capabilities, tone, and boundaries.
- `on_enter` and `on_exit` handle greetings and farewells.

### Step 4.3: Defining the Core Pipeline (CascadingPipeline and plugins)
The pipeline connects all the voice agent's core components:

```python
pipeline = CascadingPipeline(
    stt=DeepgramSTT(model="nova-2", language="en"),
    llm=OpenAILLM(model="gpt-4o"),
    tts=ElevenLabsTTS(model="eleven_flash_v2_5"),
    vad=SileroVAD(threshold=0.35),
    turn_detector=TurnDetector(threshold=0.8)
)
```

- **STT:** Deepgram's Nova-2 model for fast, accurate transcription. To learn more about integrating this, see the [Deepgram STT Plugin for voice agent](https://docs.videosdk.live/ai_agents/plugins/stt/deepgram).
- **LLM:** OpenAI's GPT-4o for advanced conversational intelligence. For setup details, refer to the [OpenAI LLM Plugin for voice agent](https://docs.videosdk.live/ai_agents/plugins/llm/openai).
- **TTS:** ElevenLabs for natural, expressive speech. Explore the [ElevenLabs TTS Plugin for voice agent](https://docs.videosdk.live/ai_agents/plugins/tts/eleven-labs) for more options.
- **VAD:** SileroVAD detects when the user is speaking. Check out the [Silero Voice Activity Detection](https://docs.videosdk.live/ai_agents/plugins/silero-vad) documentation for configuration tips.
- **TurnDetector:** Ensures the agent responds at the right time. Learn more about the [Turn detector for AI voice Agents](https://docs.videosdk.live/ai_agents/plugins/turn-detector).

### Step 4.4: Managing the Session and Startup Logic
The session orchestrates the agent's lifecycle and connects all components.

```python
async def start_session(context: JobContext):
    agent = MyVoiceAgent()
    conversation_flow = ConversationFlow(agent)
    pipeline = CascadingPipeline(
        stt=DeepgramSTT(model="nova-2", language="en"),
        llm=OpenAILLM(model="gpt-4o"),
        tts=ElevenLabsTTS(model="eleven_flash_v2_5"),
        vad=SileroVAD(threshold=0.35),
        turn_detector=TurnDetector(threshold=0.8)
    )
    session = AgentSession(
        agent=agent,
        pipeline=pipeline,
        conversation_flow=conversation_flow
    )
    try:
        await context.connect()
        await session.start()
        await asyncio.Event().wait()
    finally:
        await session.close()
        await context.shutdown()

def make_context() -> JobContext:
    room_options = RoomOptions(
        name="VideoSDK Cascaded Agent",
        playground=True
    )
    return JobContext(room_options=room_options)

if __name__ == "__main__":
    job = WorkerJob(entrypoint=start_session, jobctx=make_context)
    job.start()
```

- `start_session` sets up the agent, pipeline, and conversation flow, then starts the session.
- `make_context` configures the meeting room (with `playground=True` for easy web testing).
- The main block runs the agent as a worker job.

For more on session management, see [AI voice Agent Sessions](https://docs.videosdk.live/ai_agents/core-components/agent-session).

## 5. Running and Testing the Agent

### Step 5.1: Running the Python Script
1. Make sure your `.env` file is set up with all required API keys.
2. Run the agent script:

```bash
python main.py
```

3. In the console output, you'll see a `Playground URL`. Copy this link.

### Step 5.2: Interacting with the Agent in the Playground
- Open the Playground URL in your browser.
- Join the meeting as a participant.
- Speak into your microphone and interact with the AI Voice Agent.
- To stop the agent, press `Ctrl+C` in your terminal. The agent will gracefully shut down and release resources.

You can also experiment with your agent in the [AI Agent playground](https://docs.videosdk.live/ai_agents/playground), which provides a web-based environment for real-time testing and iteration.

## 6. Advanced Features and Customizations

### Extending Functionality with Custom Tools
You can add custom function tools to the agent for actions like checking order status, fetching account info, or integrating with CRMs. Implement these as Python functions and register them with your agent.

### Exploring Other Plugins
VideoSDK supports a wide range of plugins for STT, TTS, and LLM. Try alternatives like Cartesia STT, Google Gemini LLM, or Deepgram TTS for different needs.

## 7. Troubleshooting Common Issues

### API Key and Authentication Errors
- Double-check your `.env` file and ensure all API keys are valid and active.
- Make sure your VideoSDK account has sufficient credits and permissions.

### Audio Input/Output Problems
- Ensure your microphone and speakers are working.
- Try a different browser or device if you encounter issues in the playground.

### Dependency and Version Conflicts
- Use a fresh Python virtual environment.
- Run `pip list` to check for conflicting package versions.

## 8. Conclusion

You've built a fully functional AI Voice Agent for customer support, ready for real-world testing. This agent can handle common queries, escalate complex issues, and be extended with custom tools.

For further learning, explore VideoSDK's documentation, experiment with different plugins, and integrate your agent with real backend systems to unlock even more powerful customer support automation.

How to Build a Voice AI Agent with VideoSDK

Best Practices for Conversational AI Agents

Integrating LLMs with Real-Time Voice Applications

Learn to build a production-ready AI voice agent for customer support using Python and VideoSDK. Step-by-step tutorial with code and testing.

Build an AI Voice Agent for Customer Support (Python Guide)

AI Voice Agent for Support

Step-by-step Python tutorial to create a professional AI Voice Acting Agent with VideoSDK. Includes code, setup, and testing instructions.

## 1. Introduction to AI Voice Agents in voice acting agent

### What is an AI [Voice Agent](https://docs.videosdk.live/ai_agents/voice-agent-quick-start)?

An AI [Voice Agent](https://docs.videosdk.live/ai_agents/voice-agent-quick-start) is a software system that can interact with users using natural spoken language. It leverages speech recognition (STT), natural language processing (LLM), and speech synthesis (TTS) to hold real-time conversations, answer questions, and provide guidance.

### Why are they important for the voice acting agent industry?

In the voice acting industry, aspiring and professional voice actors often seek career advice, audition tips, and industry insights. A specialized AI Voice Acting Agent can provide immediate, friendly, and expert guidance—acting as a virtual talent agent available 24/7. This democratizes access to industry knowledge, helps users prepare for auditions, and connects them with valuable resources.

### Core Components of a [Voice Agent](https://docs.videosdk.live/ai_agents/plugins/stt/deepgram)
- **Speech-to-Text (STT):** Converts spoken words into text.
- **Natural Language Model (LLM):** Understands and generates human-like responses.
- **Text-to-Speech (TTS):** Converts responses back into natural-sounding speech.
- **Voice [Activity Detection](https://docs.videosdk.live/ai_agents/plugins/silero-vad) (VAD):** Detects when the user is speaking.
- **Turn Detection:** Determines when to take turns in conversation.

### What You'll Build in This Tutorial

In this guide, you'll build a fully functional AI Voice Acting Agent using the VideoSDK AI Agents framework. The agent will provide career advice, audition tips, and industry information for voice actors. You'll learn how to set up the environment, implement the agent, and test it live.

## 2. Architecture and Core Concepts

### High-Level Architecture Overview

Our Voice Acting Agent is built on a modular pipeline. User audio is processed by Voice Activity Detection and Turn Detection, transcribed by STT, interpreted by a Large Language Model, and spoken back using TTS. All components are orchestrated by the VideoSDK framework. For a deeper dive into the [AI voice Agent core components overview](https://docs.videosdk.live/ai_agents/core-components/overview), refer to the official documentation to understand how each part fits together.



### Understanding Key Concepts in the VideoSDK Framework

- **Agent:** The core logic that defines how your AI interacts with users. You subclass the Agent class to define custom behaviors.
- **CascadingPipeline:** Orchestrates the flow of audio and text through VAD, Turn Detection, STT, LLM, and TTS plugins. Learn more about the [Cascading pipeline in AI voice Agents](https://docs.videosdk.live/ai_agents/core-components/cascading-pipeline) to see how this enables seamless communication between components.
- **VAD & TurnDetector:** SileroVAD detects when the user is speaking; TurnDetector determines conversational turns, ensuring natural back-and-forth.

## 3. Setting Up the Development Environment

### Prerequisites
- Python 3.11 or newer
- A VideoSDK account (for API keys and dashboard access)

### Step 1: Create a Virtual Environment

Open your terminal and run:

```bash
python3 -m venv venv
source venv/bin/activate  # On Windows: venv\Scripts\activate
```

### Step 2: Install Required Packages

Install the VideoSDK AI Agents framework and plugins:

```bash
pip install videosdk-agents videosdk-plugins-silero videosdk-plugins-turn-detector videosdk-plugins-deepgram videosdk-plugins-openai videosdk-plugins-elevenlabs
```

### Step 3: Configure API Keys in a .env file

Create a `.env` file in your project directory and add your VideoSDK and plugin API keys:

```env
VIDEOSDK_API_KEY=your_videosdk_api_key
DEEPGRAM_API_KEY=your_deepgram_api_key
OPENAI_API_KEY=your_openai_api_key
ELEVENLABS_API_KEY=your_elevenlabs_api_key
```

You can find your VideoSDK API key in the dashboard after signing up. Obtain plugin keys from their respective providers.

## 4. Building the AI Voice Agent: A Step-by-Step Guide

### Full Working Code Example

Below is the complete, runnable Python script for your Voice Acting Agent. We'll break down each part in the following sections.

```python
import asyncio, os
from videosdk.agents import Agent, AgentSession, CascadingPipeline, JobContext, RoomOptions, WorkerJob, ConversationFlow
from videosdk.plugins.silero import SileroVAD
from videosdk.plugins.turn_detector import TurnDetector, pre_download_model
from videosdk.plugins.deepgram import DeepgramSTT
from videosdk.plugins.openai import OpenAILLM
from videosdk.plugins.elevenlabs import ElevenLabsTTS
from typing import AsyncIterator

# Pre-downloading the Turn Detector model
pre_download_model()

agent_instructions = "You are a professional and knowledgeable Voice Acting Agent. Your persona is friendly, supportive, and resourceful, acting as a virtual talent agent specializing in voice acting careers. 

Capabilities:
- Provide information about the voice acting industry, including career paths, audition tips, and required skills.
- Offer guidance on building a voice acting portfolio, finding auditions, and connecting with casting directors or agencies.
- Answer questions about voice acting techniques, training resources, and industry trends.
- Suggest reputable online platforms, workshops, and communities for aspiring and professional voice actors.
- Assist users in preparing for auditions by offering script reading tips and vocal warm-up exercises.

Constraints and Limitations:
- You do not represent any real-world agency or guarantee job placements.
- Do not provide legal, financial, or contractual advice; always recommend consulting a qualified professional for such matters.
- Avoid sharing personal opinions or endorsements of specific individuals or companies.
- Do not collect or store any personal information from users.
- Always encourage users to verify information independently and exercise caution when pursuing opportunities."

class MyVoiceAgent(Agent):
    def __init__(self):
        super().__init__(instructions=agent_instructions)
    async def on_enter(self): await self.session.say("Hello! How can I help?")
    async def on_exit(self): await self.session.say("Goodbye!")

async def start_session(context: JobContext):
    # Create agent and conversation flow
    agent = MyVoiceAgent()
    conversation_flow = ConversationFlow(agent)

    # Create pipeline
    pipeline = CascadingPipeline(
        stt=DeepgramSTT(model="nova-2", language="en"),
        llm=OpenAILLM(model="gpt-4o"),
        tts=ElevenLabsTTS(model="eleven_flash_v2_5"),
        vad=SileroVAD(threshold=0.35),
        turn_detector=TurnDetector(threshold=0.8)
    )

    session = AgentSession(
        agent=agent,
        pipeline=pipeline,
        conversation_flow=conversation_flow
    )

    try:
        await context.connect()
        await session.start()
        # Keep the session running until manually terminated
        await asyncio.Event().wait()
    finally:
        # Clean up resources when done
        await session.close()
        await context.shutdown()

def make_context() -> JobContext:
    room_options = RoomOptions(
    #  room_id="YOUR_MEETING_ID",  # Set to join a pre-created room; omit to auto-create
        name="VideoSDK Cascaded Agent",
        playground=True
    )

    return JobContext(room_options=room_options)

if __name__ == "__main__":
    job = WorkerJob(entrypoint=start_session, jobctx=make_context)
    job.start()
```

### Step 4.1: Generating a VideoSDK Meeting ID

To test your agent, you'll need a meeting ID. You can create one via the VideoSDK API.

Run this `curl` command (replace `YOUR_API_KEY`):

```bash
curl -X POST \
  -H "Authorization: YOUR_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{}' \
  https://api.videosdk.live/v2/rooms
```

The response will include a `roomId`. You can use this in your `RoomOptions` if you want to join a pre-created room. If omitted, a new room is auto-created.

### Step 4.2: Creating the Custom Agent Class (MyVoiceAgent)

The heart of your agent is the custom class that defines its persona and behavior.

```python
agent_instructions = "You are a professional and knowledgeable Voice Acting Agent. Your persona is friendly, supportive, and resourceful, acting as a virtual talent agent specializing in voice acting careers. 

Capabilities:
- Provide information about the voice acting industry, including career paths, audition tips, and required skills.
- Offer guidance on building a voice acting portfolio, finding auditions, and connecting with casting directors or agencies.
- Answer questions about voice acting techniques, training resources, and industry trends.
- Suggest reputable online platforms, workshops, and communities for aspiring and professional voice actors.
- Assist users in preparing for auditions by offering script reading tips and vocal warm-up exercises.

Constraints and Limitations:
- You do not represent any real-world agency or guarantee job placements.
- Do not provide legal, financial, or contractual advice; always recommend consulting a qualified professional for such matters.
- Avoid sharing personal opinions or endorsements of specific individuals or companies.
- Do not collect or store any personal information from users.
- Always encourage users to verify information independently and exercise caution when pursuing opportunities."

class MyVoiceAgent(Agent):
    def __init__(self):
        super().__init__(instructions=agent_instructions)
    async def on_enter(self):
        await self.session.say("Hello! How can I help?")
    async def on_exit(self):
        await self.session.say("Goodbye!")
```

This class sets the agent's persona and welcome/goodbye messages.

### Step 4.3: Defining the Core Pipeline (CascadingPipeline)

The pipeline connects all the plugins: STT, LLM, TTS, VAD, and Turn Detector.

```python
pipeline = CascadingPipeline(
    stt=DeepgramSTT(model="nova-2", language="en"),
    llm=OpenAILLM(model="gpt-4o"),
    tts=ElevenLabsTTS(model="eleven_flash_v2_5"),
    vad=SileroVAD(threshold=0.35),
    turn_detector=TurnDetector(threshold=0.8)
)
```

- **DeepgramSTT:** High-quality, cost-effective speech recognition.
- **OpenAILLM:** Powerful GPT-4o language model for smart responses.
- **ElevenLabsTTS:** Natural and expressive voice synthesis. For more advanced voice synthesis, check out the [ElevenLabs TTS Plugin for voice agent](https://docs.videosdk.live/ai_agents/plugins/tts/eleven-labs) to explore additional configuration options and voices.
- **SileroVAD:** Reliable voice activity detection.
- **TurnDetector:** Ensures smooth conversational turns. To better understand how turn-taking is managed, see the [Turn detector for AI voice Agents](https://docs.videosdk.live/ai_agents/plugins/turn-detector) documentation.

### Step 4.4: Managing the Session and Startup Logic

Sessions manage the lifecycle of your agent and handle connections.

```python
async def start_session(context: JobContext):
    agent = MyVoiceAgent()
    conversation_flow = ConversationFlow(agent)
    pipeline = CascadingPipeline(
        stt=DeepgramSTT(model="nova-2", language="en"),
        llm=OpenAILLM(model="gpt-4o"),
        tts=ElevenLabsTTS(model="eleven_flash_v2_5"),
        vad=SileroVAD(threshold=0.35),
        turn_detector=TurnDetector(threshold=0.8)
    )
    session = AgentSession(
        agent=agent,
        pipeline=pipeline,
        conversation_flow=conversation_flow
    )
    try:
        await context.connect()
        await session.start()
        await asyncio.Event().wait()
    finally:
        await session.close()
        await context.shutdown()

def make_context() -> JobContext:
    room_options = RoomOptions(
        name="VideoSDK Cascaded Agent",
        playground=True
    )
    return JobContext(room_options=room_options)

if __name__ == "__main__":
    job = WorkerJob(entrypoint=start_session, jobctx=make_context)
    job.start()
```

- The `make_context` function creates a room with `playground=True` for easy browser testing. You can also experiment with your agent in the [AI Agent playground](https://docs.videosdk.live/ai_agents/playground) for interactive testing and rapid prototyping.
- The main block starts the agent job.

## 5. Running and Testing the Agent

### Step 5.1: Running the Python Script

1. Make sure your `.env` file is set up with all required API keys.
2. Run the script:

```bash
python main.py
```

3. In the console output, look for a line that says `Playground URL:`. This link lets you join the agent session from your browser.

### Step 5.2: Interacting with the Agent in the Playground

- Open the Playground URL in your browser.
- Join the meeting as a participant.
- Speak or type your questions about voice acting.
- The agent will respond in real time using natural-sounding speech.

To stop the agent, press `Ctrl+C` in your terminal. This triggers a graceful shutdown, ensuring all resources are released.

## 6. Advanced Features and Customizations

### Extending Functionality with Custom Tools

You can add custom tools or actions to your agent by subclassing and extending the `Agent` class. For example, you could add a portfolio review tool, integrate with audition databases, or trigger notifications.

### Exploring Other Plugins

VideoSDK supports a variety of plugins for STT, TTS, and LLM. You can experiment with alternatives like Cartesia for STT, Deepgram for TTS, or Google Gemini for LLM to optimize for cost, quality, or language support.

## 7. Troubleshooting Common Issues

### API Key and Authentication Errors
- Double-check all API keys in your `.env` file.
- Make sure your VideoSDK account is active.

### Audio Input/Output Problems
- Ensure your microphone is enabled in the browser.
- Test with different browsers if you encounter issues.

### Dependency and Version Conflicts
- Use a clean virtual environment.
- Run `pip list` to check for any conflicting package versions.

## 8. Conclusion

In this tutorial, you built a professional AI Voice Acting Agent using the VideoSDK AI Agents framework. You learned how to set up the environment, implement a custom agent, and test it in the browser.

Explore more advanced features, try different plugins, and consider deploying your agent to help aspiring voice actors worldwide. The possibilities for customization and integration are vast—happy building!

Integrating ElevenLabs TTS with VideoSDK AI Agents

Learn to build a professional AI Voice Acting Agent using VideoSDK. Complete Python guide, code walkthrough, and testing instructions included.

Build a Voice Acting Agent AI with VideoSDK: Step-by-Step Guide

Build a Voice Acting Agent AI

Comprehensive tutorial to build, test, and customize an AI Voice Agent for BPO using VideoSDK and Python. Includes full code, setup, and troubleshooting.

## Introduction to AI Voice Agents in ai voice agent for bpo

### What is an AI Voice Agent?
AI Voice Agents are intelligent software systems that can understand, process, and respond to human speech in real time. They leverage automatic speech recognition (ASR), natural language processing (NLP), and text-to-speech (TTS) technologies to interact with users over the phone or other voice channels.

### Why are they important for the ai voice agent for bpo industry?
Business Process Outsourcing (BPO) companies handle large volumes of customer interactions. AI Voice Agents help BPOs scale their operations, reduce costs, and provide 24/7 support. They can handle routine queries, process transactions, and escalate complex issues to human agents, all while maintaining a consistent and professional tone.

### Core Components of a Voice Agent
- **Speech-to-Text (STT):** Converts spoken language into text.
- **Natural Language Understanding (NLU/LLM):** Interprets the meaning of the text.
- **Text-to-Speech (TTS):** Converts the agent's response back into natural-sounding speech.
- **Voice Activity Detection (VAD) & Turn Detection:** Determines when the user is speaking and when it's the agent's turn to respond.

If you're new to building these systems, the [Voice Agent Quick Start Guide](https://docs.videosdk.live/ai_agents/voice-agent-quick-start) provides a step-by-step introduction to get you started quickly.

### What You'll Build in This Tutorial
In this tutorial, you'll build a fully functional AI Voice Agent tailored for BPO use cases using the VideoSDK AI Agents framework. You'll learn how to set up your environment, implement the agent, and test it in a real-time voice playground.

## Architecture and Core Concepts

### High-Level Architecture Overview
The AI Voice Agent processes audio in a pipeline: user speech is captured, transcribed to text, interpreted by a language model, and then synthesized back to speech for the response. Each component is modular and can be swapped for different plugins. For a detailed explanation of these elements, see the [AI voice Agent core components overview](https://docs.videosdk.live/ai_agents/core-components/overview).

#### Data Flow Sequence
```mermaid
sequenceDiagram
    participant User
    participant AgentSession
    participant CascadingPipeline
    participant DeepgramSTT
    participant OpenAILLM
    participant ElevenLabsTTS
    participant VAD
    participant TurnDetector
    User->>AgentSession: Speaks
    AgentSession->>VAD: Detects speech activity
    VAD->>TurnDetector: Detects turn end
    TurnDetector->>DeepgramSTT: Sends audio for transcription
    DeepgramSTT->>OpenAILLM: Sends transcript
    OpenAILLM->>ElevenLabsTTS: Gets response
    ElevenLabsTTS->>AgentSession: Plays audio response
    AgentSession->>User: Responds
```

The [Cascading pipeline in AI voice Agents](https://docs.videosdk.live/ai_agents/core-components/cascading-pipeline) is central to this process, ensuring seamless integration and flow between each plugin and component.

### Understanding Key Concepts in the VideoSDK Framework
- **Agent:** The core class that defines the agent's persona and behavior.
- **CascadingPipeline:** Manages the flow of audio through STT, LLM, TTS, VAD, and turn detection.
- **VAD & TurnDetector:** Ensure the agent listens and responds at the right moments, creating a natural conversation flow.

To learn more about managing real-time agent interactions, refer to the [AI voice Agent Sessions](https://docs.videosdk.live/ai_agents/core-components/agent-session) documentation.

## Setting Up the Development Environment

### Prerequisites
- **Python 3.11+** (ensure compatibility with VideoSDK agents)
- **A VideoSDK Account:** Sign up and access your dashboard to obtain API credentials.

### Step 1: Create a Virtual Environment
It's best practice to isolate your project dependencies.
```bash
python3.11 -m venv venv
source venv/bin/activate  # On Windows: venv\Scripts\activate
```

### Step 2: Install Required Packages
Install the VideoSDK AI Agents framework and required plugins.
```bash
pip install videosdk-agents videosdk-plugins-silero videosdk-plugins-turn-detector videosdk-plugins-deepgram videosdk-plugins-openai videosdk-plugins-elevenlabs
```

### Step 3: Configure API Keys in a .env File
Create a `.env` file in your project directory and add your API keys.
```env
VIDEOSDK_API_KEY=your_videosdk_api_key
DEEPGRAM_API_KEY=your_deepgram_api_key
OPENAI_API_KEY=your_openai_api_key
ELEVENLABS_API_KEY=your_elevenlabs_api_key
```

## Building the AI Voice Agent: A Step-by-Step Guide

Let's look at the complete, runnable code for the AI Voice Agent. We'll then break down each section to understand how it works.

```python
import asyncio, os
from videosdk.agents import Agent, AgentSession, CascadingPipeline, JobContext, RoomOptions, WorkerJob, ConversationFlow
from videosdk.plugins.silero import SileroVAD
from videosdk.plugins.turn_detector import TurnDetector, pre_download_model
from videosdk.plugins.deepgram import DeepgramSTT
from videosdk.plugins.openai import OpenAILLM
from videosdk.plugins.elevenlabs import ElevenLabsTTS
from typing import AsyncIterator

# Pre-downloading the Turn Detector model
pre_download_model()

agent_instructions = "You are an efficient and professional AI Voice Agent designed specifically for Business Process Outsourcing (BPO) environments. Your persona is that of a courteous, knowledgeable, and patient customer service representative. Your primary capabilities include: answering customer queries related to products or services, handling basic troubleshooting, processing simple transactions, escalating complex issues to human agents, and providing information about company policies and procedures. You must always maintain a polite and empathetic tone, ensure customer data privacy, and strictly adhere to provided scripts and compliance guidelines. You are not authorized to make decisions outside predefined protocols, provide personal opinions, or handle sensitive financial or legal matters. Always inform the customer when you are escalating their issue to a human agent. If you are unsure or unable to assist, politely suggest that a human representative will follow up. Never collect or store sensitive personal information beyond what is explicitly permitted by company policy."

class MyVoiceAgent(Agent):
    def __init__(self):
        super().__init__(instructions=agent_instructions)
    async def on_enter(self): await self.session.say("Hello! How can I help?")
    async def on_exit(self): await self.session.say("Goodbye!")

async def start_session(context: JobContext):
    # Create agent and conversation flow
    agent = MyVoiceAgent()
    conversation_flow = ConversationFlow(agent)

    # Create pipeline
    pipeline = CascadingPipeline(
        stt=DeepgramSTT(model="nova-2", language="en"),
        llm=OpenAILLM(model="gpt-4o"),
        tts=ElevenLabsTTS(model="eleven_flash_v2_5"),
        vad=SileroVAD(threshold=0.35),
        turn_detector=TurnDetector(threshold=0.8)
    )

    session = AgentSession(
        agent=agent,
        pipeline=pipeline,
        conversation_flow=conversation_flow
    )

    try:
        await context.connect()
        await session.start()
        # Keep the session running until manually terminated
        await asyncio.Event().wait()
    finally:
        # Clean up resources when done
        await session.close()
        await context.shutdown()

def make_context() -> JobContext:
    room_options = RoomOptions(
    #  room_id="YOUR_MEETING_ID",  # Set to join a pre-created room; omit to auto-create
        name="VideoSDK Cascaded Agent",
        playground=True
    )

    return JobContext(room_options=room_options)

if __name__ == "__main__":
    job = WorkerJob(entrypoint=start_session, jobctx=make_context)
    job.start()
```

Now, let's break down each part of the code and explain its function.

### Step 4.1: Generating a VideoSDK Meeting ID
Before you can run your agent, you'll need a meeting room where the agent can interact with users. You can generate a meeting ID using the VideoSDK API.

```bash
curl -X POST \
  -H "Authorization: YOUR_VIDEOSDK_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{"region":"sg001"}' \
  https://api.videosdk.live/v2/rooms
```

The response will include a `roomId` you can use. For testing, you can let the agent auto-create the room by omitting the `room_id` in `RoomOptions`.

### Step 4.2: Creating the Custom Agent Class (MyVoiceAgent)
The agent's persona and behavior are defined in a custom class that inherits from `Agent`.

```python
agent_instructions = "You are an efficient and professional AI Voice Agent designed specifically for Business Process Outsourcing (BPO) environments. ..."

class MyVoiceAgent(Agent):
    def __init__(self):
        super().__init__(instructions=agent_instructions)
    async def on_enter(self):
        await self.session.say("Hello! How can I help?")
    async def on_exit(self):
        await self.session.say("Goodbye!")
```

- The `agent_instructions` string guides the LLM on how the agent should behave.
- `on_enter` and `on_exit` provide greetings and farewells when the session starts and ends.

### Step 4.3: Defining the Core Pipeline (CascadingPipeline and plugins)
The pipeline orchestrates the flow of audio and text through the agent.

```python
pipeline = CascadingPipeline(
    stt=DeepgramSTT(model="nova-2", language="en"),
    llm=OpenAILLM(model="gpt-4o"),
    tts=ElevenLabsTTS(model="eleven_flash_v2_5"),
    vad=SileroVAD(threshold=0.35),
    turn_detector=TurnDetector(threshold=0.8)
)
```

- **STT:** Deepgram's "nova-2" model for English transcription. For more details on integrating this, see the [Deepgram STT Plugin for voice agent](https://docs.videosdk.live/ai_agents/plugins/stt/deepgram).
- **LLM:** OpenAI's GPT-4o for natural language understanding and response generation. Learn more about configuration in the [OpenAI LLM Plugin for voice agent](https://docs.videosdk.live/ai_agents/plugins/llm/openai).
- **TTS:** ElevenLabs for high-quality voice synthesis. See the [ElevenLabs TTS Plugin for voice agent](https://docs.videosdk.live/ai_agents/plugins/tts/eleven-labs) for setup instructions.
- **VAD:** SileroVAD detects when the user is speaking. For implementation details, visit [Silero Voice Activity Detection](https://docs.videosdk.live/ai_agents/plugins/silero-vad).
- **TurnDetector:** Determines when the user's turn ends, so the agent can respond. Read more in the [Turn detector for AI voice Agents](https://docs.videosdk.live/ai_agents/plugins/turn-detector) documentation.

### Step 4.4: Managing the Session and Startup Logic
The session brings together the agent, pipeline, and conversation flow, and manages their lifecycle.

```python
async def start_session(context: JobContext):
    agent = MyVoiceAgent()
    conversation_flow = ConversationFlow(agent)
    pipeline = CascadingPipeline(...)
    session = AgentSession(agent=agent, pipeline=pipeline, conversation_flow=conversation_flow)
    try:
        await context.connect()
        await session.start()
        await asyncio.Event().wait()
    finally:
        await session.close()
        await context.shutdown()

def make_context() -> JobContext:
    room_options = RoomOptions(
        name="VideoSDK Cascaded Agent",
        playground=True
    )
    return JobContext(room_options=room_options)

if __name__ == "__main__":
    job = WorkerJob(entrypoint=start_session, jobctx=make_context)
    job.start()
```

- `start_session` initializes and starts the session, keeping it alive until manually stopped.
- `make_context` configures the meeting room and enables the playground for browser-based testing.
- The `__main__` block launches the agent.

If you'd like to experiment with your agent in a browser-based environment, the [AI Agent playground](https://docs.videosdk.live/ai_agents/playground) provides an interactive space for real-time testing and iteration.

## Running and Testing the Agent

### Step 5.1: Running the Python Script
1. Ensure your `.env` file is set up with all required API keys.
2. Run the agent script:
```bash
python main.py
```
3. The console will display a Playground URL.

### Step 5.2: Interacting with the Agent in the Playground
- Open the Playground link in your browser.
- Join as a participant; you can now speak with your AI Voice Agent in real time.
- The agent will greet you and respond to your queries.
- To stop the agent, press `Ctrl+C` in your terminal. This gracefully shuts down the session and cleans up resources.

## Advanced Features and Customizations

### Extending Functionality with Custom Tools
- You can add custom function tools to the agent for handling specific BPO workflows, such as ticket creation or CRM integration.
- Implement a function and register it with your agent to enable advanced automation.

### Exploring Other Plugins
- **STT:** Try Cartesia for best accuracy, or Rime for lower cost.
- **TTS:** Deepgram offers a cost-effective alternative to ElevenLabs.
- **LLM:** Experiment with Google Gemini or other supported models.

## Troubleshooting Common Issues

### API Key and Authentication Errors
- Double-check your `.env` file and ensure all keys are correct and active.
- If you see authentication errors, regenerate your API keys from the dashboard.

### Audio Input/Output Problems
- Ensure your microphone and speakers are working and permitted in your browser.
- Test in different browsers if you encounter issues.

### Dependency and Version Conflicts
- Use a fresh virtual environment to avoid conflicts.
- Check package versions if you encounter import errors.

## Conclusion

Congratulations! You've built a fully functional AI Voice Agent for BPO using the VideoSDK AI Agents framework. You learned how to set up the environment, implement the agent with best-in-class plugins, and test it live.

For next steps, explore advanced function tools, integrate with your BPO systems, and experiment with different plugins to optimize performance and cost. The VideoSDK framework is highly extensible, enabling you to build production-ready AI voice solutions for any BPO workflow.

How to Build a Voicebot Using VideoSDK AI Agents

Choosing the Right STT and TTS Plugins for Your AI Agent

Step-by-step guide to building an AI Voice Agent for BPO using Python and VideoSDK. Includes code, setup, testing, and troubleshooting.

Build an AI Voice Agent for BPO with VideoSDK

AI Voice Agent for BPO

Learn to build a fully functional AI voice agent for the automotive industry using VideoSDK. This hands-on tutorial covers setup, code, and testing.

## 1. Introduction to AI Voice Agents in Automotive

### What is an AI Voice Agent?
An AI Voice Agent is a software system that can understand spoken language, process it using artificial intelligence, and respond with synthesized speech. In practice, this means users can interact with technology using natural conversation, making complex tasks hands-free and intuitive.

### Why are AI Voice Agents Important for the Automotive Industry?
In the automotive world, voice agents are revolutionizing the driving experience. They enable drivers and passengers to: 
- Control navigation and infotainment systems hands-free
- Ask about vehicle features or maintenance
- Troubleshoot common issues
- Get real-time assistance while driving
- Schedule service appointments or request information about connected car technologies

These capabilities not only improve convenience but also enhance safety by reducing distractions.

### Core Components of a Voice Agent
A robust automotive voice agent relies on several key technologies:
- **Speech-to-Text (STT):** Converts spoken words into text.
- **Large Language Model (LLM):** Processes the text and generates intelligent responses.
- **Text-to-Speech (TTS):** Converts the agent's response back into natural-sounding speech.

If you're new to building voice agents, the [Voice Agent Quick Start Guide](https://docs.videosdk.live/ai_agents/voice-agent-quick-start) is a great resource to help you get started quickly.

### What You'll Build in This Tutorial
In this tutorial, you'll create a fully working AI voice agent tailored for automotive use cases. The agent will answer questions about vehicle features, maintenance, navigation, and more—all using the Videosdk AI Agents framework.

## 2. Architecture and Core Concepts

### High-Level Architecture Overview
Let's visualize how the agent processes a user's request from start to finish:

```mermaid
sequenceDiagram
    participant User
    participant Agent
    participant VAD
    participant TurnDetector
    participant STT
    participant LLM
    participant TTS
    User->>VAD: Speaks (audio input)
    VAD->>TurnDetector: Detects end of user speech
    TurnDetector->>STT: Sends audio for transcription
    STT->>LLM: Transcribed text
    LLM->>Agent: Generates response
    Agent->>TTS: Sends response text
    TTS->>User: Plays synthesized speech
```

This pipeline ensures the agent listens, understands, and responds in real time. For a deeper dive into the essential building blocks, check out the [AI voice Agent core components overview](https://docs.videosdk.live/ai_agents/core-components/overview).

### Understanding Key Concepts in the VideoSDK Framework
- **Agent:** The main class that defines your voice assistant's personality and logic.
- **CascadingPipeline:** Orchestrates the flow of audio and text through STT, LLM, TTS, and other plugins.
- **VAD (Voice Activity Detection):** Listens for when the user starts and stops speaking.
- **TurnDetector:** Determines when it's the agent's turn to speak, preventing interruptions.

These components work together to create a seamless conversational experience. To see how these elements interact, explore the [Cascading pipeline in AI voice Agents](https://docs.videosdk.live/ai_agents/core-components/cascading-pipeline) documentation.

## 3. Setting Up the Development Environment

### Prerequisites
- **Python 3.11+** (ensure you have the latest minor version)
- **VideoSDK Account:** Sign up at the VideoSDK dashboard.

### Step 1: Create a Virtual Environment
It's best to isolate your project dependencies.

```bash
python3.11 -m venv venv
source venv/bin/activate  # On Windows: venv\\Scripts\\activate
```

### Step 2: Install Required Packages
Install the Videosdk AI Agents framework and plugin dependencies:

```bash
pip install videosdk-agents videosdk-plugins-deepgram videosdk-plugins-openai videosdk-plugins-elevenlabs videosdk-plugins-silero videosdk-plugins-turn-detector
```

If you want to learn more about integrating specific plugins, such as the [Deepgram STT Plugin for voice agent](https://docs.videosdk.live/ai_agents/plugins/stt/deepgram), the [OpenAI LLM Plugin for voice agent](https://docs.videosdk.live/ai_agents/plugins/llm/openai), or the [ElevenLabs TTS Plugin for voice agent](https://docs.videosdk.live/ai_agents/plugins/tts/eleven-labs), refer to their respective documentation pages.

### Step 3: Configure API Keys in a `.env` File
Create a `.env` file in your project directory and add your API keys:

```env
VIDEOSDK_API_KEY=your_videosdk_api_key
DEEPGRAM_API_KEY=your_deepgram_api_key
OPENAI_API_KEY=your_openai_api_key
ELEVENLABS_API_KEY=your_elevenlabs_api_key
```

You can find these keys in the respective service dashboards.

## 4. Building the AI Voice Agent: A Step-by-Step Guide

Let's dive into building your automotive AI voice agent. Below is the complete, runnable code. We'll break down each part in the following sections.

```python
import asyncio, os
from videosdk.agents import Agent, AgentSession, CascadingPipeline, JobContext, RoomOptions, WorkerJob, ConversationFlow
from videosdk.plugins.silero import SileroVAD
from videosdk.plugins.turn_detector import TurnDetector, pre_download_model
from videosdk.plugins.deepgram import DeepgramSTT
from videosdk.plugins.openai import OpenAILLM
from videosdk.plugins.elevenlabs import ElevenLabsTTS
from typing import AsyncIterator

# Pre-downloading the Turn Detector model
pre_download_model()

agent_instructions = "You are an AI Voice Agent specialized for the automotive industry. Your persona is that of a knowledgeable, friendly, and efficient automotive assistant. Your primary capabilities include: answering questions about vehicle features, maintenance schedules, troubleshooting common car issues, providing navigation assistance, and helping users schedule service appointments. You can also offer information about connected car technologies, safety features, and infotainment systems. You must always prioritize user safety and privacy, never provide driving advice that could endanger users, and refrain from diagnosing complex mechanical issues—always recommend consulting a certified automotive technician for such cases. Do not store or share any personal or vehicle data. If asked for legal, financial, or emergency advice, politely decline and direct the user to the appropriate professional or emergency service. Always communicate clearly and concisely, ensuring instructions are easy to follow, especially when the user may be driving."

class MyVoiceAgent(Agent):
    def __init__(self):
        super().__init__(instructions=agent_instructions)
    async def on_enter(self): await self.session.say("Hello! How can I help?")
    async def on_exit(self): await self.session.say("Goodbye!")

async def start_session(context: JobContext):
    # Create agent and conversation flow
    agent = MyVoiceAgent()
    conversation_flow = ConversationFlow(agent)

    # Create pipeline
    pipeline = CascadingPipeline(
        stt=DeepgramSTT(model="nova-2", language="en"),
        llm=OpenAILLM(model="gpt-4o"),
        tts=ElevenLabsTTS(model="eleven_flash_v2_5"),
        vad=SileroVAD(threshold=0.35),
        turn_detector=TurnDetector(threshold=0.8)
    )

    session = AgentSession(
        agent=agent,
        pipeline=pipeline,
        conversation_flow=conversation_flow
    )

    try:
        await context.connect()
        await session.start()
        # Keep the session running until manually terminated
        await asyncio.Event().wait()
    finally:
        # Clean up resources when done
        await session.close()
        await context.shutdown()

def make_context() -> JobContext:
    room_options = RoomOptions(
    #  room_id="YOUR_MEETING_ID",  # Set to join a pre-created room; omit to auto-create
        name="VideoSDK Cascaded Agent",
        playground=True
    )

    return JobContext(room_options=room_options)

if __name__ == "__main__":
    job = WorkerJob(entrypoint=start_session, jobctx=make_context)
    job.start()
```

Now, let's break down the code step by step.

### Step 4.1: Generating a VideoSDK Meeting ID
Before running your agent, you'll need a meeting ID. You can generate one using the VideoSDK API:

```bash
curl -X POST \
  -H "Authorization: YOUR_VIDEOSDK_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{}' \
  https://api.videosdk.live/v2/rooms
```

Copy the returned `roomId` and use it in your `RoomOptions` if you want to join a specific meeting. If you omit `room_id`, a new room will be created automatically.

### Step 4.2: Creating the Custom Agent Class
The heart of your voice agent is the custom class that defines its personality and behavior.

```python
agent_instructions = "You are an AI Voice Agent specialized for the automotive industry. Your persona is that of a knowledgeable, friendly, and efficient automotive assistant. ..."

class MyVoiceAgent(Agent):
    def __init__(self):
        super().__init__(instructions=agent_instructions)
    async def on_enter(self):
        await self.session.say("Hello! How can I help?")
    async def on_exit(self):
        await self.session.say("Goodbye!")
```

- The `agent_instructions` string defines the agent's persona, capabilities, and safety constraints.
- `on_enter` and `on_exit` methods allow the agent to greet and say goodbye to users.

For more on how the agent manages dialogue, see the [conversation flow in AI voice Agents](https://docs.videosdk.live/ai_agents/core-components/conversation-flow) documentation.

### Step 4.3: Defining the Core Pipeline
The pipeline determines how audio and text flow through the system.

```python
pipeline = CascadingPipeline(
    stt=DeepgramSTT(model="nova-2", language="en"),
    llm=OpenAILLM(model="gpt-4o"),
    tts=ElevenLabsTTS(model="eleven_flash_v2_5"),
    vad=SileroVAD(threshold=0.35),
    turn_detector=TurnDetector(threshold=0.8)
)
```

- **DeepgramSTT:** Converts speech to text using Deepgram's Nova-2 model.
- **OpenAILLM:** Uses GPT-4o for intelligent, context-aware responses.
- **ElevenLabsTTS:** Produces high-quality, natural-sounding speech.
- **SileroVAD:** Detects when the user is speaking.
- **TurnDetector:** Ensures smooth turn-taking between user and agent.

If you want to understand how voice activity is detected, the [Silero Voice Activity Detection](https://docs.videosdk.live/ai_agents/plugins/silero-vad) plugin documentation provides detailed insights. Similarly, for managing conversational turns, refer to the [Turn detector for AI voice Agents](https://docs.videosdk.live/ai_agents/plugins/turn-detector) guide.

### Step 4.4: Managing the Session and Startup Logic
The session orchestrates the agent's lifecycle, from joining the room to shutting down.

```python
async def start_session(context: JobContext):
    agent = MyVoiceAgent()
    conversation_flow = ConversationFlow(agent)
    pipeline = CascadingPipeline(
        stt=DeepgramSTT(model="nova-2", language="en"),
        llm=OpenAILLM(model="gpt-4o"),
        tts=ElevenLabsTTS(model="eleven_flash_v2_5"),
        vad=SileroVAD(threshold=0.35),
        turn_detector=TurnDetector(threshold=0.8)
    )
    session = AgentSession(agent=agent, pipeline=pipeline, conversation_flow=conversation_flow)
    try:
        await context.connect()
        await session.start()
        await asyncio.Event().wait()
    finally:
        await session.close()
        await context.shutdown()

def make_context() -> JobContext:
    room_options = RoomOptions(
        name="VideoSDK Cascaded Agent",
        playground=True
    )
    return JobContext(room_options=room_options)

if __name__ == "__main__":
    job = WorkerJob(entrypoint=start_session, jobctx=make_context)
    job.start()
```

- `start_session` sets up the agent, pipeline, and conversation flow.
- `make_context` configures the meeting room and enables playground mode for easy testing.
- The main block starts the job.

## 5. Running and Testing the Agent

### Step 5.1: Running the Python Script
1. Ensure your `.env` file is configured with all required API keys.
2. Run your agent script:

```bash
python main.py
```

3. In the console output, look for a line containing a playground link. This link allows you to join the meeting and interact with your agent in a web browser.

You can experiment with your agent in the [AI Agent playground](https://docs.videosdk.live/ai_agents/playground), which provides a convenient web interface for testing and development.

### Step 5.2: Interacting with the Agent in the Playground
- Open the provided playground URL in your browser.
- Join the meeting room.
- Speak or type your questions to the agent. Try asking about vehicle features, maintenance, or navigation.
- The agent will respond with synthesized speech, demonstrating the full conversational loop.

To gracefully shut down the agent, press `Ctrl+C` in your terminal. This ensures all resources are cleaned up properly.

## 6. Advanced Features and Customizations

### Extending Functionality with Custom Tools
You can add custom capabilities to your agent using the `function_tool` decorator. For example, integrate with vehicle APIs to fetch real-time data, or add custom logic for specific automotive scenarios.

### Exploring Other Plugins
The Videosdk AI Agents framework supports a variety of plugins:
- **STT:** Cartesia (best), Deepgram (cost-effective), Rime (low-cost)
- **TTS:** ElevenLabs (best quality), Deepgram (cost-effective)
- **LLM:** OpenAI GPT-4, Google Gemini

Experiment with different combinations to optimize for your use case.

## 7. Troubleshooting Common Issues

### API Key and Authentication Errors
- Double-check your API keys in the `.env` file.
- Ensure your VideoSDK account is active and has sufficient quota.

### Audio Input/Output Problems
- Make sure your microphone and speakers are working.
- Test in the playground to isolate browser or network issues.

### Dependency and Version Conflicts
- Use a fresh virtual environment to avoid package conflicts.
- Verify that all plugins are compatible with your Python version.

## 8. Conclusion

Congratulations! You've built a fully functional AI voice agent for the automotive industry using VideoSDK. This agent can answer questions, assist with navigation, and provide valuable information—all hands-free.

To take your project further, consider integrating with real-time vehicle APIs, adding multi-language support, or deploying your agent in production vehicles.

Happy coding!

Integrating STT and TTS Plugins in VideoSDK Agents

Best Practices for Testing AI Voice Agents

Step-by-step guide to creating an automotive AI voice agent using VideoSDK. Includes code, setup, and testing instructions for developers.

Build an AI Voice Agent for Automotive with VideoSDK

AI Voice Agent for Automotive

Comprehensive tutorial to build and test a travel-focused AI voice agent using Python and VideoSDK. Includes setup, code, and testing.

## 1. Introduction to AI Voice Agents in Travel

### What is an AI Voice Agent?
An AI voice agent is a software system that understands spoken language, processes it using artificial intelligence, and responds in natural, human-like speech. These agents combine speech-to-text (STT), large language models (LLM), and text-to-speech (TTS) to enable real-time, conversational interactions.

### Why are They Important for the Travel Industry?
Travelers often need instant, reliable information about destinations, bookings, and logistics. AI voice agents can provide 24/7 assistance, answer questions about flights, hotels, and local attractions, and even help plan itineraries. This enhances user experience, reduces support costs, and enables personalized service at scale.

### Core Components of a Voice Agent
- **STT (Speech-to-Text):** Converts user speech into text.
- **LLM (Large Language Model):** Processes text and generates intelligent responses.
- **TTS (Text-to-Speech):** Converts responses back to natural-sounding speech.

### What You'll Build in This Tutorial
You will create a fully functional travel AI voice agent using Python and the VideoSDK AI Agents framework. The agent will answer travel-related questions, suggest destinations, and provide helpful tips—all via voice in a browser playground.

## 2. Architecture and Core Concepts

### High-Level Architecture Overview
The AI voice agent processes spoken input from the user, transcribes it, generates a response, and speaks it back. Here's a visual overview:

```mermaid
sequenceDiagram
    participant User
    participant Agent
    participant STT
    participant LLM
    participant TTS
    participant VAD
    participant TurnDetector

    User->>Agent: Speaks question
    Agent->>VAD: Detects speech activity
    Agent->>TurnDetector: Detects end of turn
    Agent->>STT: Transcribes speech
    STT-->>Agent: Returns text
    Agent->>LLM: Generates response
    LLM-->>Agent: Returns answer
    Agent->>TTS: Synthesizes speech
    TTS-->>Agent: Plays audio to User
```

### Understanding Key Concepts in the VideoSDK Framework
- **Agent:** The main class that defines your AI's persona and logic.
- **CascadingPipeline:** Connects STT, LLM, TTS, VAD, and turn detection plugins. For a deeper understanding of how this works, see the [Cascading pipeline in AI voice Agents](https://docs.videosdk.live/ai_agents/core-components/cascading-pipeline).
- **VAD (Voice Activity Detection):** Detects when a user starts/stops speaking.
- **TurnDetector:** Helps determine when the user's turn is over, improving natural conversation flow. Learn more about its role in the [Turn detector for AI voice Agents](https://docs.videosdk.live/ai_agents/plugins/turn-detector).

If you're new to building voice agents, the [Voice Agent Quick Start Guide](https://docs.videosdk.live/ai_agents/voice-agent-quick-start) provides a step-by-step introduction to get you started quickly.

For a comprehensive look at the essential building blocks, check out the [AI voice Agent core components overview](https://docs.videosdk.live/ai_agents/core-components/overview).

## 3. Setting Up the Development Environment

### Prerequisites
- Python 3.11+
- A free VideoSDK account (for API keys and dashboard access)

### Step 1: Create a Virtual Environment
Open your terminal and run:

```bash
python3 -m venv venv
source venv/bin/activate  # On Windows: venv\Scripts\activate
```

### Step 2: Install Required Packages
Install the VideoSDK AI Agents SDK and plugins:

```bash
pip install videosdk-agents videosdk-plugins-silero videosdk-plugins-deepgram videosdk-plugins-elevenlabs videosdk-plugins-openai
```

- For speech-to-text, you'll be using the [Deepgram STT Plugin for voice agent](https://docs.videosdk.live/ai_agents/plugins/stt/deepgram).
- For text-to-speech, leverage the [ElevenLabs TTS Plugin for voice agent](https://docs.videosdk.live/ai_agents/plugins/tts/eleven-labs).
- For language understanding, integrate the [OpenAI LLM Plugin for voice agent](https://docs.videosdk.live/ai_agents/plugins/llm/openai).
- For voice activity detection, utilize the [Silero Voice Activity Detection](https://docs.videosdk.live/ai_agents/plugins/silero-vad) plugin.

### Step 3: Configure API Keys in a .env File
Create a `.env` file in your project directory with the following (replace with your actual keys):

```env
VIDEOSDK_API_KEY=your_videosdk_key
DEEPGRAM_API_KEY=your_deepgram_key
OPENAI_API_KEY=your_openai_key
ELEVENLABS_API_KEY=your_elevenlabs_key
```

## 4. Building the AI Voice Agent: A Step-by-Step Guide

Let's build the travel assistant agent! Here is the complete, runnable code. We'll break it down step by step in the sections below.

```python
import asyncio, os
from videosdk.agents import Agent, AgentSession, CascadingPipeline, JobContext, RoomOptions, WorkerJob, ConversationFlow
from videosdk.plugins.silero import SileroVAD
from videosdk.plugins.turn_detector import TurnDetector, pre_download_model
from videosdk.plugins.deepgram import DeepgramSTT
from videosdk.plugins.openai import OpenAILLM
from videosdk.plugins.elevenlabs import ElevenLabsTTS
from typing import AsyncIterator

# Pre-downloading the Turn Detector model
pre_download_model()

agent_instructions = "You are a friendly and knowledgeable AI Voice Agent specializing in travel assistance. Your persona is that of a helpful virtual travel concierge, always polite, patient, and eager to assist travelers with their needs. 

Capabilities:
- Answer questions about travel destinations, including popular attractions, local customs, weather, and best times to visit.
- Provide up-to-date information on flights, hotels, and transportation options.
- Assist users in planning itineraries, suggesting activities, and recommending restaurants or accommodations based on user preferences.
- Offer guidance on travel documentation, visa requirements, and safety tips.
- Help with booking reminders, packing checklists, and travel alerts.

Constraints and Limitations:
- You do not have access to real-time booking systems or personal user data; you can only provide general information and suggestions.
- You are not a licensed travel agent and cannot make bookings or reservations on behalf of users.
- Always advise users to verify critical travel information (such as visa requirements or travel advisories) with official sources before making decisions.
- Do not provide medical, legal, or financial advice; for such matters, instruct users to consult qualified professionals.
- Maintain user privacy and do not request or store any sensitive personal information."

class MyVoiceAgent(Agent):
    def __init__(self):
        super().__init__(instructions=agent_instructions)
    async def on_enter(self): await self.session.say("Hello! How can I help?")
    async def on_exit(self): await self.session.say("Goodbye!")

async def start_session(context: JobContext):
    # Create agent and conversation flow
    agent = MyVoiceAgent()
    conversation_flow = ConversationFlow(agent)

    # Create pipeline
    pipeline = CascadingPipeline(
        stt=DeepgramSTT(model="nova-2", language="en"),
        llm=OpenAILLM(model="gpt-4o"),
        tts=ElevenLabsTTS(model="eleven_flash_v2_5"),
        vad=SileroVAD(threshold=0.35),
        turn_detector=TurnDetector(threshold=0.8)
    )

    session = AgentSession(
        agent=agent,
        pipeline=pipeline,
        conversation_flow=conversation_flow
    )

    try:
        await context.connect()
        await session.start()
        # Keep the session running until manually terminated
        await asyncio.Event().wait()
    finally:
        # Clean up resources when done
        await session.close()
        await context.shutdown()

def make_context() -> JobContext:
    room_options = RoomOptions(
    #  room_id="YOUR_MEETING_ID",  # Set to join a pre-created room; omit to auto-create
        name="VideoSDK Cascaded Agent",
        playground=True
    )

    return JobContext(room_options=room_options)

if __name__ == "__main__":
    job = WorkerJob(entrypoint=start_session, jobctx=make_context)
    job.start()
```

If you want a guided walkthrough of these steps, refer to the [Voice Agent Quick Start Guide](https://docs.videosdk.live/ai_agents/voice-agent-quick-start).

### Step 4.1: Generating a VideoSDK Meeting ID
Before launching the agent, you can generate a meeting ID using the VideoSDK API. In most cases, the agent will auto-create a room, but here's how to do it manually:

```bash
curl -X POST "https://api.videosdk.live/v2/rooms" -H "Authorization: your_videosdk_key"
```

Copy the returned meeting ID and use it in your code if you want to join a specific room.

### Step 4.2: Creating the Custom Agent Class
The `MyVoiceAgent` class defines your travel assistant's persona and behaviors.

```python
agent_instructions = "You are a friendly and knowledgeable AI Voice Agent specializing in travel assistance. ..."

class MyVoiceAgent(Agent):
    def __init__(self):
        super().__init__(instructions=agent_instructions)
    async def on_enter(self):
        await self.session.say("Hello! How can I help?")
    async def on_exit(self):
        await self.session.say("Goodbye!")
```

- **Persona:** The instructions define the agent's travel expertise and helpful, polite tone.
- **on_enter/on_exit:** These methods greet and say goodbye to users.

### Step 4.3: Defining the Core Pipeline
The `CascadingPipeline` connects all the plugins: STT, LLM, TTS, VAD, and TurnDetector.

```python
pipeline = CascadingPipeline(
    stt=DeepgramSTT(model="nova-2", language="en"),
    llm=OpenAILLM(model="gpt-4o"),
    tts=ElevenLabsTTS(model="eleven_flash_v2_5"),
    vad=SileroVAD(threshold=0.35),
    turn_detector=TurnDetector(threshold=0.8)
)
```

- **DeepgramSTT:** High-accuracy speech-to-text. For more details, see the [Deepgram STT Plugin for voice agent](https://docs.videosdk.live/ai_agents/plugins/stt/deepgram).
- **OpenAILLM:** GPT-4o for intelligent, context-aware responses. Learn more in the [OpenAI LLM Plugin for voice agent](https://docs.videosdk.live/ai_agents/plugins/llm/openai).
- **ElevenLabsTTS:** Natural, expressive voice output. See the [ElevenLabs TTS Plugin for voice agent](https://docs.videosdk.live/ai_agents/plugins/tts/eleven-labs).
- **SileroVAD & TurnDetector:** For smooth, interruption-free conversations. Explore the [Silero Voice Activity Detection](https://docs.videosdk.live/ai_agents/plugins/silero-vad) and [Turn detector for AI voice Agents](https://docs.videosdk.live/ai_agents/plugins/turn-detector) plugins for more information.

### Step 4.4: Managing the Session and Startup Logic
The session manages the conversation and connects everything.

```python
async def start_session(context: JobContext):
    agent = MyVoiceAgent()
    conversation_flow = ConversationFlow(agent)
    pipeline = CascadingPipeline(
        stt=DeepgramSTT(model="nova-2", language="en"),
        llm=OpenAILLM(model="gpt-4o"),
        tts=ElevenLabsTTS(model="eleven_flash_v2_5"),
        vad=SileroVAD(threshold=0.35),
        turn_detector=TurnDetector(threshold=0.8)
    )
    session = AgentSession(
        agent=agent,
        pipeline=pipeline,
        conversation_flow=conversation_flow
    )
    try:
        await context.connect()
        await session.start()
        await asyncio.Event().wait()
    finally:
        await session.close()
        await context.shutdown()

def make_context() -> JobContext:
    room_options = RoomOptions(
        name="VideoSDK Cascaded Agent",
        playground=True
    )
    return JobContext(room_options=room_options)

if __name__ == "__main__":
    job = WorkerJob(entrypoint=start_session, jobctx=make_context)
    job.start()
```

- **JobContext:** Handles room creation and playground mode.
- **WorkerJob:** Launches the agent and manages its lifecycle.

## 5. Running and Testing the Agent

### Step 5.1: Running the Python Script
1. Ensure your `.env` file is set up with all API keys.
2. Run the agent script:

```bash
python main.py
```

3. In the console, you'll see a 'Playground' link.

### Step 5.2: Interacting with the Agent in the Playground
- Open the playground link in your browser.
- Join as a participant.
- Speak or type questions about travel.
- The agent will respond in real-time with voice and text.

To experiment and test your agent in real time, use the [AI Agent playground](https://docs.videosdk.live/ai_agents/playground) for hands-on interaction.

**Graceful Shutdown:** Press `Ctrl+C` in your terminal to stop the agent and clean up resources.

## 6. Advanced Features and Customizations

### Extending Functionality with Custom Tools
- Integrate additional APIs (e.g., weather, flight status) for richer answers.
- Add custom plugins for language translation or sentiment analysis.

### Exploring Other Plugins
- Try alternative STT/TTS providers (Cartesia, Rime, Deepgram).
- Experiment with different LLMs (Google Gemini, Anthropic Claude).
- Adjust VAD and TurnDetector thresholds for different environments.

## 7. Troubleshooting Common Issues

### API Key and Authentication Errors
- Double-check your `.env` file for typos.
- Ensure all required API keys are active and not expired.

### Audio Input/Output Problems
- Check your browser's microphone permissions.
- Test with a different browser or device if audio isn't working.

### Dependency and Version Conflicts
- Use a fresh virtual environment.
- Run `pip install --upgrade` for all packages if you see import errors.

## 8. Conclusion

You've built a fully functional AI voice agent tailored for travel assistance! This agent can answer questions, suggest destinations, and provide travel tips—all via natural voice.

**Next Steps:**
- Deploy your agent for real-world users.
- Integrate more APIs for live data.
- Explore advanced customization with the VideoSDK framework.

Happy travels—and happy coding!

Integrating OpenAI with VideoSDK Agents

Step-by-step guide to building a travel AI voice agent using Python and VideoSDK. Learn setup, code, testing, and customization.

Build an AI Voice Agent for Travel with Python & VideoSDK

AI Voice Agent for Travel Guide

A practical, step-by-step guide to building a fully functional AI voice agent using Python and the VideoSDK AI Agents framework. Includes code, testing, and troubleshooting.

## Introduction to AI Voice Agents in ai voice agent

AI voice agents are transforming the way we interact with technology. By leveraging advances in speech recognition, natural language processing, and real-time audio streaming, these agents can understand spoken queries and respond intelligently. In this tutorial, you will learn how to build a production-ready AI voice agent using Python and the VideoSDK AI Agents framework.

### What is an AI Voice Agent?

An AI voice agent is a software system that can engage in spoken conversations with users. It listens to audio input, transcribes it to text, processes the text using a language model, generates a response, and then speaks the response back to the user. This creates a seamless, conversational experience.

### Why are they important for the ai voice agent industry?

AI voice agents are crucial in industries such as customer support, healthcare, education, and smart devices. They automate repetitive tasks, provide instant assistance, and make technology more accessible. Their ability to handle natural language makes them invaluable for improving user engagement and satisfaction.

### Core Components of a Voice Agent

A typical AI voice agent consists of:
- **Speech-to-Text (STT):** Converts spoken words into text.
- **Voice [Activity Detection](https://docs.videosdk.live/ai_agents/plugins/silero-vad) (VAD):** Detects when the user is speaking.
- **Turn Detection:** Determines when it's the agent's turn to respond.
- **Large Language Model (LLM):** Processes the transcribed text and generates a response.
- **Text-to-Speech (TTS):** Converts the response text back to audio.

For a more detailed explanation of each part, check out the [AI voice Agent core components overview](https://docs.videosdk.live/ai_agents/core-components/overview).

### What You'll Build in This Tutorial

You will build a fully functional AI voice agent using Python and VideoSDK. The agent will handle real-time conversations, leveraging state-of-the-art plugins for each component. By the end, you'll have a working agent you can test and extend. If you're eager to get started quickly, refer to the [Voice Agent Quick Start Guide](https://docs.videosdk.live/ai_agents/voice-agent-quick-start) for a streamlined setup.

## Architecture and Core Concepts

### High-Level Architecture Overview

Let's visualize the data flow in our AI voice agent using a UML sequence diagram:

![Diagram](https://strapi.videosdk.live/uploads/0890a16ae3672781a46be42bfccce354_abe5ef4713.png)

This diagram shows how user speech is processed through various components before a spoken response is delivered. The [cascading pipeline in AI voice Agents](https://docs.videosdk.live/ai_agents/core-components/cascading-pipeline) is central to this process, as it orchestrates the flow between STT, LLM, TTS, VAD, and turn detection.

### Understanding Key Concepts in the VideoSDK Framework

- **Agent:** The core logic that determines how the agent responds. You'll subclass this to define your agent's behavior.
- **CascadingPipeline:** Chains together STT, VAD, Turn Detection, LLM, and TTS plugins for seamless processing.
- **VAD (Voice Activity Detection):** Detects when the user starts and stops speaking. Here, we use SileroVAD.
- **TurnDetector:** Determines conversational turns, ensuring the agent responds at the right moment. Learn more about the [Turn detector for AI voice Agents](https://docs.videosdk.live/ai_agents/plugins/turn-detector) and how it improves conversational flow.
- **AgentSession:** Manages the lifecycle of the conversation. For details on session management, see [AI voice Agent Sessions](https://docs.videosdk.live/ai_agents/core-components/agent-session).
- **ConversationFlow:** Orchestrates the dialogue between the user and the agent. Explore how [conversation flow in AI voice Agents](https://docs.videosdk.live/ai_agents/core-components/conversation-flow) is structured for dynamic interactions.

## Setting Up the Development Environment

Before diving into code, let's set up everything you need.

### Prerequisites (Python 3.11+, VideoSDK Account)

- **Python 3.11+** is required for compatibility with the VideoSDK AI Agents framework.
- **VideoSDK Account:** Sign up for a free account to access API keys and the dashboard.

### Step 1: Create a Virtual Environment

It's best practice to use a virtual environment to manage dependencies.

```bash
python3.11 -m venv venv
source venv/bin/activate
```

### Step 2: Install Required Packages

Install the VideoSDK AI Agents SDK and plugin dependencies:

```bash
pip install videosdk-agents videosdk-plugins-silero videosdk-plugins-turn-detector videosdk-plugins-deepgram videosdk-plugins-openai videosdk-plugins-elevenlabs
```

### Step 3: Configure API Keys in a `.env` file

Create a `.env` file in your project directory and add your API keys:

```env
VIDEOSDK_API_KEY=YOUR_VIDEOSDK_API_KEY
DEEPGRAM_API_KEY=YOUR_DEEPGRAM_API_KEY
OPENAI_API_KEY=YOUR_OPENAI_API_KEY
ELEVENLABS_API_KEY=YOUR_ELEVENLABS_API_KEY
```

Replace the placeholders with your actual keys from the respective dashboards.

## Building the AI Voice Agent: A Step-by-Step Guide

Let's walk through the process of building your AI voice agent.

First, here is the complete, runnable code for your agent:

```python
import asyncio, os
from videosdk.agents import Agent, AgentSession, CascadingPipeline, JobContext, RoomOptions, WorkerJob, ConversationFlow
from videosdk.plugins.silero import SileroVAD
from videosdk.plugins.turn_detector import TurnDetector, pre_download_model
from videosdk.plugins.deepgram import DeepgramSTT
from videosdk.plugins.openai import OpenAILLM
from videosdk.plugins.elevenlabs import ElevenLabsTTS
from typing import AsyncIterator

# Pre-downloading the Turn Detector model
pre_download_model()

agent_instructions = "You are an AI Voice Agent designed to assist users with a wide range of general inquiries and tasks in a friendly, professional, and efficient manner. Your persona is that of a knowledgeable and approachable digital assistant, always eager to help and provide clear, concise information.

Capabilities:
- Answer general questions about AI voice agents, their features, and use cases.
- Guide users through basic troubleshooting steps for common voice agent issues.
- Provide information about integrating AI voice agents into various platforms and workflows.
- Offer tips on optimizing user experience with AI voice agents.
- Assist with scheduling, reminders, and simple task management if requested.

Constraints and Limitations:
- Do not provide personal opinions or make unsupported claims about AI technology.
- Do not offer legal, medical, or financial advice; always recommend consulting a qualified professional for such matters.
- Do not collect or store any personal or sensitive user data.
- If unsure about an answer, politely acknowledge the limitation and suggest seeking further assistance from official documentation or support channels.
- Always maintain user privacy and adhere to ethical guidelines in all interactions."

class MyVoiceAgent(Agent):
    def __init__(self):
        super().__init__(instructions=agent_instructions)
    async def on_enter(self): await self.session.say("Hello! How can I help?")
    async def on_exit(self): await self.session.say("Goodbye!")

async def start_session(context: JobContext):
    # Create agent and conversation flow
    agent = MyVoiceAgent()
    conversation_flow = ConversationFlow(agent)

    # Create pipeline
    pipeline = CascadingPipeline(
        stt=DeepgramSTT(model="nova-2", language="en"),
        llm=OpenAILLM(model="gpt-4o"),
        tts=ElevenLabsTTS(model="eleven_flash_v2_5"),
        vad=SileroVAD(threshold=0.35),
        turn_detector=TurnDetector(threshold=0.8)
    )

    session = AgentSession(
        agent=agent,
        pipeline=pipeline,
        conversation_flow=conversation_flow
    )

    try:
        await context.connect()
        await session.start()
        # Keep the session running until manually terminated
        await asyncio.Event().wait()
    finally:
        # Clean up resources when done
        await session.close()
        await context.shutdown()

def make_context() -> JobContext:
    room_options = RoomOptions(
    #  room_id="YOUR_MEETING_ID",  # Set to join a pre-created room; omit to auto-create
        name="VideoSDK Cascaded Agent",
        playground=True
    )

    return JobContext(room_options=room_options)

if __name__ == "__main__":
    job = WorkerJob(entrypoint=start_session, jobctx=make_context)
    job.start()
```

Now, let's break down the code step by step.

### Step 4.1: Generating a VideoSDK Meeting ID (curl example)

Before running your agent, you'll need a meeting ID. You can generate one using the VideoSDK API:

```bash
curl -X POST \
  -H "Authorization: YOUR_VIDEOSDK_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{"region": "sg001"}' \
  https://api.videosdk.live/v2/rooms
```

Replace `YOUR_VIDEOSDK_API_KEY` with your actual key. The response will include a `roomId` you can use.

### Step 4.2: Creating the Custom Agent Class

The agent's behavior is defined in a custom class that inherits from `Agent`. Let's look at that part of the code:

```python
class MyVoiceAgent(Agent):
    def __init__(self):
        super().__init__(instructions=agent_instructions)
    async def on_enter(self):
        await self.session.say("Hello! How can I help?")
    async def on_exit(self):
        await self.session.say("Goodbye!")
```

- The agent is initialized with a detailed instruction prompt.
- `on_enter` and `on_exit` provide greetings and farewells to the user.

### Step 4.3: Defining the Core Pipeline

The pipeline chains together all the plugins for speech, language, and audio processing:

```python
pipeline = CascadingPipeline(
    stt=DeepgramSTT(model="nova-2", language="en"),
    llm=OpenAILLM(model="gpt-4o"),
    tts=ElevenLabsTTS(model="eleven_flash_v2_5"),
    vad=SileroVAD(threshold=0.35),
    turn_detector=TurnDetector(threshold=0.8)
)
```

- **STT:** Deepgram's Nova-2 for accurate English transcription. For more on integrating Deepgram, see the [Deepgram STT Plugin for voice agent](https://docs.videosdk.live/ai_agents/plugins/stt/deepgram).
- **LLM:** OpenAI's GPT-4o for intelligent responses. Learn about the [OpenAI LLM Plugin for voice agent](https://docs.videosdk.live/ai_agents/plugins/llm/openai) to unlock advanced conversational abilities.
- **TTS:** ElevenLabs for natural-sounding speech; see the [ElevenLabs TTS Plugin for voice agent](https://docs.videosdk.live/ai_agents/plugins/tts/eleven-labs) for more details.
- **VAD:** SileroVAD detects when the user is speaking.
- **TurnDetector:** Ensures the agent responds at the right time.

### Step 4.4: Managing the Session and Startup Logic

Session management ensures the agent joins the meeting, starts the conversation, and handles cleanup:

```python
async def start_session(context: JobContext):
    agent = MyVoiceAgent()
    conversation_flow = ConversationFlow(agent)
    pipeline = CascadingPipeline(
        stt=DeepgramSTT(model="nova-2", language="en"),
        llm=OpenAILLM(model="gpt-4o"),
        tts=ElevenLabsTTS(model="eleven_flash_v2_5"),
        vad=SileroVAD(threshold=0.35),
        turn_detector=TurnDetector(threshold=0.8)
    )
    session = AgentSession(
        agent=agent,
        pipeline=pipeline,
        conversation_flow=conversation_flow
    )
    try:
        await context.connect()
        await session.start()
        await asyncio.Event().wait()
    finally:
        await session.close()
        await context.shutdown()
```

- The session is started and kept alive until you manually stop it (Ctrl+C).
- Graceful shutdown ensures resources are released.

The `make_context()` function sets up the meeting room options, including the playground flag for easy testing:

```python
def make_context() -> JobContext:
    room_options = RoomOptions(
        name="VideoSDK Cascaded Agent",
        playground=True
    )
    return JobContext(room_options=room_options)
```

## Running and Testing the Agent

### Step 5.1: Running the Python Script

Start your agent with:

```bash
python main.py
```

The script will print a "playground" URL in the console. This lets you join the meeting as a user and test the agent in real time.

### Step 5.2: Interacting with the Agent in the Playground

- Open the playground URL in your browser.
- Join the meeting room.
- Speak into your microphone; the agent will listen, process your query, and respond aloud.
- To stop the agent, press Ctrl+C in your terminal. This triggers a graceful shutdown.

## Advanced Features and Customizations

### Extending Functionality with Custom Tools

You can add custom tools (function_tool) to extend your agent's capabilities, such as integrating with external APIs or databases. Define new methods in your agent class and register them as tools.

### Exploring Other Plugins (STT/LLM/TTS options)

- **STT:** Try Cartesia for best accuracy or Rime for low-cost transcription.
- **TTS:** ElevenLabs offers top quality, but Deepgram TTS is cost-effective.
- **LLM:** Use Google Gemini for alternative conversational AI.

Swap out plugins in the pipeline to experiment with different providers and models.

## Troubleshooting Common Issues

### API Key and Authentication Errors

- Double-check your `.env` file for typos.
- Ensure all required API keys are present and valid.

### Audio Input/Output Problems

- Verify your microphone and speakers are working.
- Check browser permissions if using the playground.

### Dependency and Version Conflicts

- Use Python 3.11+ and install packages in a fresh virtual environment.
- Run `pip freeze` to check installed versions.

## Conclusion

You've built a fully functional, production-ready AI voice agent using Python and VideoSDK. This agent can handle real-time conversations and is easily extensible with custom tools and plugins. Explore the VideoSDK documentation to unlock more advanced features, and consider deploying your agent to serve real users.

Happy building!

How to Build a Custom AI Agent with VideoSDK

Integrating OpenAI with VideoSDK AI Agents

Real-Time Voicebot Deployment with VideoSDK

Learn to build a production-ready AI voice agent using Python and VideoSDK. Step-by-step tutorial with code, testing, and troubleshooting tips.

How to Build an AI Voice Agent with Python and VideoSDK

Build an AI Voice Agent

Comprehensive tutorial to build and deploy an AI voice agent for insurance using VideoSDK, Python, and best-in-class plugins. Includes full code, setup, and testing.

## Introduction to AI Voice Agents in ai voice agent for insurance

AI voice agents are transforming the way insurance companies interact with their customers. These intelligent virtual assistants use advanced speech and language technologies to understand, process, and respond to user queries in real time. In this tutorial, we'll explore how to build a robust AI voice agent tailored for the insurance industry using the VideoSDK AI Agents framework.

### What is an AI Voice Agent?

An AI voice agent is a software application that can engage in spoken conversations with users. It leverages speech-to-text (STT) to convert spoken words into text, a large language model (LLM) to understand and generate responses, and text-to-speech (TTS) to reply in natural-sounding audio.

### Why are they important for the ai voice agent for insurance industry?

In the insurance sector, voice agents can:
- Answer questions about policy coverage, claims, and quotes
- Guide customers through the claims process
- Provide 24/7 support and reduce call center workload
- Improve customer satisfaction with quick, accurate responses

### Core Components of a Voice Agent
- **STT (Speech-to-Text):** Transcribes user speech into text.
- **LLM (Large Language Model):** Understands intent and generates responses.
- **TTS (Text-to-Speech):** Converts the agent's reply back into audio.

To get a comprehensive understanding of these elements, refer to the [AI voice Agent core components overview](https://docs.videosdk.live/ai_agents/core-components/overview) in the VideoSDK documentation.

### What You'll Build in This Tutorial

By the end of this guide, you'll have a fully functional AI voice agent for insurance that you can test in the VideoSDK playground. The agent will be able to answer insurance-related questions, guide users through claims, and provide general support.

## Architecture and Core Concepts

### High-Level Architecture Overview

Let's look at how the voice agent processes a conversation:

1. **User speaks** into their microphone.
2. **STT** transcribes the audio to text.
3. **LLM** analyzes the text and generates a response.
4. **TTS** converts the response to audio.
5. **Agent** delivers the reply back to the user.


### Understanding Key Concepts in the VideoSDK Framework

- **Agent:** The main class that defines your AI assistant's behavior and persona.
- **CascadingPipeline:** Manages the flow of audio and text through STT, LLM, TTS, VAD, and turn detection. For more details on how this works, check out the [Cascading pipeline in AI voice Agents](https://docs.videosdk.live/ai_agents/core-components/cascading-pipeline).
- **VAD (Voice Activity Detection):** Detects when the user is speaking.
- **TurnDetector:** Determines when it's the agent's turn to respond.

These components work together to create a seamless conversational experience.

## Setting Up the Development Environment

### Prerequisites

Before you begin, ensure you have:
- **Python 3.11+** installed
- A **VideoSDK account** (sign up at app.videosdk.live)
- API keys for Deepgram (STT), ElevenLabs (TTS), and OpenAI (LLM)

If you're just getting started, the [Voice Agent Quick Start Guide](https://docs.videosdk.live/ai_agents/voice-agent-quick-start) provides step-by-step instructions for setup.

### Step 1: Create a Virtual Environment

Open your terminal and run:

```bash
python3.11 -m venv venv
source venv/bin/activate  # On Windows: venv\Scripts\activate
```

### Step 2: Install Required Packages

Install the necessary dependencies:

```bash
pip install videosdk-agents videosdk-plugins-openai videosdk-plugins-deepgram videosdk-plugins-elevenlabs videosdk-plugins-silero videosdk-plugins-turn-detector
```

### Step 3: Configure API Keys in a `.env` File

Create a `.env` file in your project directory and add your API keys:

```env
VIDEOSDK_API_KEY=your_videosdk_api_key
OPENAI_API_KEY=your_openai_api_key
DEEPGRAM_API_KEY=your_deepgram_api_key
ELEVENLABS_API_KEY=your_elevenlabs_api_key
```

## Building the AI Voice Agent: A Step-by-Step Guide

Let's dive into the code! Here's the complete, runnable script for your insurance voice agent.

```python
import asyncio, os
from videosdk.agents import Agent, AgentSession, CascadingPipeline, JobContext, RoomOptions, WorkerJob, ConversationFlow
from videosdk.plugins.silero import SileroVAD
from videosdk.plugins.turn_detector import TurnDetector, pre_download_model
from videosdk.plugins.deepgram import DeepgramSTT
from videosdk.plugins.openai import OpenAILLM
from videosdk.plugins.elevenlabs import ElevenLabsTTS
from typing import AsyncIterator

# Pre-downloading the Turn Detector model
pre_download_model()

agent_instructions = "You are an AI Voice Agent specializing in insurance services. Your persona is that of a knowledgeable, friendly, and patient insurance assistant. Your primary goal is to help users understand insurance products, answer questions about policy coverage, assist with claims processes, provide quotes, and guide users through common insurance procedures. You can explain different types of insurance (health, auto, home, life), clarify policy terms, and help users find relevant resources or contact information. 

Capabilities:
- Answer questions about various insurance products and coverage options.
- Guide users through the process of filing a claim or checking claim status.
- Provide general information about premiums, deductibles, and policy benefits.
- Offer quotes or estimate premiums based on user input (if data is available).
- Direct users to appropriate resources or connect them with a human agent for complex issues.

Constraints and Limitations:
- You are not a licensed insurance agent and cannot provide legally binding advice or finalize policy sales.
- Do not collect or store sensitive personal information such as Social Security Numbers or payment details.
- Always recommend that users consult with a licensed insurance professional for personalized advice or to complete transactions.
- If unsure or if the user's request is outside your scope, politely suggest speaking with a human insurance representative.
- Maintain user privacy and confidentiality at all times."

class MyVoiceAgent(Agent):
    def __init__(self):
        super().__init__(instructions=agent_instructions)
    async def on_enter(self): await self.session.say("Hello! How can I help?")
    async def on_exit(self): await self.session.say("Goodbye!")

async def start_session(context: JobContext):
    # Create agent and conversation flow
    agent = MyVoiceAgent()
    conversation_flow = ConversationFlow(agent)

    # Create pipeline
    pipeline = CascadingPipeline(
        stt=DeepgramSTT(model="nova-2", language="en"),
        llm=OpenAILLM(model="gpt-4o"),
        tts=ElevenLabsTTS(model="eleven_flash_v2_5"),
        vad=SileroVAD(threshold=0.35),
        turn_detector=TurnDetector(threshold=0.8)
    )

    session = AgentSession(
        agent=agent,
        pipeline=pipeline,
        conversation_flow=conversation_flow
    )

    try:
        await context.connect()
        await session.start()
        # Keep the session running until manually terminated
        await asyncio.Event().wait()
    finally:
        # Clean up resources when done
        await session.close()
        await context.shutdown()

def make_context() -> JobContext:
    room_options = RoomOptions(
    #  room_id="YOUR_MEETING_ID",  # Set to join a pre-created room; omit to auto-create
        name="VideoSDK Cascaded Agent",
        playground=True
    )

    return JobContext(room_options=room_options)

if __name__ == "__main__":
    job = WorkerJob(entrypoint=start_session, jobctx=make_context)
    job.start()
```

Let's break down each part of this code to understand how the agent works.

### Step 4.1: Generating a VideoSDK Meeting ID

Before launching your agent, you need a meeting room where users and the agent can interact. You can generate a meeting ID using the VideoSDK API:

```bash
curl -X POST "https://api.videosdk.live/v2/rooms" \
  -H "Authorization: your_videosdk_api_key" \
  -H "Content-Type: application/json"
```

The response will include a `roomId` you can use in your agent configuration. However, if you set `playground=True` in `RoomOptions`, the agent will auto-create a room and provide a test link for you.

### Step 4.2: Creating the Custom Agent Class

The heart of your voice agent is the custom class that defines its persona and behavior.

```python
agent_instructions = "You are an AI Voice Agent specializing in insurance services. ..."

class MyVoiceAgent(Agent):
    def __init__(self):
        super().__init__(instructions=agent_instructions)
    async def on_enter(self):
        await self.session.say("Hello! How can I help?")
    async def on_exit(self):
        await self.session.say("Goodbye!")
```

- The `agent_instructions` string defines the agent's capabilities, constraints, and persona.
- The `MyVoiceAgent` class inherits from `Agent` and sets up the initial greeting and goodbye messages.

### Step 4.3: Defining the Core Pipeline

The pipeline coordinates all the plugins that power your agent:

```python
pipeline = CascadingPipeline(
    stt=DeepgramSTT(model="nova-2", language="en"),
    llm=OpenAILLM(model="gpt-4o"),
    tts=ElevenLabsTTS(model="eleven_flash_v2_5"),
    vad=SileroVAD(threshold=0.35),
    turn_detector=TurnDetector(threshold=0.8)
)
```

- **DeepgramSTT**: Converts speech to text. Learn more about integrating this with your agent in the [Deepgram STT Plugin for voice agent](https://docs.videosdk.live/ai_agents/plugins/stt/deepgram) documentation.
- **OpenAILLM**: Processes the text and generates a response. For setup and configuration, see the [OpenAI LLM Plugin for voice agent](https://docs.videosdk.live/ai_agents/plugins/llm/openai).
- **ElevenLabsTTS**: Synthesizes the agent's reply into audio. Explore options in the [ElevenLabs TTS Plugin for voice agent](https://docs.videosdk.live/ai_agents/plugins/tts/eleven-labs).
- **SileroVAD**: Detects when the user is speaking. For more on this technology, refer to [Silero Voice Activity Detection](https://docs.videosdk.live/ai_agents/plugins/silero-vad).
- **TurnDetector**: Determines when the agent should respond. Detailed guidance is available in the [Turn detector for AI voice Agents](https://docs.videosdk.live/ai_agents/plugins/turn-detector).

You can swap out these plugins for others supported by VideoSDK as needed.

### Step 4.4: Managing the Session and Startup Logic

The session orchestrates the entire interaction lifecycle.

```python
async def start_session(context: JobContext):
    agent = MyVoiceAgent()
    conversation_flow = ConversationFlow(agent)
    pipeline = CascadingPipeline(...)
    session = AgentSession(
        agent=agent,
        pipeline=pipeline,
        conversation_flow=conversation_flow
    )
    try:
        await context.connect()
        await session.start()
        await asyncio.Event().wait()
    finally:
        await session.close()
        await context.shutdown()

def make_context() -> JobContext:
    room_options = RoomOptions(
        name="VideoSDK Cascaded Agent",
        playground=True
    )
    return JobContext(room_options=room_options)

if __name__ == "__main__":
    job = WorkerJob(entrypoint=start_session, jobctx=make_context)
    job.start()
```

- `start_session` sets up the agent, pipeline, and session, then starts everything.
- `make_context` configures the meeting room, enabling the playground for easy testing.
- The main block launches the agent as a job.

## Running and Testing the Agent

### Step 5.1: Running the Python Script

Start your agent by running:

```bash
python main.py
```

In the console, you'll see a playground link (e.g., `https://playground.videosdk.live/agent/XXXX`).

### Step 5.2: Interacting with the Agent in the Playground

1. Open the playground link in your browser.
2. Allow microphone access when prompted.
3. Speak to the agent and listen to its responses.
4. To stop the agent, press `Ctrl+C` in your terminal. This will gracefully shut down the session and free resources.

For hands-on experimentation, try the [AI Agent playground](https://docs.videosdk.live/ai_agents/playground) to test and refine your agent's capabilities in real time.

## Advanced Features and Customizations

### Extending Functionality with Custom Tools

You can add custom tools (functions) to your agent for more advanced tasks, such as fetching policy details from a database or integrating with CRM systems. Implement a `function_tool` and register it with your agent for specialized actions.

### Exploring Other Plugins

VideoSDK supports various plugins. For example:
- **STT:** Cartesia (best quality), Deepgram (cost-effective), Rime (low-cost)
- **TTS:** ElevenLabs (high quality), Deepgram (cost-effective)
- **LLM:** OpenAI GPT-4, Google Gemini

Experiment with different plugins to optimize cost, speed, and accuracy for your use case.

## Troubleshooting Common Issues

### API Key and Authentication Errors
- Double-check that all API keys are correct and active in your `.env` file.
- Ensure your VideoSDK account is in good standing.

### Audio Input/Output Problems
- Make sure your browser has microphone permissions enabled.
- Test with different browsers if issues persist.

### Dependency and Version Conflicts
- Use Python 3.11+ as required.
- If you encounter errors, recreate your virtual environment and reinstall dependencies.

## Conclusion

Congratulations! You've built a production-ready AI voice agent for insurance using VideoSDK and Python. This agent can answer policy questions, guide users through claims, and provide quotes.

To take your agent further, consider:
- Integrating with backend insurance systems
- Adding multilingual support
- Deploying on cloud infrastructure

For guidance on taking your solution live, see the [AI voice Agent deployment](https://docs.videosdk.live/ai_agents/deploy-your-agents) documentation.

Continue exploring the VideoSDK documentation to unlock even more capabilities for your AI voice agents.

How to Build an AI Voice Bot with VideoSDK Agents

Integrating OpenAI GPT-4 with Voice Agents

Choosing the Best STT and TTS Plugins for Voice AI

Step-by-step guide to building an AI voice agent for insurance using Python and VideoSDK. Includes code, setup, and testing instructions.

Build an AI Voice Agent for Insurance with VideoSDK

AI Voice Agent for Insurance Guide

Explore tutorials, use cases, and tools for building intelligent, real-time AI voice agents for any application.

Explore comprehensive guides, tutorials, and use cases for building advanced AI Voice Agents. Learn to create real-time, conversational AI for any application.