Build a Conversational AI for Insurance

Step-by-step guide to building an AI Voice Agent for the insurance industry using VideoSDK.

Introduction to AI Voice Agents in Conversational AI for Insurance

In the rapidly evolving landscape of artificial intelligence, AI Voice Agents have emerged as a transformative force, particularly in industries like insurance. These agents are designed to interact with users through natural language, providing a seamless and efficient way to handle inquiries and tasks. In this tutorial, you'll learn how to build a conversational AI

Voice Agent

 tailored for the insurance industry using VideoSDK.

What is an AI

Voice Agent

?

An AI

Voice Agent

 is a software entity that uses artificial intelligence to interpret and respond to human speech. It leverages technologies like speech-to-text (STT), language models (LLM), and text-to-speech (TTS) to facilitate natural conversations. These agents can perform tasks ranging from answering questions to processing transactions, making them invaluable in customer service and support roles.

Why are they Important for the Insurance Industry?

In the insurance sector, AI Voice Agents can streamline operations by handling common customer inquiries, assisting with claims processing, and providing information about policies. This not only enhances customer satisfaction but also reduces operational costs by minimizing the need for human intervention in routine tasks.

Core Components of a

Voice Agent

  • Speech-to-Text (STT): Converts spoken language into text.
  • Language Model (LLM): Understands and processes the text to generate appropriate responses.
  • Text-to-Speech (TTS): Converts the text response back into speech for the user.

What You'll Build in This Tutorial

In this guide, we'll walk through the process of building a conversational AI

Voice Agent

 using VideoSDK. You'll learn to set up the development environment, create a custom agent class, define a processing pipeline, and test the agent in a simulated environment.

Architecture and Core Concepts

Understanding the architecture and core concepts is crucial for building an effective AI

Voice Agent

. Let's explore how the components interact within the VideoSDK framework.

High-Level Architecture Overview

The architecture of an AI Voice Agent involves several stages, starting from capturing the user's speech to generating a response. Here's a simplified flow:
  • User Speech: The user speaks into the system.
  • Speech-to-Text (STT): The audio is converted into text.
  • Language Model (LLM): The text is processed to understand the user's intent.
  • Text-to-Speech (TTS): The generated response is converted back into audio.
  • Agent Response: The agent delivers the audio response to the user.
Diagram

Understanding Key Concepts in the VideoSDK Framework

  • Agent: The core class representing your bot, responsible for managing interactions and responses.
  • CascadingPipeline: Defines the flow of audio processing, including STT, LLM, and TTS.
  • VAD & TurnDetector: These components help the agent determine when to listen and when to speak, ensuring smooth interactions.

Setting Up the Development Environment

Before diving into the code, let's set up the necessary tools and environment.

Prerequisites

Ensure you have Python 3.11+ installed and a VideoSDK account. You can sign up at app.videosdk.live.

Step 1: Create a Virtual Environment

Creating a virtual environment helps manage dependencies and avoid conflicts.
1python3 -m venv venv
2source venv/bin/activate  # On Windows use `venv\Scripts\activate`
3

Step 2: Install Required Packages

Install the necessary packages using pip.
1pip install videosdk
2pip install python-dotenv
3

Step 3: Configure API Keys in a .env File

Store your VideoSDK API keys and other sensitive information in a .env file for security.
1VIDEOSDK_API_KEY=your_api_key_here
2

Building the AI Voice Agent: A Step-by-Step Guide

Let's dive into building the AI Voice Agent. Here's the complete code:
1import asyncio, os
2from videosdk.agents import Agent, AgentSession, CascadingPipeline, JobContext, RoomOptions, WorkerJob, ConversationFlow
3from videosdk.plugins.silero import SileroVAD
4from videosdk.plugins.turn_detector import TurnDetector, pre_download_model
5from videosdk.plugins.deepgram import DeepgramSTT
6from videosdk.plugins.openai import OpenAILLM
7from videosdk.plugins.elevenlabs import ElevenLabsTTS
8from typing import AsyncIterator
9
10# Pre-downloading the Turn Detector model
11pre_download_model()
12
13agent_instructions = "You are an AI Voice Agent specialized in the insurance industry, acting as a 'helpful insurance advisor'. Your primary role is to assist users with inquiries related to insurance policies, claims, and coverage options. You can provide information on different types of insurance such as health, auto, home, and life insurance. You are capable of guiding users through the process of filing claims, explaining policy terms, and offering general advice on choosing the right insurance plan. However, you are not a licensed insurance agent and cannot provide personalized financial advice or make policy changes. Always remind users to consult with a licensed insurance professional for specific advice and policy modifications. You must ensure user data privacy and comply with relevant data protection regulations."
14
15class MyVoiceAgent(Agent):
16    def __init__(self):
17        super().__init__(instructions=agent_instructions)
18    async def on_enter(self): await self.session.say("Hello! How can I help?")
19    async def on_exit(self): await self.session.say("Goodbye!")
20
21async def start_session(context: JobContext):
22    # Create agent and conversation flow
23    agent = MyVoiceAgent()
24    conversation_flow = ConversationFlow(agent)
25
26    # Create pipeline
27    pipeline = CascadingPipeline(
28        stt=DeepgramSTT(model="nova-2", language="en"),
29        llm=OpenAILLM(model="gpt-4o"),
30        tts=ElevenLabsTTS(model="eleven_flash_v2_5"),
31        vad=[Silero Voice Activity Detection](https://docs.videosdk.live/ai_agents/plugins/silero-vad)(threshold=0.35),
32        turn_detector=[Turn detector for AI voice Agents](https://docs.videosdk.live/ai_agents/plugins/turn-detector)(threshold=0.8)
33    )
34
35    session = [AI voice Agent Sessions](https://docs.videosdk.live/ai_agents/core-components/agent-session)(
36        agent=agent,
37        pipeline=pipeline,
38        conversation_flow=conversation_flow
39    )
40
41    try:
42        await context.connect()
43        await session.start()
44        # Keep the session running until manually terminated
45        await asyncio.Event().wait()
46    finally:
47        # Clean up resources when done
48        await session.close()
49        await context.shutdown()
50
51def make_context() -> JobContext:
52    room_options = RoomOptions(
53    #  room_id="YOUR_MEETING_ID",  # Set to join a pre-created room; omit to auto-create
54        name="VideoSDK Cascaded Agent",
55        playground=True
56    )
57
58    return JobContext(room_options=room_options)
59
60if __name__ == "__main__":
61    job = WorkerJob(entrypoint=start_session, jobctx=make_context)
62    job.start()
63

Step 4.1: Generating a VideoSDK Meeting ID

To interact with the agent, you'll need a meeting ID. You can generate this using the VideoSDK API. Here's an example using curl:
1curl -X POST \
2  https://api.videosdk.live/v1/meetings \
3  -H "Authorization: Bearer YOUR_API_KEY" \
4  -H "Content-Type: application/json"
5

Step 4.2: Creating the Custom Agent Class

The MyVoiceAgent class is the heart of our voice agent. It inherits from the Agent class and defines custom behavior for entering and exiting conversations.
1class MyVoiceAgent(Agent):
2    def __init__(self):
3        super().__init__(instructions=agent_instructions)
4    async def on_enter(self): await self.session.say("Hello! How can I help?")
5    async def on_exit(self): await self.session.say("Goodbye!")
6

Step 4.3: Defining the Core Pipeline

The

Cascading pipeline in AI voice Agents

 integrates various plugins to process the audio input and generate responses. Each plugin plays a specific role:
  • DeepgramSTT: Converts speech to text.
  • OpenAILLM: Processes the text to understand user intent.
  • ElevenLabsTTS: Converts the response text back to speech.
  • SileroVAD & TurnDetector: Manage when the agent should listen and respond.
1pipeline = CascadingPipeline(
2    stt=DeepgramSTT(model="nova-2", language="en"),
3    llm=OpenAILLM(model="gpt-4o"),
4    tts=ElevenLabsTTS(model="eleven_flash_v2_5"),
5    vad=SileroVAD(threshold=0.35),
6    turn_detector=TurnDetector(threshold=0.8)
7)
8

Step 4.4: Managing the Session and Startup Logic

The start_session function sets up the agent session and manages the lifecycle of interactions. The make_context function configures the session environment.
1async def start_session(context: JobContext):
2    agent = MyVoiceAgent()
3    conversation_flow = ConversationFlow(agent)
4
5    pipeline = CascadingPipeline(
6        stt=DeepgramSTT(model="nova-2", language="en"),
7        llm=OpenAILLM(model="gpt-4o"),
8        tts=ElevenLabsTTS(model="eleven_flash_v2_5"),
9        vad=SileroVAD(threshold=0.35),
10        turn_detector=TurnDetector(threshold=0.8)
11    )
12
13    session = AgentSession(
14        agent=agent,
15        pipeline=pipeline,
16        conversation_flow=conversation_flow
17    )
18
19    try:
20        await context.connect()
21        await session.start()
22        await asyncio.Event().wait()
23    finally:
24        await session.close()
25        await context.shutdown()
26
27def make_context() -> JobContext:
28    room_options = RoomOptions(
29        name="VideoSDK Cascaded Agent",
30        playground=True
31    )
32    return JobContext(room_options=room_options)
33
34if __name__ == "__main__":
35    job = WorkerJob(entrypoint=start_session, jobctx=make_context)
36    job.start()
37

Running and Testing the Agent

With the setup complete, it's time to run and test your AI Voice Agent.

Step 5.1: Running the Python Script

Execute the script to start the agent:
1python main.py
2

Step 5.2: Interacting with the Agent in the Playground

Once the script is running, you'll receive a playground link in the console. Use this link to join the session and interact with your agent. You can type or speak your queries, and the agent will respond accordingly.

Advanced Features and Customizations

Enhance your AI Voice Agent with additional features and custom tools.

Extending Functionality with Custom Tools

The function_tool concept allows you to integrate custom logic or external APIs, expanding the agent's capabilities beyond predefined plugins.

Exploring Other Plugins

Experiment with different STT, LLM, and TTS plugins to optimize performance and cost. Options include Cartesia for STT and Google Gemini for LLM.

Troubleshooting Common Issues

Here are some solutions to common problems you might encounter.

API Key and Authentication Errors

Ensure your API keys are correctly set in the .env file and that your account is active.

Audio Input/Output Problems

Check your microphone and speaker settings. Ensure they are configured correctly on your system.

Dependency and Version Conflicts

Make sure all dependencies are up-to-date and compatible with Python 3.11+.

Conclusion

Congratulations on building your AI Voice Agent! You've learned how to set up a development environment, create a custom agent, and test it using VideoSDK. As next steps, consider exploring more advanced features and integrating additional plugins to enhance your agent's capabilities.

Start Building With Free $20 Balance

No credit card required to start.

Want to level-up your learning? Subscribe now

Subscribe to our newsletter for more tech based insights

FAQ