Build an AI Voice Agent for Lease Renewal

Step-by-step guide to building an AI Voice Agent for lease renewal with VideoSDK, complete with code examples and testing instructions.

Introduction to AI Voice Agents in Lease Renewal

AI Voice Agents are revolutionizing the way businesses interact with their customers by providing automated, intelligent responses to user queries. In the context of lease renewal, these agents can streamline the process by offering tenants quick access to information and assistance without the need for direct human intervention.

What is an AI

Voice Agent

?

An AI

Voice Agent

 is a software application that uses artificial intelligence to understand and respond to human speech. These agents are capable of performing tasks such as answering questions, providing information, and even completing transactions based on voice commands.

Why are they important for the lease renewal industry?

In the lease renewal industry, AI Voice Agents can significantly reduce the workload of property managers by handling routine inquiries and guiding tenants through the renewal process. This not only improves efficiency but also enhances tenant satisfaction by providing immediate assistance.

Core Components of a

Voice Agent

  • Speech-to-Text (STT): Converts spoken language into text.
  • Language Model (LLM): Processes the text to understand and generate appropriate responses.
  • Text-to-Speech (TTS): Converts text responses back into spoken language.
For a comprehensive understanding, refer to the

AI voice Agent core components overview

.

What You'll Build in This Tutorial

In this tutorial, you'll learn how to build an AI

Voice Agent

 specifically designed to assist with lease renewal processes. We'll use the VideoSDK AI Agents framework to create a fully functional agent that can interact with users in real-time.

Architecture and Core Concepts

High-Level Architecture Overview

The architecture of an AI

Voice Agent

 involves several key components working together to process user input and generate responses. The typical flow is as follows:
  1. User Speech: The user speaks into the system.
  2. Speech-to-Text (STT): The spoken words are converted into text.
  3. Language Model (LLM): The text is processed to determine the appropriate response.
  4. Text-to-Speech (TTS): The response is converted back into speech.
  5. User Interaction: The agent speaks the response back to the user.
Diagram

Understanding Key Concepts in the VideoSDK Framework

  • Agent: This is the core class representing your bot. It handles the interaction logic and manages the conversation flow.

  • Cascading Pipeline in AI voice Agents

    : This component manages the flow of audio processing through various stages such as STT, LLM, and TTS.
  • VAD & TurnDetector: These tools help the agent determine when to listen and when to respond, ensuring smooth interactions.

Setting Up the Development Environment

Prerequisites

Before you begin, ensure you have Python 3.11+ installed on your system. Additionally, you'll need a VideoSDK account, which you can create at app.videosdk.live.

Step 1: Create a Virtual Environment

To keep your project dependencies organized, it's recommended to use a virtual environment. You can create one using the following command:
1python -m venv myenv
2
Activate the virtual environment:
  • On Windows: bash myenv\Scripts\activate
  • On macOS/Linux: bash source myenv/bin/activate

Step 2: Install Required Packages

Install the necessary packages using pip:
1pip install videosdk
2

Step 3: Configure API Keys in a .env file

Create a .env file in your project directory and add your VideoSDK API key:
1VIDEOSDK_API_KEY=your_api_key_here
2

Building the AI Voice Agent: A Step-by-Step Guide

To build your AI Voice Agent, we'll start by presenting the complete, runnable code block, and then break it down into smaller parts for detailed explanations.

Complete Code Overview

1import asyncio, os
2from videosdk.agents import Agent, AgentSession, CascadingPipeline, JobContext, RoomOptions, WorkerJob, ConversationFlow
3from videosdk.plugins.silero import SileroVAD
4from videosdk.plugins.turn_detector import TurnDetector, pre_download_model
5from videosdk.plugins.deepgram import DeepgramSTT
6from videosdk.plugins.openai import OpenAILLM
7from videosdk.plugins.elevenlabs import ElevenLabsTTS
8from typing import AsyncIterator
9
10# Pre-downloading the Turn Detector model
11pre_download_model()
12
13agent_instructions = "You are an AI Voice Agent specialized in assisting tenants with lease renewal processes. Your persona is that of a knowledgeable and friendly real estate assistant. Your primary capabilities include providing information about lease renewal terms, guiding users through the renewal process, and answering frequently asked questions related to lease agreements. You can also assist in scheduling appointments with property managers for further discussions. However, you are not a legal advisor and must include a disclaimer advising users to consult a legal professional for any legal advice or concerns. You should always ensure that the information you provide is up-to-date and relevant to the user's specific location and property management company policies. Your responses should be clear, concise, and helpful, aiming to make the lease renewal process as smooth as possible for the user."
14
15class MyVoiceAgent(Agent):
16    def __init__(self):
17        super().__init__(instructions=agent_instructions)
18    async def on_enter(self): await self.session.say("Hello! How can I help?")
19    async def on_exit(self): await self.session.say("Goodbye!")
20
21async def start_session(context: JobContext):
22    # Create agent and conversation flow
23    agent = MyVoiceAgent()
24    conversation_flow = ConversationFlow(agent)
25
26    # Create pipeline
27    pipeline = CascadingPipeline(
28        stt=DeepgramSTT(model="nova-2", language="en"),
29        llm=OpenAILLM(model="gpt-4o"),
30        tts=ElevenLabsTTS(model="eleven_flash_v2_5"),
31        vad=SileroVAD(threshold=0.35),
32        turn_detector=TurnDetector(threshold=0.8)
33    )
34
35    session = [AgentSession](https://docs.videosdk.live/ai_agents/core-components/agent-session)(
36        agent=agent,
37        pipeline=pipeline,
38        conversation_flow=conversation_flow
39    )
40
41    try:
42        await context.connect()
43        await session.start()
44        # Keep the session running until manually terminated
45        await asyncio.Event().wait()
46    finally:
47        # Clean up resources when done
48        await session.close()
49        await context.shutdown()
50
51def make_context() -> JobContext:
52    room_options = RoomOptions(
53    #  room_id="YOUR_MEETING_ID",  # Set to join a pre-created room; omit to auto-create
54        name="VideoSDK Cascaded Agent",
55        playground=True
56    )
57
58    return JobContext(room_options=room_options)
59
60if __name__ == "__main__":
61    job = WorkerJob(entrypoint=start_session, jobctx=make_context)
62    job.start()
63

Step 4.1: Generating a VideoSDK Meeting ID

To interact with your AI Voice Agent, you'll need a meeting ID. You can generate one using the VideoSDK API. Here's an example using curl:
1curl -X POST \\
2  https://api.videosdk.live/v1/meetings \\
3  -H "Authorization: Bearer YOUR_API_KEY" \\
4  -H "Content-Type: application/json"
5

Step 4.2: Creating the Custom Agent Class

The MyVoiceAgent class extends the Agent class and defines the behavior of your voice agent. It uses the agent_instructions to guide its interactions.
1class MyVoiceAgent(Agent):
2    def __init__(self):
3        super().__init__(instructions=agent_instructions)
4    async def on_enter(self): await self.session.say("Hello! How can I help?")
5    async def on_exit(self): await self.session.say("Goodbye!")
6

Step 4.3: Defining the Core Pipeline

The

CascadingPipeline

 is responsible for processing audio through various stages such as STT, LLM, and TTS. Each plugin plays a crucial role in ensuring smooth interactions.
1pipeline = CascadingPipeline(
2    stt=DeepgramSTT(model="nova-2", language="en"),
3    llm=OpenAILLM(model="gpt-4o"),
4    tts=ElevenLabsTTS(model="eleven_flash_v2_5"),
5    vad=[Silero Voice Activity Detection](https://docs.videosdk.live/ai_agents/plugins/silero-vad)(threshold=0.35),
6    turn_detector=[Turn detector for AI voice Agents](https://docs.videosdk.live/ai_agents/plugins/turn-detector)(threshold=0.8)
7)
8

Step 4.4: Managing the Session and Startup Logic

The start_session function initializes the agent session and manages the lifecycle of the conversation. The make_context function sets up the room options for the VideoSDK.
1def make_context() -> JobContext:
2    room_options = RoomOptions(
3        name="VideoSDK Cascaded Agent",
4        playground=True
5    )
6    return JobContext(room_options=room_options)
7
8async def start_session(context: JobContext):
9    agent = MyVoiceAgent()
10    conversation_flow = ConversationFlow(agent)
11    pipeline = CascadingPipeline(
12        stt=DeepgramSTT(model="nova-2", language="en"),
13        llm=OpenAILLM(model="gpt-4o"),
14        tts=ElevenLabsTTS(model="eleven_flash_v2_5"),
15        vad=SileroVAD(threshold=0.35),
16        turn_detector=TurnDetector(threshold=0.8)
17    )
18    session = AgentSession(
19        agent=agent,
20        pipeline=pipeline,
21        conversation_flow=conversation_flow
22    )
23    try:
24        await context.connect()
25        await session.start()
26        await asyncio.Event().wait()
27    finally:
28        await session.close()
29        await context.shutdown()
30

Running and Testing the Agent

Step 5.1: Running the Python Script

To start your AI Voice Agent, run the following command in your terminal:
1python main.py
2

Step 5.2: Interacting with the Agent in the Playground

After starting the agent, you'll see a playground link in your console. Use this link to join the session and interact with your voice agent.

Advanced Features and Customizations

Extending Functionality with Custom Tools

The VideoSDK framework allows you to extend your agent's functionality by integrating custom tools. This can include additional plugins or custom logic to handle specific tasks.

Exploring Other Plugins

While this tutorial uses specific plugins for STT, LLM, and TTS, the VideoSDK framework supports a variety of options. Consider exploring other plugins to enhance your agent's capabilities.

Troubleshooting Common Issues

API Key and Authentication Errors

Ensure your API key is correctly configured in the .env file. Double-check for any typos or missing values.

Audio Input/Output Problems

Verify your microphone and speaker settings. Ensure they are correctly configured and accessible by your system.

Dependency and Version Conflicts

Check that all required packages are installed and compatible with your Python version. Use a virtual environment to manage dependencies effectively.

Conclusion

Summary of What You've Built

In this tutorial, you've built a fully functional AI Voice Agent capable of assisting with lease renewal processes. You've learned how to set up the development environment, create a custom agent, and run it using the VideoSDK framework.

Next Steps and Further Learning

Consider exploring additional features and plugins to enhance your agent's capabilities. The VideoSDK documentation provides further resources for advanced customizations and integrations.

Start Building With Free $20 Balance

No credit card required to start.

Want to level-up your learning? Subscribe now

Subscribe to our newsletter for more tech based insights

FAQ