Build an AI Voice Assistant for Lease Renewal

Step-by-step guide to building an AI voice assistant for lease renewal using VideoSDK.

Introduction to AI Voice Agents in Lease Renewal

AI Voice Agents are automated systems that interact with users through voice commands, providing assistance and information. In the context of lease renewal, these agents can streamline processes by offering guidance on lease terms, renewal deadlines, and necessary documentation.

What is an AI

Voice Agent

?

An AI

Voice Agent

 is a software application capable of understanding and responding to human speech using technologies like Speech-to-Text (STT), Language Models (LLM), and Text-to-Speech (TTS). These agents are designed to handle specific tasks, such as customer service or information retrieval.

Why are they important for the Lease Renewal Industry?

In the lease renewal industry, AI Voice Agents can significantly reduce the workload of property managers by automating routine inquiries and guiding tenants through the renewal process. This includes explaining lease terms, deadlines, and required documentation, making the process more efficient and less error-prone.

Core Components of a

Voice Agent

  • STT (Speech-to-Text): Converts spoken language into text.
  • LLM (Large Language Model): Processes text input to generate a response.
  • TTS (Text-to-Speech): Converts text responses back into speech.

What You'll Build in This Tutorial

In this tutorial, you will build a fully functional AI Voice Assistant for lease renewal using the VideoSDK framework. This agent will assist users by providing information and guidance on lease renewal processes.

Architecture and Core Concepts

High-Level Architecture Overview

The AI

Voice Agent

 processes user speech through a series of steps: capturing audio, converting it to text, generating a response, and then converting that response back to speech.
Diagram

Understanding Key Concepts in the VideoSDK Framework

  • Agent: The core class representing your bot, responsible for handling user interactions.

  • Cascading Pipeline in AI voice Agents

    :     Manages the flow of audio processing, integrating STT, LLM, and TTS.
  • VAD & TurnDetector: These components help the agent determine when to listen and respond.

Setting Up the Development Environment

Prerequisites

To get started, ensure you have Python 3.11+ installed and a VideoSDK account. Sign up at app.videosdk.live.

Step 1: Create a Virtual Environment

Create a virtual environment to manage dependencies:
1python -m venv lease-renewal-env
2source lease-renewal-env/bin/activate  # On Windows use `lease-renewal-env\Scripts\activate`
3

Step 2: Install Required Packages

Install the necessary Python packages:
1pip install videosdk-agents videosdk-plugins
2

Step 3: Configure API Keys in a .env file

Create a .env file in your project directory and add your VideoSDK API keys:
1VIDEOSDK_API_KEY=your_api_key_here
2

Building the AI Voice Agent: A Step-by-Step Guide

Below is the complete, runnable code for the AI Voice Agent. We will break it down into smaller parts for detailed explanations:
1import asyncio, os
2from videosdk.agents import Agent, AgentSession, CascadingPipeline, JobContext, RoomOptions, WorkerJob, ConversationFlow
3from videosdk.plugins.silero import SileroVAD
4from videosdk.plugins.turn_detector import TurnDetector, pre_download_model
5from videosdk.plugins.deepgram import DeepgramSTT
6from videosdk.plugins.openai import OpenAILLM
7from videosdk.plugins.elevenlabs import ElevenLabsTTS
8from typing import AsyncIterator
9
10# Pre-downloading the Turn Detector model
11pre_download_model()
12
13agent_instructions = "You are a knowledgeable and efficient AI Voice Assistant specialized in lease renewal processes. Your primary role is to assist tenants and landlords by providing information and guidance on lease renewal procedures. You can answer questions related to lease terms, renewal deadlines, and required documentation. You can also guide users through the steps of renewing a lease, including explaining any legal considerations and potential changes in lease agreements. However, you are not a legal advisor and must inform users to consult a legal professional for specific legal advice. You should maintain a friendly and professional tone, ensuring clarity and accuracy in your responses. Your goal is to make the lease renewal process as smooth and understandable as possible for all parties involved."
14
15class MyVoiceAgent(Agent):
16    def __init__(self):
17        super().__init__(instructions=agent_instructions)
18    async def on_enter(self): await self.session.say("Hello! How can I help?")
19    async def on_exit(self): await self.session.say("Goodbye!")
20
21async def start_session(context: JobContext):
22    # Create agent and conversation flow
23    agent = MyVoiceAgent()
24    conversation_flow = ConversationFlow(agent)
25
26    # Create pipeline
27    pipeline = CascadingPipeline(
28        stt=DeepgramSTT(model="nova-2", language="en"),
29        llm=[OpenAI LLM Plugin for voice agent](https://docs.videosdk.live/ai_agents/plugins/llm/openai)(model="gpt-4o"),
30        tts=ElevenLabsTTS(model="eleven_flash_v2_5"),
31        vad=[Silero Voice Activity Detection](https://docs.videosdk.live/ai_agents/plugins/silero-vad)(threshold=0.35),
32        turn_detector=[Turn detector for AI voice Agents](https://docs.videosdk.live/ai_agents/plugins/turn-detector)(threshold=0.8)
33    )
34
35    session = [AI voice Agent Sessions](https://docs.videosdk.live/ai_agents/core-components/agent-session)(
36        agent=agent,
37        pipeline=pipeline,
38        conversation_flow=conversation_flow
39    )
40
41    try:
42        await context.connect()
43        await session.start()
44        # Keep the session running until manually terminated
45        await asyncio.Event().wait()
46    finally:
47        # Clean up resources when done
48        await session.close()
49        await context.shutdown()
50
51def make_context() -> JobContext:
52    room_options = RoomOptions(
53    #  room_id="YOUR_MEETING_ID",  # Set to join a pre-created room; omit to auto-create
54        name="VideoSDK Cascaded Agent",
55        playground=True
56    )
57
58    return JobContext(room_options=room_options)
59
60if __name__ == "__main__":
61    job = WorkerJob(entrypoint=start_session, jobctx=make_context)
62    job.start()
63

Step 4.1: Generating a VideoSDK Meeting ID

To interact with the agent, you need a meeting ID. You can generate one using the VideoSDK API:
1curl -X POST "https://api.videosdk.live/v1/rooms" \
2-H "Authorization: Bearer YOUR_API_KEY" \
3-H "Content-Type: application/json" \
4-d '{"name": "Lease Renewal Meeting"}'
5

Step 4.2: Creating the Custom Agent Class

The MyVoiceAgent class inherits from the Agent class. It defines the behavior of the agent when entering and exiting a session:
1class MyVoiceAgent(Agent):
2    def __init__(self):
3        super().__init__(instructions=agent_instructions)
4    async def on_enter(self): await self.session.say("Hello! How can I help?")
5    async def on_exit(self): await self.session.say("Goodbye!")
6

Step 4.3: Defining the Core Pipeline

The CascadingPipeline integrates various plugins to process audio input and output:
1pipeline = CascadingPipeline(
2    stt=DeepgramSTT(model="nova-2", language="en"),
3    llm=OpenAILLM(model="gpt-4o"),
4    tts=ElevenLabsTTS(model="eleven_flash_v2_5"),
5    vad=SileroVAD(threshold=0.35),
6    turn_detector=TurnDetector(threshold=0.8)
7)
8
  • STT (DeepgramSTT): Converts the user's speech to text.
  • LLM (OpenAILLM): Processes the text to generate a response.
  • TTS (ElevenLabsTTS): Converts the response text back to speech.
  • VAD (SileroVAD): Detects when the user is speaking.
  • TurnDetector: Determines when the agent should respond.

Step 4.4: Managing the Session and Startup Logic

The start_session function initializes the agent session and manages its lifecycle:
1async def start_session(context: JobContext):
2    agent = MyVoiceAgent()
3    conversation_flow = ConversationFlow(agent)
4    pipeline = CascadingPipeline(
5        stt=DeepgramSTT(model="nova-2", language="en"),
6        llm=OpenAILLM(model="gpt-4o"),
7        tts=ElevenLabsTTS(model="eleven_flash_v2_5"),
8        vad=SileroVAD(threshold=0.35),
9        turn_detector=TurnDetector(threshold=0.8)
10    )
11    session = AgentSession(
12        agent=agent,
13        pipeline=pipeline,
14        conversation_flow=conversation_flow
15    )
16    try:
17        await context.connect()
18        await session.start()
19        await asyncio.Event().wait()
20    finally:
21        await session.close()
22        await context.shutdown()
23
The make_context function sets up the room options:
1def make_context() -> JobContext:
2    room_options = RoomOptions(
3        name="VideoSDK Cascaded Agent",
4        playground=True
5    )
6    return JobContext(room_options=room_options)
7

Running and Testing the Agent

Step 5.1: Running the Python Script

Run the script using Python:
1python main.py
2

Step 5.2: Interacting with the Agent in the Playground

Once the script is running, you will see a playground link in the console. Use this link to join the session and interact with the agent. Speak your queries, and the agent will respond with relevant information about lease renewal.

Advanced Features and Customizations

Extending Functionality with Custom Tools

You can extend the agent's capabilities by integrating custom tools. These tools can handle specific tasks or queries, enhancing the agent's functionality.

Exploring Other Plugins

The VideoSDK framework supports various plugins for STT, LLM, and TTS. Explore options like Cartesia for STT or Google Gemini for LLM to customize your agent further.

Troubleshooting Common Issues

API Key and Authentication Errors

Ensure your API keys are correctly configured in the .env file. Double-check the authorization headers in your API requests.

Audio Input/Output Problems

Verify your microphone and speaker settings. Ensure the correct audio devices are selected.

Dependency and Version Conflicts

Use a virtual environment to manage dependencies and avoid version conflicts. Ensure all required packages are installed.

Conclusion

Summary of What You've Built

You have successfully built an AI Voice Assistant for lease renewal using the VideoSDK framework. This agent can assist users by providing information and guidance on lease renewal processes.

Next Steps and Further Learning

Explore additional features and plugins to enhance your agent. Consider integrating more complex workflows or expanding the agent's capabilities to cover other real estate processes.

Start Building With Free $20 Balance

No credit card required to start.

Want to level-up your learning? Subscribe now

Subscribe to our newsletter for more tech based insights

FAQ