Build an AI Voice Assistant for Real Estate

Step-by-step guide to building an AI voice assistant for real estate using VideoSDK.

Introduction to AI Voice Agents in Real Estate

In recent years, AI voice agents have become an integral part of various industries, and the real estate sector is no exception. These intelligent systems are designed to interact with users through natural language, providing information and assistance in a conversational manner.

What is an AI

Voice Agent

?

An AI

voice agent

 is a software application that uses artificial intelligence to understand and respond to voice commands. It leverages technologies such as Speech-to-Text (STT), Language Learning Models (LLM), and Text-to-Speech (TTS) to process user input, generate responses, and communicate them back to the user.

Why are they important for the Real Estate Industry?

In the real estate industry, AI voice agents can revolutionize how potential buyers and sellers interact with property listings. They can provide instant information about properties, answer questions about buying or renting, and even schedule viewings. This not only enhances user experience but also increases efficiency for real estate professionals.

Core Components of a

Voice Agent

  • Speech-to-Text (STT): Converts spoken language into text.
  • Language Learning Model (LLM): Processes the text to understand and generate appropriate responses.
  • Text-to-Speech (TTS): Converts the generated text response back into speech.
For a comprehensive understanding of these components, refer to the

AI voice Agent core components overview

.

What You'll Build in This Tutorial

In this tutorial, you will learn how to build a fully functional AI voice assistant tailored for the real estate industry using the VideoSDK framework. We will guide you through setting up the development environment, constructing the

voice agent

, and testing it in a real-world scenario.

Architecture and Core Concepts

Creating an AI

voice agent

 involves understanding its architecture and the flow of data from user input to agent response.

High-Level Architecture Overview

The

voice agent

 architecture involves several components working together. When a user speaks, their voice is captured and converted into text using STT. The text is then processed by an LLM to generate a response, which is finally converted back into speech using TTS.
Diagram

Understanding Key Concepts in the VideoSDK Framework

  • Agent: This is the core class representing your AI voice agent. It defines how the agent behaves and interacts with users.
  • CascadingPipeline: This component manages the flow of audio processing, integrating STT, LLM, and TTS. Learn more about the

    Cascading pipeline in AI voice Agents

    .
  • VAD & TurnDetector: These tools help the agent determine when to listen and when to speak, ensuring smooth interaction. For more details, see the

    Turn detector for AI voice Agents

    .

Setting Up the Development Environment

Before building the AI voice assistant, ensure your development environment is properly configured.

Prerequisites

To get started, you will need:
  • Python 3.11 or higher
  • A VideoSDK account (sign up at app.videosdk.live)

Step 1: Create a Virtual Environment

Create a virtual environment to manage your project dependencies:
1python -m venv venv
2source venv/bin/activate  # On Windows use `venv\\Scripts\\activate`
3

Step 2: Install Required Packages

Install the necessary packages using pip:
1pip install videosdk
2pip install python-dotenv
3

Step 3: Configure API Keys in a .env File

Create a .env file in your project directory and add your VideoSDK API key:
1VIDEOSDK_API_KEY=your_api_key_here
2

Building the AI Voice Agent: A Step-by-Step Guide

To build the AI voice agent, we will use the VideoSDK framework. Below is the complete, runnable code:
1import asyncio, os
2from videosdk.agents import Agent, AgentSession, CascadingPipeline, JobContext, RoomOptions, WorkerJob, ConversationFlow
3from videosdk.plugins.silero import SileroVAD
4from videosdk.plugins.turn_detector import TurnDetector, pre_download_model
5from videosdk.plugins.deepgram import DeepgramSTT
6from videosdk.plugins.openai import OpenAILLM
7from videosdk.plugins.elevenlabs import ElevenLabsTTS
8from typing import AsyncIterator
9
10# Pre-downloading the Turn Detector model
11pre_download_model()
12
13agent_instructions = "You are a knowledgeable and friendly AI Voice Assistant specialized in the real estate industry. Your primary role is to assist users by providing information about real estate properties, answering queries related to buying, selling, and renting properties, and offering insights into market trends. You can also help schedule property viewings and connect users with real estate agents. However, you are not a licensed real estate agent, and you must inform users to consult with a professional for legal or financial advice. You should always prioritize user privacy and adhere to data protection regulations."
14
15class MyVoiceAgent(Agent):
16    def __init__(self):
17        super().__init__(instructions=agent_instructions)
18    async def on_enter(self): await self.session.say("Hello! How can I help?")
19    async def on_exit(self): await self.session.say("Goodbye!")
20
21async def start_session(context: JobContext):
22    # Create agent and conversation flow
23    agent = MyVoiceAgent()
24    conversation_flow = ConversationFlow(agent)
25
26    # Create pipeline
27    pipeline = CascadingPipeline(
28        stt=DeepgramSTT(model="nova-2", language="en"),
29        llm=OpenAILLM(model="gpt-4o"),
30        tts=ElevenLabsTTS(model="eleven_flash_v2_5"),
31        vad=SileroVAD(threshold=0.35),
32        turn_detector=TurnDetector(threshold=0.8)
33    )
34
35    session = AgentSession(
36        agent=agent,
37        pipeline=pipeline,
38        conversation_flow=conversation_flow
39    )
40
41    try:
42        await context.connect()
43        await session.start()
44        # Keep the session running until manually terminated
45        await asyncio.Event().wait()
46    finally:
47        # Clean up resources when done
48        await session.close()
49        await context.shutdown()
50
51def make_context() -> JobContext:
52    room_options = RoomOptions(
53    #  room_id="YOUR_MEETING_ID",  # Set to join a pre-created room; omit to auto-create
54        name="VideoSDK Cascaded Agent",
55        playground=True
56    )
57
58    return JobContext(room_options=room_options)
59
60if __name__ == "__main__":
61    job = WorkerJob(entrypoint=start_session, jobctx=make_context)
62    job.start()
63

Step 4.1: Generating a VideoSDK Meeting ID

To interact with the AI voice agent, you need a meeting ID. Use the following curl command to generate one:
1curl -X POST \\
2  https://api.videosdk.live/v1/meetings \\
3  -H "Authorization: Bearer YOUR_API_KEY" \\
4  -H "Content-Type: application/json"
5

Step 4.2: Creating the Custom Agent Class

The MyVoiceAgent class extends the Agent class from the VideoSDK framework. It defines the agent's behavior upon entering and exiting a session:
1class MyVoiceAgent(Agent):
2    def __init__(self):
3        super().__init__(instructions=agent_instructions)
4    async def on_enter(self): await self.session.say("Hello! How can I help?")
5    async def on_exit(self): await self.session.say("Goodbye!")
6
This class is where you can customize the agent's instructions and responses.

Step 4.3: Defining the Core Pipeline

The CascadingPipeline is crucial as it integrates various plugins for STT, LLM, TTS, VAD, and turn detection:
1pipeline = CascadingPipeline(
2    stt=DeepgramSTT(model="nova-2", language="en"),
3    llm=OpenAILLM(model="gpt-4o"),
4    tts=ElevenLabsTTS(model="eleven_flash_v2_5"),
5    vad=SileroVAD(threshold=0.35),
6    turn_detector=TurnDetector(threshold=0.8)
7)
8
Each component plays a vital role in processing audio and generating responses.

Step 4.4: Managing the Session and Startup Logic

The start_session function manages the session lifecycle, while make_context sets up the environment:
1async def start_session(context: JobContext):
2    agent = MyVoiceAgent()
3    conversation_flow = ConversationFlow(agent)
4    pipeline = CascadingPipeline(
5        stt=DeepgramSTT(model="nova-2", language="en"),
6        llm=OpenAILLM(model="gpt-4o"),
7        tts=ElevenLabsTTS(model="eleven_flash_v2_5"),
8        vad=SileroVAD(threshold=0.35),
9        turn_detector=TurnDetector(threshold=0.8)
10    )
11    session = AgentSession(
12        agent=agent,
13        pipeline=pipeline,
14        conversation_flow=conversation_flow
15    )
16    try:
17        await context.connect()
18        await session.start()
19        await asyncio.Event().wait()
20    finally:
21        await session.close()
22        await context.shutdown()
23
24def make_context() -> JobContext:
25    room_options = RoomOptions(
26        name="VideoSDK Cascaded Agent",
27        playground=True
28    )
29    return JobContext(room_options=room_options)
30
31if __name__ == "__main__":
32    job = WorkerJob(entrypoint=start_session, jobctx=make_context)
33    job.start()
34
For more detailed information on managing sessions, refer to

AI voice Agent Sessions

.

Running and Testing the Agent

Step 5.1: Running the Python Script

With your development environment set up and code ready, run the script:
1python main.py
2

Step 5.2: Interacting with the Agent in the Playground

After running the script, look for the playground link in the console output. Open it in a browser to join the session and interact with your AI voice agent.

Advanced Features and Customizations

Extending Functionality with Custom Tools

You can extend the functionality of your AI voice agent by integrating custom tools. This allows for more specialized tasks and responses.

Exploring Other Plugins

The VideoSDK framework supports various plugins for STT, LLM, and TTS. Experiment with different options to find the best fit for your use case.

Troubleshooting Common Issues

API Key and Authentication Errors

Ensure your API key is correctly set in the .env file and matches your VideoSDK account.

Audio Input/Output Problems

Check your microphone and speaker settings to ensure they are configured correctly.

Dependency and Version Conflicts

Verify that all dependencies are installed and compatible with your Python version.

Conclusion

Summary of What You've Built

You have successfully built an AI voice assistant tailored for the real estate industry using the VideoSDK framework.

Next Steps and Further Learning

Explore additional features and plugins to enhance your AI voice agent. Consider integrating with other services for a more robust solution.

Start Building With Free $20 Balance

No credit card required to start.

Want to level-up your learning? Subscribe now

Subscribe to our newsletter for more tech based insights

FAQ