Build an AI Voice Agent for Recruitment

Step-by-step guide to building an AI Voice Agent for recruitment using VideoSDK. Includes code and testing instructions.

Introduction to AI Voice Agents in ai voice agent for recruitment

What is an AI Voice Agent?

An AI Voice Agent is a software application that uses artificial intelligence to interact with users through voice commands. These agents can understand spoken language, process the information, and respond appropriately. They combine technologies like Speech-to-Text (STT), Natural Language Processing (NLP), and Text-to-Speech (TTS) to create a seamless conversational experience.

Why are they important for the ai voice agent for recruitment industry?

In the recruitment industry, AI Voice Agents can significantly streamline processes by handling initial candidate interactions, answering frequently asked questions, and scheduling interviews. This automation allows human recruiters to focus on more complex tasks, improving efficiency and candidate experience.

Core Components of a Voice Agent

  • STT (Speech-to-Text): Converts spoken language into text.
  • LLM (Large Language Model): Processes the text and generates appropriate responses.
  • TTS (Text-to-Speech): Converts the text response back into spoken language.

What You'll Build in This Tutorial

In this tutorial, you will build an AI Voice Agent tailored for recruitment tasks. The agent will be able to answer questions about job openings, guide candidates through the application process, and schedule interviews using the VideoSDK framework. To get started, refer to the

Voice Agent Quick Start Guide

.

Architecture and Core Concepts

High-Level Architecture Overview

The AI Voice Agent architecture involves several key components working together to process user input and generate responses. The process begins with capturing user speech, which is then converted to text using STT. The text is processed by an LLM to understand the user's intent and generate a response. Finally, the response is converted back to speech using TTS. For a detailed explanation, see the

AI voice Agent core components overview

.
Diagram

Understanding Key Concepts in the VideoSDK Framework

  • Agent: The core class representing your bot, responsible for managing interactions.
  • CascadingPipeline: Manages the flow of audio processing from STT to LLM to TTS. Learn more about the

    Cascading pipeline in AI voice Agents

    .
  • VAD & TurnDetector: These components help the agent determine when to listen and when to respond.

Setting Up the Development Environment

Prerequisites

To build this AI Voice Agent, you need Python 3.11+ and a VideoSDK account, which you can create at app.videosdk.live.

Step 1: Create a Virtual Environment

Create a virtual environment to manage dependencies:
1python -m venv venv
2source venv/bin/activate  # On Windows use `venv\\Scripts\\activate`
3

Step 2: Install Required Packages

Install the necessary packages using pip:
1pip install videosdk-agents videosdk-plugins-silero videosdk-plugins-deepgram videosdk-plugins-openai videosdk-plugins-elevenlabs
2

Step 3: Configure API Keys in a .env file

Create a .env file in your project directory and add your API keys:
1VIDEOSDK_API_KEY=your_videosdk_api_key
2DEEPGRAM_API_KEY=your_deepgram_api_key
3OPENAI_API_KEY=your_openai_api_key
4ELEVENLABS_API_KEY=your_elevenlabs_api_key
5

Building the AI Voice Agent: A Step-by-Step Guide

Here is the complete code to build your AI Voice Agent:
1import asyncio, os
2from videosdk.agents import Agent, AgentSession, CascadingPipeline, JobContext, RoomOptions, WorkerJob, ConversationFlow
3from videosdk.plugins.silero import SileroVAD
4from videosdk.plugins.turn_detector import TurnDetector, pre_download_model
5from videosdk.plugins.deepgram import DeepgramSTT
6from videosdk.plugins.openai import OpenAILLM
7from videosdk.plugins.elevenlabs import ElevenLabsTTS
8from typing import AsyncIterator
9
10# Pre-downloading the Turn Detector model
11pre_download_model()
12
13agent_instructions = "You are an AI Voice Agent specialized in recruitment. Your persona is that of a professional and approachable recruitment assistant. Your primary capabilities include answering questions about job openings, guiding candidates through the application process, and providing information about company culture and benefits. You can also schedule interviews and follow up with candidates on their application status. However, you are not authorized to make hiring decisions or provide personal opinions about candidates. Always remind users to refer to the official company website or contact HR for detailed information. Maintain confidentiality and adhere to data privacy regulations at all times."
14
15class MyVoiceAgent(Agent):
16    def __init__(self):
17        super().__init__(instructions=agent_instructions)
18    async def on_enter(self): await self.session.say("Hello! How can I help?")
19    async def on_exit(self): await self.session.say("Goodbye!")
20
21async def start_session(context: JobContext):
22    # Create agent and conversation flow
23    agent = MyVoiceAgent()
24    conversation_flow = ConversationFlow(agent)
25
26    # Create pipeline
27    pipeline = CascadingPipeline(
28        stt=DeepgramSTT(model="nova-2", language="en"),
29        llm=OpenAILLM(model="gpt-4o"),
30        tts=ElevenLabsTTS(model="eleven_flash_v2_5"),
31        vad=SileroVAD(threshold=0.35),
32        turn_detector=TurnDetector(threshold=0.8)
33    )
34
35    session = AgentSession(
36        agent=agent,
37        pipeline=pipeline,
38        conversation_flow=conversation_flow
39    )
40
41    try:
42        await context.connect()
43        await session.start()
44        # Keep the session running until manually terminated
45        await asyncio.Event().wait()
46    finally:
47        # Clean up resources when done
48        await session.close()
49        await context.shutdown()
50
51def make_context() -> JobContext:
52    room_options = RoomOptions(
53    #  room_id="YOUR_MEETING_ID",  # Set to join a pre-created room; omit to auto-create
54        name="VideoSDK Cascaded Agent",
55        playground=True
56    )
57
58    return JobContext(room_options=room_options)
59
60if __name__ == "__main__":
61    job = WorkerJob(entrypoint=start_session, jobctx=make_context)
62    job.start()
63

Step 4.1: Generating a VideoSDK Meeting ID

To generate a meeting ID, use the following curl command:
1curl -X POST "https://api.videosdk.live/v1/meetings" \
2-H "Content-Type: application/json" \
3-H "Authorization: Bearer YOUR_VIDEOSDK_API_KEY"
4

Step 4.2: Creating the Custom Agent Class

The MyVoiceAgent class extends the Agent class from the VideoSDK framework. It defines the agent's behavior when entering or exiting a session. This is where you specify the agent's persona and capabilities.
1class MyVoiceAgent(Agent):
2    def __init__(self):
3        super().__init__(instructions=agent_instructions)
4    async def on_enter(self): await self.session.say("Hello! How can I help?")
5    async def on_exit(self): await self.session.say("Goodbye!")
6

Step 4.3: Defining the Core Pipeline

The CascadingPipeline is crucial for processing user input and generating responses. It connects the STT, LLM, TTS, VAD, and TurnDetector plugins. For more information, check out the

ElevenLabs TTS Plugin for voice agent

 and

Deepgram STT Plugin for voice agent

.
1pipeline = CascadingPipeline(
2    stt=DeepgramSTT(model="nova-2", language="en"),
3    llm=OpenAILLM(model="gpt-4o"),
4    tts=ElevenLabsTTS(model="eleven_flash_v2_5"),
5    vad=SileroVAD(threshold=0.35),
6    turn_detector=TurnDetector(threshold=0.8)
7)
8

Step 4.4: Managing the Session and Startup Logic

The start_session function initializes the agent and starts the session. The make_context function sets up the room options for the agent. For more details on managing sessions, refer to

AI voice Agent Sessions

.
1async def start_session(context: JobContext):
2    # Create agent and conversation flow
3    agent = MyVoiceAgent()
4    conversation_flow = ConversationFlow(agent)
5
6    # Create pipeline
7    pipeline = CascadingPipeline(
8        stt=DeepgramSTT(model="nova-2", language="en"),
9        llm=OpenAILLM(model="gpt-4o"),
10        tts=ElevenLabsTTS(model="eleven_flash_v2_5"),
11        vad=SileroVAD(threshold=0.35),
12        turn_detector=TurnDetector(threshold=0.8)
13    )
14
15    session = AgentSession(
16        agent=agent,
17        pipeline=pipeline,
18        conversation_flow=conversation_flow
19    )
20
21    try:
22        await context.connect()
23        await session.start()
24        # Keep the session running until manually terminated
25        await asyncio.Event().wait()
26    finally:
27        # Clean up resources when done
28        await session.close()
29        await context.shutdown()
30
31def make_context() -> JobContext:
32    room_options = RoomOptions(
33    #  room_id="YOUR_MEETING_ID",  # Set to join a pre-created room; omit to auto-create
34        name="VideoSDK Cascaded Agent",
35        playground=True
36    )
37
38    return JobContext(room_options=room_options)
39
40if __name__ == "__main__":
41    job = WorkerJob(entrypoint=start_session, jobctx=make_context)
42    job.start()
43

Running and Testing the Agent

Step 5.1: Running the Python Script

To run your agent, execute the Python script:
1python main.py
2

Step 5.2: Interacting with the Agent in the Playground

Once the agent is running, use the

AI Agent playground

 link provided in the console to interact with your AI Voice Agent. Join the session and test the agent's capabilities in a real-time environment.

Advanced Features and Customizations

Extending Functionality with Custom Tools

The VideoSDK framework allows you to extend the agent's functionality by integrating custom tools. This can include additional data sources or processing capabilities tailored to your specific needs.

Exploring Other Plugins

While this tutorial uses specific plugins for STT, LLM, and TTS, the VideoSDK framework supports various options. Explore other plugins to enhance your agent's performance and capabilities, such as the

Silero Voice Activity Detection

.

Troubleshooting Common Issues

API Key and Authentication Errors

Ensure your API keys are correctly configured in the .env file. Double-check for any typos or missing values.

Audio Input/Output Problems

Verify that your microphone and speakers are functioning correctly. Check system settings and permissions to ensure proper audio input and output.

Dependency and Version Conflicts

Ensure all installed packages are compatible with Python 3.11+. Use a virtual environment to manage dependencies and avoid conflicts.

Conclusion

Summary of What You've Built

In this tutorial, you've built a fully functional AI Voice Agent for recruitment using the VideoSDK framework. The agent can handle various recruitment tasks, providing a seamless experience for both recruiters and candidates.

Next Steps and Further Learning

Explore additional features and customizations to enhance your AI Voice Agent. Consider integrating more advanced NLP capabilities or connecting the agent to external data sources for richer interactions.

Start Building With Free $20 Balance

No credit card required to start.

Want to level-up your learning? Subscribe now

Subscribe to our newsletter for more tech based insights

FAQ