Build a Conversational AI for eCommerce

Step-by-step guide to building a conversational AI voice agent for eCommerce using VideoSDK.

Introduction to AI Voice Agents in Conversational AI for eCommerce

What is an AI

Voice Agent

?

An AI

Voice Agent

 is a software application designed to interact with users through voice commands. These agents are capable of understanding spoken language, processing the information, and responding in a human-like manner. In the context of eCommerce, AI Voice Agents can assist customers in navigating online stores, answering product-related queries, and providing personalized shopping experiences.

Why are they important for the conversational AI for eCommerce industry?

AI Voice Agents play a crucial role in enhancing customer engagement and satisfaction. They offer hands-free navigation, quick access to information, and personalized recommendations, which are vital for improving the shopping experience. By integrating voice agents, eCommerce platforms can provide a seamless and interactive shopping journey, ultimately boosting sales and customer loyalty.

Core Components of a

Voice Agent

The core components of an AI

Voice Agent

 include:
  • Speech-to-Text (STT): Converts spoken language into text.
  • Language Model (LLM): Processes the text to understand and generate responses.
  • Text-to-Speech (TTS): Converts text responses back into spoken language.

What You’ll Build in This Tutorial

In this tutorial, you will learn how to build a conversational AI

voice agent

 tailored for eCommerce using the VideoSDK framework. This agent will be capable of handling customer queries, providing product information, and offering personalized recommendations.

Architecture and Core Concepts

High-Level Architecture Overview

The architecture of an AI Voice Agent involves several stages, from capturing user speech to generating a response. The process begins with capturing audio input, which is then converted into text using STT. The text is processed by a language model to generate an appropriate response, which is then converted back into speech using TTS. For a detailed understanding, you can explore the

AI voice Agent core components overview

.
Diagram

Understanding Key Concepts in the VideoSDK Framework

  • Agent: The core class representing your bot, responsible for handling interactions and managing the conversation flow.
  • CascadingPipeline: Defines the flow of audio processing, integrating STT, LLM, and TTS components. Learn more about the

    Cascading pipeline in AI voice Agents

    .
  • VAD & TurnDetector: Tools that help the agent determine when to listen and when to respond, ensuring smooth interaction. Specifically, the

    Silero Voice Activity Detection

     and the

    Turn detector for AI voice Agents

     are crucial for managing conversation flow.

Setting Up the Development Environment

Prerequisites

To get started, ensure you have Python 3.11+ installed and a VideoSDK account, which you can create at app.videosdk.live.

Step 1: Create a Virtual Environment

Create a virtual environment to manage dependencies:
1python -m venv venv
2source venv/bin/activate  # On Windows use `venv\Scripts\activate`
3

Step 2: Install Required Packages

Install the necessary packages using pip:
1pip install videosdk-agents videosdk-plugins
2

Step 3: Configure API Keys in a .env file

Create a .env file in your project directory and add your VideoSDK API key:
1VIDEOSDK_API_KEY=your_api_key_here
2

Building the AI Voice Agent: A Step-by-Step Guide

Here is the complete, runnable code for the AI Voice Agent:
1import asyncio, os
2from videosdk.agents import Agent, AgentSession, CascadingPipeline, JobContext, RoomOptions, WorkerJob, ConversationFlow
3from videosdk.plugins.silero import SileroVAD
4from videosdk.plugins.turn_detector import TurnDetector, pre_download_model
5from videosdk.plugins.deepgram import DeepgramSTT
6from videosdk.plugins.openai import OpenAILLM
7from videosdk.plugins.elevenlabs import ElevenLabsTTS
8from typing import AsyncIterator
9
10# Pre-downloading the Turn Detector model
11pre_download_model()
12
13agent_instructions = "You are a friendly and knowledgeable e-commerce assistant designed to enhance the shopping experience for users. Your primary role is to assist customers by providing product information, answering queries about orders, and offering personalized recommendations based on user preferences and browsing history. You can also help users navigate the website, track their orders, and provide information on promotions and discounts. However, you are not authorized to process payments or handle sensitive personal information. Always remind users to check the official website for the most accurate and up-to-date information. Your responses should be concise, informative, and engaging, ensuring a seamless and enjoyable shopping experience for the user."
14
15class MyVoiceAgent(Agent):
16    def __init__(self):
17        super().__init__(instructions=agent_instructions)
18    async def on_enter(self): await self.session.say("Hello! How can I help?")
19    async def on_exit(self): await self.session.say("Goodbye!")
20
21async def start_session(context: JobContext):
22    # Create agent and conversation flow
23    agent = MyVoiceAgent()
24    conversation_flow = ConversationFlow(agent)
25
26    # Create pipeline
27    pipeline = CascadingPipeline(
28        stt=DeepgramSTT(model="nova-2", language="en"),
29        llm=OpenAILLM(model="gpt-4o"),
30        tts=ElevenLabsTTS(model="eleven_flash_v2_5"),
31        vad=SileroVAD(threshold=0.35),
32        turn_detector=TurnDetector(threshold=0.8)
33    )
34
35    session = AgentSession(
36        agent=agent,
37        pipeline=pipeline,
38        conversation_flow=conversation_flow
39    )
40
41    try:
42        await context.connect()
43        await session.start()
44        # Keep the session running until manually terminated
45        await asyncio.Event().wait()
46    finally:
47        # Clean up resources when done
48        await session.close()
49        await context.shutdown()
50
51def make_context() -> JobContext:
52    room_options = RoomOptions(
53    #  room_id="YOUR_MEETING_ID",  # Set to join a pre-created room; omit to auto-create
54        name="VideoSDK Cascaded Agent",
55        playground=True
56    )
57
58    return JobContext(room_options=room_options)
59
60if __name__ == "__main__":
61    job = WorkerJob(entrypoint=start_session, jobctx=make_context)
62    job.start()
63

Step 4.1: Generating a VideoSDK Meeting ID

To generate a meeting ID, use the following curl command:
1curl -X POST https://api.videosdk.live/v1/meetings -H "Authorization: Bearer YOUR_API_KEY"
2
This command returns a meeting ID that you can use to create or join sessions.

Step 4.2: Creating the Custom Agent Class

The MyVoiceAgent class extends the Agent class and defines the behavior of the voice agent. It uses the agent_instructions to guide interactions and provides entry and exit messages to users.

Step 4.3: Defining the Core Pipeline

The CascadingPipeline is crucial as it integrates all the plugins required for the voice agent to function. It includes:
  • DeepgramSTT: Converts speech to text.
  • OpenAILLM: Processes text and generates responses.
  • ElevenLabsTTS: Converts text responses to speech.
  • SileroVAD: Detects when the user is speaking.
  • TurnDetector: Manages conversation turns.

Step 4.4: Managing the Session and Startup Logic

The start_session function initializes the agent session and starts the conversation flow. The make_context function sets up the room options, and the main block runs the agent using WorkerJob. For more information on managing sessions, refer to

AI voice Agent Sessions

.

Running and Testing the Agent

Step 5.1: Running the Python Script

Execute the script using:
1python main.py
2
This command starts the agent and outputs a playground link in the console.

Step 5.2: Interacting with the Agent in the Playground

Access the playground link to interact with your agent. You can speak to the agent, and it will respond based on the defined instructions and pipeline.

Advanced Features and Customizations

Extending Functionality with Custom Tools

You can extend the agent's functionality by integrating custom tools using the function_tool concept, allowing you to add more specific features to your agent.

Exploring Other Plugins

The VideoSDK framework supports various STT, LLM, and TTS plugins. Consider experimenting with other options to optimize your agent's performance.

Troubleshooting Common Issues

API Key and Authentication Errors

Ensure your API key is correctly configured in the .env file and that you have the necessary permissions.

Audio Input/Output Problems

Check your microphone and speaker settings to ensure they are functioning correctly.

Dependency and Version Conflicts

Ensure all dependencies are installed with compatible versions. Use a virtual environment to manage package versions effectively.

Conclusion

Summary of What You’ve Built

You have successfully built a conversational AI voice agent for eCommerce using the VideoSDK framework. This agent can handle customer interactions, providing a seamless shopping experience.

Next Steps and Further Learning

Explore additional features and plugins to enhance your agent's capabilities further. Consider diving deeper into the VideoSDK documentation for more advanced customizations.

Start Building With Free $20 Balance

No credit card required to start.

Want to level-up your learning? Subscribe now

Subscribe to our newsletter for more tech based insights

FAQ