Introduction

Voice-first applications are reshaping how users interact with software, from virtual assistants to real-time transcription and telehealth solutions. For developers building high-performance desktop, IoT, or gaming applications, integrating these capabilities often requires a robust c++ voice api. C++ offers unparalleled control over system resources and latency, making it the ideal choice for demanding audio processing tasks. VideoSDK provides a comprehensive platform that bridges the gap, offering both real-time communication (RTC) capabilities and advanced AI Voice Agents. By leveraging VideoSDK's REST APIs and AI infrastructure, C++ developers can seamlessly embed voice-driven features into their native applications without compromising on performance.

Why Choose a C++ Voice API?

When it comes to audio processing and real-time communication, C++ remains the gold standard for performance-critical applications. A dedicated c++ voice api allows developers to tap into low-latency streaming and real-time voice communication C++ library features directly within their existing codebases. Unlike higher-level languages, C++ provides deterministic memory management and direct access to audio hardware, minimizing overhead. A well-designed Voice API leverages these advantages to deliver real-time audio processing. Furthermore, a cross-platform audio SDK ensures that your voice applications run consistently across Windows, Linux, and macOS. Whether you are building an on-device speech AI C++ solution or integrating a WebRTC C++ SDK, C++ guarantees the speed and reliability required for seamless user experiences.

Core Concepts of VideoSDK for Voice

VideoSDK's architecture is built around a Rooms-based model, initially designed for its flagship Video Calling API & SDKs and Audio Calling & Voice Rooms. This same architecture extends to its AI Voice Agents and REST APIs, making it a versatile choice for a c++ voice api integration.
When using VideoSDK's Voice API, you interact with two primary product surfaces:
  1. REST APIs & Server-Side Orchestration: Developer-friendly HTTP endpoints for room management, participant control, and session analytics. From C++, you can use libraries like libcurl or cpprestsdk to call these endpoints.
  2. AI Voice Agents: A Python-based AI Agent SDK that connects LLMs, Speech-to-Text (STT), and Text-to-Text (TTS) providers to VideoSDK rooms. While the agent itself runs in Python, your C++ application can interact with the agent by joining the same VideoSDK Room as a participant and exchanging media streams, or by orchestrating the agent via REST APIs.
Key terminology to understand:
  • Room: The virtual space where participants and AI agents interact.
  • Participant: An entity (user or AI) connected to the room.
  • Agent: The AI worker process managing the conversation pipeline.
  • Token: A validation string required to join a room or authenticate API requests.
  • Stream: The audio/media data flowing between participants.

Setting Up the Development Environment

To integrate the c++ voice api using VideoSDK's REST endpoints, you need a modern C++ environment.
Prerequisites:
  • A C++17 compatible compiler (GCC, Clang, or MSVC)
  • CMake (version 3.15 or higher)
  • libcurl or cpprestsdk for HTTP requests
  • OpenSSL for TLS/SSL support
  • nlohmann/json for JSON parsing
You can install these dependencies on Ubuntu using apt:
1sudo apt-get update
2sudo apt-get install build-essential cmake libcurl4-openssl-dev libssl-dev nlohmann-json3-dev
3
Here is a sample CMakeLists.txt snippet to configure your project:
1cmake_minimum_required(VERSION 3.15)
2project(VideoSDKVoiceAPI)
3
4set(CMAKE_CXX_STANDARD 17)
5set(CMAKE_CXX_STANDARD_REQUIRED ON)
6
7find_package(CURL REQUIRED)
8find_package(OpenSSL REQUIRED)
9find_package(nlohmann_json REQUIRED)
10
11add_executable(voice_app main.cpp)
12
13target_link_libraries(voice_app PRIVATE CURL::libcurl OpenSSL::SSL nlohmann_json::nlohmann_json)
14
This configuration prepares your project to communicate with the Voice API.

Authenticating and Creating a Voice Session

Authentication is the first step in using the c++ voice api. You need a VideoSDK access token to create rooms and manage sessions. Typically, you generate this token on your backend server using your VideoSDK API Key and Secret. However, for demonstration purposes, we will show a C++ function that requests a token from your server.
1#include <iostream>
2#include <string>
3#include <curl/curl.h>
4#include <nlohmann/json.hpp>
5
6using json = nlohmann::json;
7
8size_t WriteCallback(void* contents, size_t size, size_t nmemb, std::string* userp) {
9    userp->append((char*)contents, size * nmemb);
10    return size * nmemb;
11}
12
13std::string fetchVideoSDKToken(const std::string& serverUrl) {
14    CURL* curl;
15    CURLcode res;
16    std::string readBuffer;
17
18    curl = curl_easy_init();
19    if(curl) {
20        struct curl_slist* headers = NULL;
21        headers = curl_slist_append(headers, "Content-Type: application/json");
22        
23        json payload = {
24            {"roomId", ""},
25            {"permissions", {"allow_join", "allow_mod"}}
26        };
27        std::string jsonStr = payload.dump();
28
29        curl_easy_setopt(curl, CURLOPT_URL, serverUrl.c_str());
30        curl_easy_setopt(curl, CURLOPT_POST, 1L);
31        curl_easy_setopt(curl, CURLOPT_HTTPHEADER, headers);
32        curl_easy_setopt(curl, CURLOPT_POSTFIELDS, jsonStr.c_str());
33        curl_easy_setopt(curl, CURLOPT_WRITEFUNCTION, WriteCallback);
34        curl_easy_setopt(curl, CURLOPT_WRITEDATA, &readBuffer);
35
36        res = curl_easy_perform(curl);
37        if(res != CURLE_OK) {
38            std::cerr << "curl_easy_perform() failed: " << curl_easy_strerror(res) << std::endl;
39        } else {
40            auto responseJson = json::parse(readBuffer);
41            return responseJson["token"].get<std::string>();
42        }
43        curl_easy_cleanup(curl);
44    }
45    return "";
46}
47
Once you have the token, you can initialize a Voice Agent session by creating a Room via the VideoSDK REST API and configuring your AI Agent parameters.

Real-Time Speech-to-Text with VideoSDK

Integrating real-time transcription into your C++ application involves streaming audio data to VideoSDK's transcription endpoint or routing it through a VideoSDK Room where an AI Voice Agent is listening. Using the c++ voice api, you can stream a PCM audio buffer using libcurl's multi-handle for non-blocking I/O.
Here is a conceptual flow of how audio is processed:
1flowchart LR
2    A[Audio Capture C++] --> B[PCM Buffer 16kHz Mono]
3    B --> C[HTTP Stream libcurl]
4    C --> D[VideoSDK Transcription]
5    D --> E[Interim Results JSON]
6    D --> F[Final Results JSON]
7    E --> G[UI Update]
8    F --> G
9
To stream audio, you configure a CURLOPT_READFUNCTION to continuously feed your audio buffer:
1// Conceptual code for streaming audio via libcurl
2size_t read_audio_callback(char* ptr, size_t size, size_t nmemb, void* userdata) {
3    AudioBuffer* buffer = static_cast<AudioBuffer*>(userdata);
4    size_t bytes_to_read = size * nmemb;
5    size_t available = buffer->getAvailableBytes();
6    
7    if (available == 0) {
8        return 0; // No data available
9    }
10    
11    size_t read = std::min(bytes_to_read, available);
12    memcpy(ptr, buffer->getData(), read);
13    buffer->consume(read);
14    return read;
15}
16
17void startTranscriptionStream(const std::string& token, const std::string& roomId) {
18    CURL* curl = curl_easy_init();
19    if(curl) {
20        std::string url = "https://api.videosdk.live/v2/rooms/" + roomId + "/transcription";
21        struct curl_slist* headers = NULL;
22        headers = curl_slist_append(headers, ("Authorization: " + token).c_str());
23        headers = curl_slist_append(headers, "Content-Type: application/octet-stream");
24
25        AudioBuffer audioBuffer; // Your custom audio buffer class
26
27        curl_easy_setopt(curl, CURLOPT_URL, url.c_str());
28        curl_easy_setopt(curl, CURLOPT_POST, 1L);
29        curl_easy_setopt(curl, CURLOPT_HTTPHEADER, headers);
30        curl_easy_setopt(curl, CURLOPT_READFUNCTION, read_audio_callback);
31        curl_easy_setopt(curl, CURLOPT_READDATA, &audioBuffer);
32        curl_easy_setopt(curl, CURLOPT_UPLOAD_BUFFERSIZE, 32000L);
33        
34        // Execute in a separate thread or use multi-handle for async
35        CURLcode res = curl_easy_perform(curl);
36        // Handle response...
37        curl_easy_cleanup(curl);
38    }
39}
40
Handling interim and final results requires parsing the JSON responses sent back from the server, updating your UI as the user speaks.

Text-to-Speech and Voice Cloning

To generate speech from text, you can call the TTS endpoint from C++. VideoSDK's AI Voice Agents support various TTS providers like ElevenLabs, OpenAI TTS, and Azure Speech. By building a CreateTTSRequest JSON payload, you can synthesize audio and even utilize voice cloning by passing a custom voice ID.
1sequenceDiagram
2    participant C++ App
3    participant VideoSDK API
4    participant TTS Provider
5    C++ App->>VideoSDK API: POST /tts (Text, Voice ID)
6    VideoSDK API->>TTS Provider: Synthesize Request
7    TTS Provider-->>VideoSDK API: Audio Payload
8    VideoSDK API-->>C++ App: Audio File (WAV/MP3)
9    C++ App->>C++ App: Playback Audio
10
Here is an example of calling the TTS endpoint:
1void generateSpeech(const std::string& token, const std::string& text, const std::string& voiceId) {
2    CURL* curl = curl_easy_init();
3    if(curl) {
4        std::string url = "https://api.videosdk.live/v2/ai/tts";
5        struct curl_slist* headers = NULL;
6        headers = curl_slist_append(headers, ("Authorization: " + token).c_str());
7        headers = curl_slist_append(headers, "Content-Type: application/json");
8
9        json payload = {
10            {"text", text},
11            {"voiceId", voiceId}, // Use custom voice ID for cloning
12            {"format", "wav"}
13        };
14        std::string jsonStr = payload.dump();
15
16        std::string readBuffer;
17        curl_easy_setopt(curl, CURLOPT_URL, url.c_str());
18        curl_easy_setopt(curl, CURLOPT_POST, 1L);
19        curl_easy_setopt(curl, CURLOPT_HTTPHEADER, headers);
20        curl_easy_setopt(curl, CURLOPT_POSTFIELDS, jsonStr.c_str());
21        curl_easy_setopt(curl, CURLOPT_WRITEFUNCTION, WriteCallback);
22        curl_easy_setopt(curl, CURLOPT_WRITEDATA, &readBuffer);
23
24        CURLcode res = curl_easy_perform(curl);
25        if(res == CURLE_OK) {
26            // Save readBuffer as a .wav file and play it
27            std::ofstream out("output.wav", std::ios::binary);
28            out.write(readBuffer.c_str(), readBuffer.size());
29            out.close();
30        }
31        curl_easy_cleanup(curl);
32    }
33}
34

Advanced Features: Voice Agent Tools

VideoSDK's AI Voice Agents come with advanced capabilities like Function Tools, Context Management, and Multi-Agent Switching. You can configure these via API calls from your C++ backend. For instance, if your voice agent needs to check a user's calendar, you can define a function tool that the agent can invoke.
1json createToolPayload() {
2    return {
3        {"name", "check_calendar"},
4        {"description", "Checks the user's calendar for available slots"},
5        {"parameters", {
6            {"type", "object"},
7            {"properties", {
8                {"date", {{"type", "string"}, {"description", "Date to check in YYYY-MM-DD format"}}}
9            }},
10            {"required", {"date"}}
11        }}
12    };
13}
14
When the agent determines it needs to check the calendar, it triggers this tool. Your C++ application or backend server receives the request, executes the logic, and returns the result to the agent, allowing the conversation to continue seamlessly.

Performance Tips & Best Practices

To get the most out of your c++ voice api integration:
  • Use Asynchronous I/O: Avoid blocking the main thread. Use libcurl's multi-handle or asynchronous HTTP clients to handle network requests.
  • Keep Connections Alive: Reuse TCP connections with HTTP keep-alive to reduce latency on subsequent API calls.
  • Enable Compression: Use gzip compression for JSON payloads to reduce bandwidth usage.
  • Optimize Audio Format: Send audio in 16 kHz mono PCM format for optimal STT performance and lower latency. Higher sample rates increase bandwidth without significantly improving transcription accuracy.

Conclusion & Next Steps

Integrating a c++ voice api into your native applications unlocks powerful voice-first experiences without sacrificing performance. VideoSDK provides the robust REST APIs and AI Voice Agent infrastructure needed to build everything from real-time transcription to voice cloning and interactive AI assistants.
Ready to get started? Try it for free and clone the sample repository, explore the comprehensive AI Voice Agent documentation, and join the VideoSDK developer community to share your builds and ask questions. Build the future of voice applications today.

Free $20 Balance for AI Voice Agents & Video Calls

FAQ