C++ Voice API: Build Real-Time Voice Apps with VideoSDK

Real-time voice communication is no longer a luxury—it's a requirement for modern applications, from multiplayer games and IoT devices to telemedicine platforms and AI-powered virtual assistants. If you're building in C++ for performance, low-level hardware access, or cross-platform portability, you need a voice API that matches your stack. VideoSDK provides a C++ voice API that enables sub-500ms latency for real-time voice calling, supporting up to 10,000 participants per room with built-in noise suppression and echo cancellation. This article shows you how to integrate it, optimize it, and extend it with AI voice agents.

What is a C++ Voice API?

A C++ Voice API is a software interface that allows developers to integrate voice capabilities—such as real-time audio streaming, speech recognition, or text-to-speech—into C++ applications. These APIs abstract the complexities of audio capture, encoding, network transport, and playback, letting you focus on your application logic.
There are two broad categories:
  • Real-time voice calling APIs (WebRTC-based) handle live audio streaming between participants with low latency. They manage signaling, NAT traversal, and media encryption.
  • Voice AI APIs provide automatic speech recognition (ASR) or text-to-speech (TTS) services, often via cloud-based models.
VideoSDK’s C++ voice API focuses on real-time communication, offering WebRTC-based audio calling with built-in noise suppression and echo cancellation. It is designed for developers who need a production-ready, scalable voice layer without building a custom WebRTC stack from scratch.

Why Use C++ for Voice Applications?

C++ remains the language of choice for performance-critical systems. When building voice applications, the advantages are clear:
  • Low latency: Direct memory management and minimal runtime overhead ensure audio packets are processed with microsecond precision.
  • Hardware access: C++ can interface directly with audio devices, DSP chips, and embedded microphones.
  • Cross-platform portability: A single C++ codebase can target Windows, Linux, macOS, Android, iOS, and embedded systems.
  • Ecosystem: Game engines (Unreal, Unity native plugins), IoT frameworks, and high-performance servers are predominantly C++. For Unity developers, VideoSDK also provides a dedicated unity video and audio calling sdk, enabling seamless integration of voice and video into Unity games.
For real-time voice, every millisecond counts. C++ gives you the control to hit sub-500ms glass-to-glass latency, which is critical for natural conversation.

Key Features of a Production-Ready C++ Voice API

A Voice API must deliver more than just audio streaming. Here are the non-negotiable features:
  • Ultra-low latency: <500ms end-to-end for natural conversation.
  • Scalability: Support for thousands of concurrent participants in a single room.
  • Codec flexibility: Opus for adaptive bitrate and PCM for uncompressed quality.
  • Security: DTLS-SRTP encryption, token-based authentication, and room-level access control.
  • Audio processing: Acoustic echo cancellation (AEC), automatic gain control (AGC), and noise suppression.
  • Network resilience: Jitter buffer, packet loss concealment, and bandwidth estimation.
VideoSDK’s C++ API delivers all of these out of the box. You don’t need to integrate separate libraries for echo cancellation or write your own congestion control—the SDK handles it with optimized defaults.

Getting Started with VideoSDK’s C++ Voice API

Installation and Setup

Prerequisites:
  • CMake 3.16+
  • C++17 compatible compiler (GCC 9+, Clang 10+, MSVC 2019+)
  • A VideoSDK account (sign up at videosdk.live)
Add the VideoSDK C++ SDK to your project using CMake:
1cmake_minimum_required(VERSION 3.16)
2project(MyVoiceApp)
3
4# Fetch VideoSDK C++ SDK
5include(FetchContent)
6FetchContent_Declare(
7  videosdk
8  GIT_REPOSITORY https://github.com/videosdk-live/videosdk-cpp.git
9  GIT_TAG        v2.0.0
10)
11FetchContent_MakeAvailable(videosdk)
12
13add_executable(my_app main.cpp)
14target_link_libraries(my_app videosdk::videosdk)
15

Initializing the SDK

Create a VideoSDK client and join a room with a token generated from your VideoSDK dashboard.
1#include <videosdk/videosdk.hpp>
2#include <iostream>
3
4int main() {
5    videosdk::Client client;
6    client.initialize("YOUR_API_KEY", "YOUR_API_SECRET");
7
8    videosdk::RoomConfig config;
9    config.roomId = "my-voice-room";
10    config.displayName = "UserA";
11    config.token = "GENERATED_JWT_TOKEN";
12
13    auto room = client.joinRoom(config);
14    if (room) {
15        std::cout << "Joined room successfully!" << std::endl;
16    }
17    return 0;
18}
19

Making Your First Voice Call

A complete voice call involves creating a room, joining it, publishing your local audio stream, and subscribing to remote participants’ streams. The following example demonstrates a minimal two-party call, similar to a phone call api.
1#include <videosdk/videosdk.hpp>
2#include <iostream>
3#include <thread>
4#include <chrono>
5
6class MyAudioHandler : public videosdk::AudioHandler {
7public:
8    void onRemoteAudioStream(videosdk::AudioStream& stream) override {
9        std::cout << "Receiving remote audio from: " << stream.participantId() << std::endl;
10        // Play the stream through default audio device
11        stream.play();
12    }
13};
14
15int main() {
16    videosdk::Client client;
17    client.initialize("YOUR_API_KEY", "YOUR_API_SECRET");
18
19    videosdk::RoomConfig config;
20    config.roomId = "voice-demo-room";
21    config.displayName = "Caller";
22    config.token = "GENERATED_JWT_TOKEN";
23
24    auto room = client.joinRoom(config);
25    if (!room) {
26        std::cerr << "Failed to join room" << std::endl;
27        return -1;
28    }
29
30    // Publish local microphone audio
31    auto localAudio = room->createLocalAudioTrack();
32    localAudio->startCapture();
33    room->publish(localAudio);
34
35    // Subscribe to remote audio
36    MyAudioHandler handler;
37    room->addAudioHandler(&handler);
38
39    // Keep the call alive for 30 seconds
40    std::this_thread::sleep_for(std::chrono::seconds(30));
41
42    room->leave();
43    return 0;
44}
45
With just a few lines, you have a working phone call api integration.
The sequence diagram below illustrates the signaling and media flow between two clients:
1sequenceDiagram
2    participant A as Client A (C++)
3    participant V as VideoSDK Server
4    participant B as Client B (C++)
5
6    A->>V: Join room (token, roomId)
7    V-->>A: Room joined, participant list
8    B->>V: Join room
9    V-->>B: Room joined, participant list (includes A)
10    A->>V: Publish local audio track
11    V->>B: New audio stream from A
12    B->>B: Subscribe and play remote audio
13    B->>V: Publish local audio track
14    V->>A: New audio stream from B
15    A->>A: Subscribe and play remote audio
16    Note over A,B: Bidirectional real-time voice call established
17

Comparing C++ Voice APIs: VideoSDK vs. Alternatives

Choosing the right Voice API depends on your use case. The table below compares VideoSDK with raw WebRTC, traditional CPaaS providers, and voice AI SDKs.
Feature / Capability VideoSDK C++ API Raw WebRTC (libwebrtc) Twilio Voice Agora C++ SDK Camb.ai / sherpa-onnx
Real-time latency <500ms <500ms (with custom tuning) <500ms <400ms N/A (AI generation latency)
Max participants per room 10,000 Depends on SFU implementation 50 (Programmable Voice) 1,000 (Voice Calling) N/A
Built-in audio processing AEC, AGC, noise suppression Requires custom integration AEC, noise suppression AEC, AGC, noise suppression None (raw audio I/O)
Ease of integration High (single SDK, token-based) Low (complex build, signaling, TURN) Medium (REST + SDK) Medium (SDK, custom signaling) Medium (model integration)
AI voice agent support Native (Conversational Graph) Requires external AI integration Limited (Media Streams) Limited Primary focus (TTS/ASR)
Pricing Usage-based, transparent Free (but high dev cost) Per-minute Per-minute Model hosting / API costs
VideoSDK is purpose-built for real-time voice communication at scale. Unlike raw WebRTC, it eliminates the need to build and maintain signaling servers, TURN/STUN infrastructure, and audio processing pipelines. While Twilio and Agora offer robust phone call api solutions, VideoSDK provides a more scalable and AI-ready option. If you're evaluating livekit alternatives, VideoSDK stands out with its C++ support and AI capabilities.

Advanced Use Cases: AI Voice Agents and Beyond

Real-time Voice API solutions become even more powerful when combined with AI. You can build:
  • Voice bots that answer customer calls and interact using natural speech.
  • Virtual assistants for in-game characters or smart home devices.
  • Interactive voice response (IVR) systems with dynamic, AI-generated responses.
VideoSDK’s C++ API can stream audio to and from AI models. For example, you can capture a participant’s speech, send it to a speech-to-text engine (like Deepgram or sherpa-onnx), process the text with an LLM, and then stream the synthesized response back into the room using a text-to-speech service (like Camb.ai or ElevenLabs).
VideoSDK simplifies this with its AI voice agent capabilities and Conversational Graph—a framework for defining multi-turn voice interactions. You can orchestrate the entire pipeline within your C++ application, maintaining ultra-low latency throughout.

Performance Optimization Tips for C++ Voice Apps

To get the best quality and lowest latency, tune your audio configuration:
  • Codec selection: Use Opus for most scenarios—it adapts to network conditions and provides excellent quality at low bitrates. Use PCM only when uncompressed audio is required (e.g., for AI processing).
  • Bitrate: For voice, 32–64 kbps is sufficient. Higher bitrates increase bandwidth without noticeable quality improvement.
  • Jitter buffer: VideoSDK’s adaptive jitter buffer works well out of the box, but you can adjust the target delay if you need tighter control.
Example of configuring the audio encoder in VideoSDK:
1videosdk::AudioEncoderConfig encoderConfig;
2encoderConfig.codec = videosdk::AudioCodec::OPUS;
3encoderConfig.bitrate = 48000; // 48 kbps
4encoderConfig.complexity = 5;  // Opus complexity (0-10)
5
6localAudio->setEncoderConfig(encoderConfig);
7
For network resilience, ensure your application handles reconnection gracefully. VideoSDK automatically reconnects and resyncs audio streams after temporary disconnects.

Conclusion and Next Steps

VideoSDK’s C++ Voice API gives you the fastest path to adding real-time voice communication to your C++ applications. With sub-500ms latency, built-in audio processing, and native AI voice agent support, you can build everything from simple voice calls to sophisticated voice bots. Beyond voice, VideoSDK also offers a comprehensive Video Calling API for full audio-video conferencing.
Next steps:
Related guides:

Free $20 Balance for AI Voice Agents & Video Calls

FAQ