Product Updates - May 2026 : Agent Runtime v2, Encrypted Recordings, and the Next-Gen Voice Pipeline

Welcome to the May edition of the VideoSDK Monthly Updates! This month, we have focused on bringing production maturity, security compliance, and lower latency to your real-time video and AI applications.

We are introducing Agent Runtime v2 on the dashboard, Encrypted Recordings for sensitive compliance workflows, an expanded voice pipeline supporting xAI Grok, and a native C++ SDK for embedded Linux architectures like Raspberry Pi and NVIDIA Jetson. Let's explore the updates!

Dashboard Evolution: Agent Runtime v2 & Security

As developer requirements expand, our dashboard tools must scale to support complex user management, compliance, and direct system access.

Agent Runtime v2

The no-code/low-code Agent Runtime has received a major visual and feature upgrade (v2). While maintaining the core setup you are familiar with, v2 introduces key features designed for production-level management:

• Fallback Model Support: Configure a secondary model to automatically take over if your primary model experiences issues or latency.

• Branching Logic: Manage agent versions securely by branching your configurations.

• Traffic Splitting: Safely deploy updates by splitting incoming call traffic among different branches for live A/B testing and gradual rollouts.

Secure, Encrypted Recordings

Previously, session recordings saved in the cloud could be played back directly as standard MP4 files.

Starting this month, you can configure Recording Encryption. Files are encrypted and only users possessing the private key can decrypt and play them back, ensuring zero unauthorized access.

📄 Read The Recording Encryption Docs

Dashboard Telephony & Feedback Loop

• Consolidated Phone Numbers UI: We have completely redesigned the Telephony interface to provide maximum flexibility, consolidating gateways directly into the Phone Numbers tab. It is now split into two clean views:
  • Phone Numbers: Buy, import, or SIP configure your numbers along with gateway settings (includes 2 free demo numbers to start testing instantly).
  
  
  • Routing Rules: Define precisely where incoming calls are directed, with the flexibility to map multiple phone numbers to a single routing rule.
• Native Feedback Tool: We want to hear from you! A new feedback button on the dashboard allows you to report bugs, suggest features, request improvements, or send a compliment directly to the VideoSDK team.

AI Agents: Next-Gen Voice Pipeline

Our voice framework received its most extensive optimizations yet to deliver ultra-low Time-To-First-Byte (TTFB) latencies and a more human-like flow.

Major Performance Overhaul (Agents SDK v1.0.13)

We rebuilt key components of the streaming voice pipeline to minimize delays and prevent stutter:

• Websocket Prewarm & Reuse: Persistent connections are pre-warmed and recycled to eliminate connection handshakes.
• Streaming-Safe Tokenizer: Text is passed through a new flow: LLM → text-filter → chunker → TTS to prevent pauses mid-sentence.
• Multilingual Normalization: Added script-aware segmentation, currency verbalization, and an expanded Indic-language tokenizer.

Ecosystem Expansion: xAI Grok Support (v1.0.12)

You can now use xAI's Grok directly in your pipeline. We've introduced XAISTT (WebSocket-based streaming STT) and XAITTS (WebSocket-based expressive synthesis with support for inline tag formatting).

from videosdk.agents import Pipeline
from videosdk.plugins.xai import XAILLM, XAISTT, XAITTS

pipeline = Pipeline(
    stt=XAISTT(),
    llm=XAILLM(),
    tts=XAITTS()
)

Conversational Nuance: TTS Caching & Interruption Fading

• Audio Caching (v1.0.15): Pass an optional audio_data parameter to bypass the TTS engine entirely for static, predictable phrases (like welcomes or goodbyes), dropping latency to near zero.
• Interruption Fade-out: Instead of cutting audio mid-sample when a user speaks over the agent, the buffered TTS audio is ducked down with an exponential fade-out to sound natural.

# Play a pre-rendered PCM file for zero-latency greeting
await session.say(
    "Thanks for calling. How can I help?",
    audio_data=load_audio_file("greeting.ogg")
)

Telephony, Observability & Failover

Warm Handoffs: SIP-to-SIP Transfers

Agents SDK v1.0.13 adds native support for attended SIP transfers. Your voice agent can now place a caller on hold, dial a human supervisor, brief them using an AI-generated summary of the conversation history, and gracefully execute a warm transfer without dropping the call.

Latency-Based Provider Fallback (v1.0.16)

Failover is no longer binary. You can now configure fallback providers to trigger not just when a service is completely down, but when its observed latency (over consecutive turns) crosses your budget threshold.

stt = FallbackSTT(
    [DeepgramSTT(), OpenAI_STT()],
    latency_threshold_ms=1200,      # Failover if STT takes > 1.2s
    consecutive_latency_hits=3      # Trigger on 3 slow responses
)

Realtime Mode Hooks Parity (v1.0.12)

You can now use `@pipeline.on` turn and LLM hooks in the Realtime (S2S) pipeline. This brings complete code parity between Cascade and Realtime pipelines. The same hooks run unchanged across both architectures.

@pipeline.on("user_turn_start")
async def on_user_turn_start(transcript: str):
    logging.info(f"[USER] {transcript}")

@pipeline.on("llm")
async def on_llm(data: dict):
    logging.info(f"[AGENT] {data['text']}")

Read full release notes on GitHub

Core SDK updates

JS & React: Pre-Call Testing & Node 22

We have introduced runPreCallTest() to JS v0.8.0 and React v0.10.0. This allows developers to check a user's network stability and hardware capabilities before they join the session, replacing the deprecated network stats API.

Additionally, our dependency update to mediasoup brings official support for Node 22 across JS and React SDKs.

iOS SDK (v2.8.0): AV1 & VP9 Support

iOS developers can now leverage high-efficiency video codecs. Native support for AV1 and VP9 has been added. If the hardware does not support the requested codec, the SDK handles graceful fallback to VP8 and throws a clear exception (ERROR_VIDEO_CODEC_NOT_SUPPORTED).

VideoSDK C++ SDK (v0.0.1-beta.3)

We are launching our official C++ SDK bringing native WebRTC video and audio capabilities to Linux ARM64 (aarch64) platforms. Perfect for embedded hardware like Raspberry Pi or NVIDIA Jetson boards.

curl -fsSL https://raw.githubusercontent.com/videosdk-live/videosdk-rtc-cpp-sdk/main/install.sh | sudo sh

• JS SDK v0.8.0
• React SDK v0.10.0
• iOS SDK v2.8.0
• Android SDK 1.2.1
• C++ SDK v0.0.1-beta.3

📚 New Content & Resources

• React Starter App: How to Add a Voice AI Agent to Your React App with VideoSDK
• Starter App Blogs: Checkout new guides to add ai voice agent to your react and android apps

SDK Sketches

This month's sketch: Downloading an unencrypted recording vs. Private-key encrypted security.

Happy building!
Team VideoSDK