Welcome to the November edition of the VideoSDK Monthly Updates! This month, we’re changing the game for AI development with the launch of our Agent Runtime, a No-Code/Low-Code Agent Builder on the dashboard which will also allow you to add Knowledge Base from dashboard. We're also giving your AI agents vision capabilities now with OpenAI Realtime and rolling out the Realtime Store, a powerful new data synchronization feature across all major SDKs. Let’s dive into the biggest updates!
Build AI Agents Runtime with No Code
We are thrilled to announce the launch of the Agent Runtime, our new no-code/low-code AI agent builder, now live on your developer dashboard!
Building, testing, and deploying powerful conversational AI has never been easier. The Agent Runtime allows you to create and configure complex AI agents through an intuitive visual interface, dramatically reducing development time and making advanced AI accessible to everyone. Connect your favorite LLMs, configure responses, and deploy agents without writing a single line of code.
Connect a Knowledge Base from the Dashboard
Making your agents smarter is now just a few clicks away. You can now upload documents and create a knowledge base directly from the dashboard. Our system automatically handles the vectorization and retrieval (RAG), allowing your no-code agents to provide accurate answers based on your own custom data. It's the easiest way to build a specialized, expert AI agent.
New Feature Spotlight: The Realtime Store
We're excited to introduce the Realtime Store, a synchronized key-value database built directly into your meeting sessions. This powerful feature acts as a shared data layer, allowing you to store, update, and observe custom data across all participants in real time.
It's perfect for building collaborative features like shared whiteboards, live polls, synchronized presentations, or tracking any shared state within your application.
The Realtime Store is now available on:
- JavaScript SDK (>v0.3.10)
- React SDK (>v0.4.10)
- React Native SDK (>v0.5.0)
- iOS SDK (>v2.2.7)
- Android SDK (>v1.1.0)
- => View Docs
AI Agents Vision Updates & A Simpler Lifecycle
We are fully invested in our Agents SDK and have given tones of upgrade this month, making agents smarter, more capable, and easier to manage.
- Enhanced Vision Capabilities (Agents SDK v0.0.42):
- Vision in cascading pipeline! With the new capture_frames() method, you can pass video frames directly into your agent's reply() method. This unlocks powerful new use cases like real-time visual analysis, describing on-screen activity, and more in either of the pipelines. We've even added support for continuous vision frame processing in our OpenAI and Gemini Live plugins.
- Simplified Session Lifecycle (Agents SDK v0.0.44):
- We've made managing agent sessions effortless. You can now use session.start(wait_for_participant=True, run_until_shutdown=True) to handle the entire lifecycle with a single line of code, while still retaining full manual control if you need it.
- An Expanding AI Ecosystem (Agents SDK v0.0.42-46):
- We've added support for Deepgram STT V2, ElevenLabs Scribe V2, and improved our plugins for Sarvam AI, Gemini Live, and OpenAI.
- View full Agents SDK changelog
A Better Developer Experience
We’ve shipped a series of quality-of-life improvements to give you deeper observability and control.
- Deeper Insight into "Why":
- We’ve added reason and code parameters to the meeting-left and participant-left events across our JS, React, React Native, Android, and Flutter SDKs. You'll now know exactly why a user's session ended.
- Enhanced Observability:
- We've made all trace messages user-facing and added a sessionId parameter across our JS, React, and React Native SDKs, enabling you to fetch and analyze traces directly from the dashboard.
- New WHIP/WHEP Documentation:
- For developers working with standardized ingest and egress protocols, our comprehensive WHIP/WHEP documentation is now live! => View the Docs
Core SDK Enhancements
- iOS SDK (v2.2.9 & v2.3.0):
- Now includes advanced video track optimization with BitrateMode and maxLayer parameters for fine-tuning quality and bandwidth. => View Changelogs
- Flutter SDK (v3.2.0):
- You can now pause() and resume() active streams, giving you more control over media playback. => View Changelogs
- Android SDK (v1.0.0):
- Added the OldMessagesReceived event to retrieve persisted PubSub messages and set the default consumer video quality to high. => View Changelogs
- React Native SDK (v0.5.0):
- This massive release includes Beta support for Adaptive Subscriptions, new quality limitation and stream state events, and the useStream hook for easier stream management. => View Changelogs
📚 New Content & Resources
New Platform-Specific Quickstart Guides for Agents Runtime
- Getting started with VideoSDK Agents has never been easier. With the release of Agent Runtime, we've also published a full suite of new quick-start agent integration guides tailored to your favourite platform to connect the no code agent from within your meeting room. Whether you're building for web or mobile, we've got you covered.
Find your guide here:
| Name | Link |
|---|---|
| Web | |
| JavaScript Quickstart | [Doc] |
| React Quickstart | [Doc] |
| Mobile | |
| React Native Quickstart | [Doc] |
| iOS Quickstart | [Doc] |
| Flutter Quickstart | [Doc] |
Guide: Integrate with Traditional Telephony using SIP-Connect
Bridge the gap between your digital meetings and the world of traditional telephony. Our new guide shows you how to use SIP-Connect to enable dial-in participants and connect your application with any phone line.
Guide: AI Voice Agent Observability
Dive into our new practical guide on debugging latency and improving your AI agents' performance. Learn how to leverage VideoSDK traces for a deeper understanding of your agent's behavior.
✨ Community Spotlight
Explore how CoderSchool achieves student engagement growth of 100,000+ sessions per month and builds huge tech talent network in Southeast Asia.
SDK Sketches
This month's sketch: Using Agent Runtime vs Traditional Coding
That’s a wrap for November! From our new no-code builder to realtime store across SDKs, we can't wait to see what you build with these new tools.
We'd love to hear your feedback! If you have any questions, suggestions, or issues, please don't hesitate to contact our support team.
➡️ New to VideoSDK? Sign up now and get 10,000 free minutes to start building amazing audio & video experiences!
