SIP (Session Initiation Protocol) handles call signaling by setting up, modifying, and tearing down communication sessions over IP networks. RTP (Real-time Transport Protocol) carries the actual voice and video media packets in real time. Together, they form the backbone of every VoIP call: SIP negotiates session parameters through SDP exchange, and RTP streams the media. VideoSDK bridges these traditional telephony protocols with modern WebRTC through its SIP and telephony integration.
When a user dials a colleague on a VoIP phone, two distinct protocols swing into action behind the scenes. One handles the conversation setup, ringing, and teardown. The other carries the actual voice packets across the network. Understanding how SIP and RTP work together is essential for developers building voice or video applications, troubleshooting call quality issues, or integrating traditional telephony with modern WebRTC infrastructure.
Whether you are building a telehealth platform with embedded video calling, an AI voice agent that answers phone calls, or a SIP trunking gateway that bridges PBX systems to the cloud, the SIP and RTP relationship sits at the core of your media pipeline. Get the signaling wrong and calls never connect. Get the media transport wrong and calls connect but sound terrible.
What Is SIP? The Signaling Maestro
SIP is defined as a text-based application-layer control protocol used to establish, modify, and terminate multimedia sessions over IP networks. Developed by the IETF and standardized in RFC 3261, SIP borrows heavily from HTTP in its design philosophy. It uses request-response transactions, human-readable headers, and URI-based addressing.
Core Responsibilities of SIP
SIP handles three primary functions in a VoIP call. First, it locates the called party through registrar and proxy servers, translating a SIP URI into a routable IP address. Second, it negotiates session parameters by carrying an SDP (Session Description Protocol) payload that describes codecs, port numbers, and media types. Third, it manages the session lifecycle: modifying calls (like adding video mid-call), transferring calls, and tearing them down with a BYE message.
SIP does not transport voice or video media. That distinction is critical and is the source of much confusion for developers new to VoIP.
Key SIP Components
A SIP architecture includes several logical components. A User Agent (UA) is an endpoint that initiates or receives calls, typically a softphone, hard phone, or SIP-enabled application. A Proxy Server forwards SIP requests toward their destination, acting much like an HTTP reverse proxy. A Registrar accepts registration requests from User Agents and maintains a location database mapping SIP URIs to current IP addresses. A Back-to-Back User Agent (B2BUA) sits between two User Agents and maintains state for both legs of the call, enabling features like call transfer, billing, and media manipulation.
How SIP Differs From Media Transport
Think of SIP as the telephone operator who connects your call and hangs it up when you are done. SIP never carries your voice. It only carries the instructions that tell the network how to set up the media path. The actual voice packets travel over RTP, which we will cover next. This separation of signaling and media is fundamental to understanding how VoIP works.
What Is RTP? The Media Delivery Engine
RTP is defined as a network protocol for delivering audio and video over IP networks in real time. RTP runs over UDP (not TCP) because real-time media prioritizes low latency over guaranteed delivery. A retransmitted packet that arrives 200ms late is worse than a dropped packet, since the playback buffer has already moved past that point.
RTP Packetization and Structure
Every RTP packet contains a header with several critical fields. The sequence number increments by one for each packet sent, allowing the receiver to detect packet loss and reorder out-of-sequence packets. The timestamp indicates the sampling instant of the first byte in the payload, enabling the receiver to reconstruct timing and play back audio at the correct rate. The payload type field identifies the codec (like G.711, Opus, or G.722), so the receiver knows how to decode the payload. The synchronization source identifier (SSRC) uniquely identifies the source of the stream within a session.
RTCP and Optional Security
RTP is typically paired with RTCP (RTP Control Protocol), which runs on the port number one higher than the RTP stream. RTCP provides out-of-band statistics about the media session: packet counts, octet counts, jitter measurements, and round-trip time estimates. These statistics are what network engineers examine when diagnosing call quality issues.
For security, RTP can be encrypted using SRTP (Secure Real-time Transport Protocol), which adds confidentiality and integrity to each packet. DTLS-SRTP is commonly used in WebRTC to negotiate SRTP keys, and VideoSDK supports encrypted media through its video calling SDK with end-to-end encryption capabilities.
How Does SIP and RTP Work Together?
SIP and RTP work together through a clear division of labor: SIP handles the conversation setup and teardown, while RTP carries the actual voice and video media. The bridge between them is SDP (Session Description Protocol), which SIP carries inside its message body to negotiate the parameters that RTP will use.
SIP Initiates, RTP Carries
When a caller dials a number, the SIP User Agent sends an INVITE message containing an SDP offer. The SDP describes what codecs the caller supports, which IP address and port the caller wants to receive media on, and what media types (audio, video) are in scope. The callee responds with a 200 OK containing an SDP answer that selects the codec, confirms the IP address and port, and agrees on the media parameters.
Once both sides agree, RTP streams begin flowing in both directions. SIP steps back and waits. It only re-engages when the call needs modification (like putting someone on hold) or teardown (the BYE message).
SDP Negotiation Inside SIP Messages
The SDP exchange is the critical handshake that makes everything work. The offer-answer model, defined in RFC 4566, ensures both parties agree on a single codec, a single set of IP addresses and ports, and a common understanding of what the session contains. If the SDP negotiation fails or the parties cannot agree on a codec, the call fails even though SIP signaling itself succeeded.
Establishing the Media Path
After the SDP negotiation completes, each side knows where to send RTP packets. The media path is typically direct between the two endpoints (or through a media relay if NAT traversal requires it). SIP proxies generally do not handle media at all. This separation means signaling servers can be lightweight and scalable, while media servers handle the heavy lifting of packet forwarding.
Timing Considerations and Jitter Handling
RTP timestamps allow receivers to reconstruct the original timing of the audio stream. Network conditions introduce jitter (variation in packet arrival times), which the receiver handles with a jitter buffer. The jitter buffer holds incoming packets briefly and releases them at a steady rate, smoothing out network variability. Larger jitter buffers add latency but improve quality on unstable connections. Smaller buffers reduce latency but risk audible gaps if packets arrive late.
How Does SIP and RTP Work: The Detailed Call Flow
A typical SIP call follows a predictable sequence of messages between the caller, any intermediate proxies, and the callee. Understanding this flow is essential for troubleshooting failed calls and for building SIP-compatible applications.
The INVITE to 200 OK to ACK Sequence
The call begins when the caller's User Agent sends an INVITE message to the callee's SIP address. The INVITE contains the SDP offer describing the caller's media capabilities. The first proxy server receives the INVITE and forwards it toward the callee, potentially after querying the registrar to resolve the callee's current IP address.
The callee's phone receives the INVITE and immediately sends a 100 Trying response back through the proxy. This tells the caller that the request was received and is being processed. The callee's phone then rings and sends a 180 Ringing response, which tells the caller's phone to play a ringback tone.
When the callee picks up, their phone sends a 200 OK response containing the SDP answer. This answer confirms the selected codec, the callee's IP address and port for receiving media, and any other agreed parameters. The caller's phone acknowledges receipt of the 200 OK by sending an ACK message. At this point, the SIP signaling is complete and RTP media flows in both directions.
When either party hangs up, their phone sends a BYE message. The other side responds with a 200 OK, and the session is terminated. RTP streams stop immediately.
Role of Each SIP Method
Each SIP method in the call flow serves a specific purpose. INVITE initiates the session and carries the SDP offer. 100 Trying is a provisional response confirming receipt. 180 Ringing is a provisional response indicating the callee is being alerted. 200 OK is the final response confirming the session is established, carrying the SDP answer. ACK confirms receipt of the final response and completes the three-way handshake. BYE terminates the session.
How SDP Is Carried and Interpreted
SDP travels as the message body of the INVITE and 200 OK messages. The SIP headers indicate the content type as application/sdp, and the receiving User Agent parses the SDP to extract codec lists, IP addresses, port numbers, and media types. The offer-answer model ensures that only one codec is ultimately selected, even if multiple were offered.
RTP Transport and Packet Structure
RTP packets are built by the sending application and transmitted over UDP to the IP address and port negotiated during the SDP exchange. Each packet carries a small piece of the audio or video stream, typically 20ms of audio for voice calls.
Building and Sending RTP Packets
The sender reads audio from the microphone, encodes it using the negotiated codec, and packages each encoded chunk into an RTP packet. The RTP header is prepended with the sequence number, timestamp, payload type, and SSRC. The packet is then sent over UDP to the destination port specified in the SDP answer.
Sequence Numbers and Timestamps for Ordering
Sequence numbers allow the receiver to detect missing packets and reorder any packets that arrived out of sequence. If the receiver sees sequence numbers 1, 2, 4, 5, it knows packet 3 was lost and can apply packet loss concealment to mask the gap. Timestamps tell the receiver when each packet's audio should be played back, independent of when the packet actually arrived. This separation of transport time and playback time is what makes RTP resilient to network jitter.
Jitter Buffer Function on the Receiver Side
The jitter buffer is the receiver's primary defense against network variability. It holds incoming RTP packets for a short period (typically 20 to 60ms) and releases them to the decoder at a steady pace. If a packet arrives late, the jitter buffer may have already moved past its playback time, resulting in a gap. If a packet arrives early, the buffer holds it until its timestamp indicates it is time for playback. Adaptive jitter buffers dynamically adjust their size based on observed network conditions, growing when jitter increases and shrinking when the network stabilizes.
NAT Traversal and Security Considerations
NAT (Network Address Translation) is the most common source of SIP and RTP problems in real-world deployments. The issue is that SIP messages carry IP addresses and port numbers inside their SDP payloads, and these private addresses are meaningless to the remote party trying to send RTP traffic.
Why NAT Breaks SIP Address Information
When a SIP User Agent behind a NAT router sends an INVITE, the SDP body contains the agent's private IP address (like 192.168.1.100). The remote party receives this address and tries to send RTP packets to it, but that address is not routable on the public internet. The result is one-way audio or no audio at all. SIP ALG (Application Layer Gateway) features in some routers attempt to fix this by rewriting SDP addresses, but they often introduce more problems than they solve.
Common Solutions: STUN, TURN, ICE, and SIP ALG
STUN (Session Traversal Utilities for NAT) allows a client to discover its public IP address and port as seen by the internet. TURN (Traversal Using Relays around NAT) provides a relay server that forwards media when direct connectivity fails. ICE (Interactive Connectivity Establishment) is a framework that tries direct connection first, then STUN-assisted connection, then TURN relay as a last resort. These are the same mechanisms used in WebRTC, and VideoSDK handles all of this automatically through its WebRTC-based video calling infrastructure.
Securing the Media: SRTP and DTLS-SRTP
RTP itself provides no encryption. SRTP adds AES encryption and HMAC integrity protection to each RTP packet. DTLS-SRTP, used in WebRTC, performs a DTLS handshake to negotiate SRTP keys, ensuring the media path is encrypted end to end. For traditional SIP deployments, SDES-SRTP is more common, where keys are exchanged in the SDP payload (which should itself be encrypted).
Securing the Signaling: TLS and SIP Digest Authentication
SIP signaling can be secured using TLS (Transport Layer Security), which encrypts the entire SIP message including the SDP payload. SIP digest authentication challenges the client with a nonce and expects a hashed response, preventing unauthorized call setup. Together, TLS for transport and digest authentication for identity provide a reasonable security baseline for SIP deployments.
Common Issues and Troubleshooting Tips
When VoIP calls fail or sound poor, the problems almost always fall into one of three categories: signaling failures, media path issues, or quality degradation.
One-Way Audio, High Latency, and Packet Loss
One-way audio is the most common VoIP complaint and usually indicates a NAT traversal problem where RTP flows in one direction but not the other. Check whether the SDP contains a routable public IP address and whether firewalls are blocking the negotiated RTP port range. High latency (above 150ms one-way) makes conversations feel unnatural and is typically caused by long network paths, excessive jitter buffer sizes, or codec overhead. Packet loss above 1 percent degrades voice quality noticeably and is usually a network infrastructure issue.
Reading SIP Traces to Pinpoint Problems
SIP traces reveal the exact sequence of messages exchanged during a call. Look for missing responses (no 200 OK means the callee never answered), unexpected error codes (487 Request Terminated suggests the caller hung up before the callee answered), and SDP mismatches where the offer and answer do not align on codecs or ports. Tools like Wireshark can decode SIP and RTP streams, showing the full message flow and media statistics side by side.
Quick Checks for RTP Stream Health
RTCP reports provide the fastest way to assess RTP health. Look at the jitter value (should be under 30ms for good quality), packet loss percentage (should be under 1 percent), and round-trip time (should be under 150ms). Sequence number gaps indicate lost packets. If the jitter buffer is consistently overflowing, the network path may be too unstable for real-time media without a relay.
Definitions Glossary
SIP (Session Initiation Protocol): A text-based application-layer protocol for establishing, modifying, and terminating multimedia sessions over IP networks. SIP carries SDP payloads to negotiate media parameters but does not transport voice or video itself.
RTP (Real-time Transport Protocol): A UDP-based protocol for delivering audio and video packets in real time. RTP uses sequence numbers and timestamps to handle packet loss, reordering, and jitter on the receiver side.
SDP (Session Description Protocol): A format for describing media sessions, including codecs, IP addresses, port numbers, and media types. SDP is carried inside SIP messages during the offer-answer negotiation.
RTCP (RTP Control Protocol): A companion protocol to RTP that provides feedback on media quality, including packet counts, jitter measurements, and round-trip time estimates.
Jitter Buffer: A receiver-side buffer that holds incoming RTP packets briefly and releases them at a steady rate, compensating for network timing variability.
SRTP (Secure Real-time Transport Protocol): An extension of RTP that adds encryption and integrity protection to each media packet, ensuring voice and video content cannot be intercepted or tampered with.
Key Takeaways
- SIP and RTP serve fundamentally different roles in VoIP: SIP handles signaling (call setup, modification, teardown) while RTP carries the actual voice and video media packets over UDP.
- SDP negotiation inside SIP messages is the critical bridge between signaling and media, determining which codec, IP address, and port both parties will use for the RTP stream.
- RTP relies on sequence numbers and timestamps to handle packet loss, reordering, and jitter, with the jitter buffer on the receiver side smoothing out network variability for clean playback.
- NAT traversal remains the most common source of VoIP problems, and solutions like STUN, TURN, and ICE are essential for reliable media delivery across networks.
- VideoSDK bridges traditional SIP telephony with modern WebRTC through its telephony and SIP integration, handling NAT traversal, encryption, and media routing automatically so developers can focus on building features rather than debugging packet flows.
Conclusion
Understanding how SIP and RTP work gives you the foundation to build, debug, and scale real-time communication systems. SIP handles the conversation: finding the callee, negotiating parameters through SDP, and managing the session lifecycle. RTP handles the payload: streaming encoded audio and video packets with timing information that lets receivers reconstruct the original media despite network imperfections. The separation of signaling and media is what makes VoIP architectures scalable, but it is also what makes them tricky to debug when things go wrong.
If you are building applications that need to bridge traditional telephony with modern WebRTC, VideoSDK's SIP integration handles the protocol complexity for you. You can also explore VideoSDK's video calling SDK for WebRTC-native communication or dive into AI voice agents that connect phone calls to LLM-powered pipelines. Sign up free at app.videosdk.live/login to start building.
What are you building with SIP, RTP, or WebRTC? Drop a comment below and let me know what kind of VoIP or real-time communication use case you are working on.
FAQ