FFmpeg copy HLS to RTP is the process of using FFmpeg's stream copy mode to relay an HTTP Live Streaming source directly into Real-time Transport Protocol packets without re-encoding. This approach preserves original codec quality, adds near-zero CPU overhead, and minimizes latency because FFmpeg skips decoding and encoding stages. You can integrate the resulting RTP stream into media servers, WebRTC gateways, or VideoSDK rooms for real-time communication pipelines. For building interactive live streaming or video calling applications, explore the VideoSDK Interactive Live Streaming guide.
Moving live HLS streams into RTP opens the door to low-latency use cases that HTTP-based delivery cannot serve well. Live sports broadcasts, IP camera feeds, surveillance systems, and real-time communication pipelines all benefit from the sub-second responsiveness that RTP provides. HLS, while excellent for adaptive bitrate delivery to millions of viewers, typically introduces 6 to 30 seconds of latency due to its segment-based architecture. RTP, by contrast, packetizes media at the frame level and sends it immediately, making it the backbone of real-time media distribution.
FFmpeg's copy mode is the most efficient path for this conversion. Instead of decoding each HLS segment and re-encoding it into a new format, FFmpeg simply demuxes the HLS source and remuxes the media into RTP packets. This means the original H.264 video and AAC audio pass through untouched. The result is a lightweight media relay that runs on minimal hardware and introduces only the latency required for packetization and network transit.

Why Use ffmpeg copy hls to rtp

The primary advantage of using FFmpeg copy mode for HLS to RTP conversion is the elimination of the decode-encode cycle. When you transcode, FFmpeg must decode every video frame and audio sample, apply any filters or transformations, and then re-encode the media into the target format. This process consumes significant CPU resources, introduces processing latency, and can degrade quality through generational loss. Copy mode bypasses all of that.
When the source HLS stream already uses H.264 for video and AAC for audio, copy mode works seamlessly because these codecs are natively supported in RTP packetization. FFmpeg reads the encoded frames from HLS segments, wraps them in RTP headers with appropriate timestamps and sequence numbers, and sends them on their way. The CPU overhead is negligible because no compression or decompression happens. This makes copy mode ideal for running on edge devices, low-cost VPS instances, or containers with limited compute budgets.
Latency is another critical factor. Transcoding adds anywhere from 100 to 500 milliseconds of processing delay per pass, depending on the codec settings and hardware. Copy mode adds only the time required to read the segment, extract the frames, and packetize them, which is typically under 10 milliseconds. For applications where every millisecond counts, such as live auction platforms or real-time surveillance, this difference is substantial. The bandwidth impact is also neutral: the RTP stream consumes the same bitrate as the original HLS source because the media payload is identical.

Prerequisites and Environment Setup

Before setting up an FFmpeg copy HLS to RTP pipeline, you need to ensure your environment meets several requirements. First, install FFmpeg version 4.4 or later. Earlier versions may lack proper HLS demuxing support or have bugs in RTP muxing that cause packet loss or timestamp issues. You can verify your installed version by running FFmpeg with its version flag and checking the build configuration output.
Your FFmpeg build must include libavformat with HLS and RTP protocol support. Most pre-built binaries from popular package managers include these by default, but if you are compiling FFmpeg from source, make sure the build configuration enables both protocols. You should also confirm that libavcodec includes H.264 and AAC decoders and that the RTP muxer is available.
On the network side, you need a server or machine with sufficient bandwidth to handle the incoming HLS stream and the outgoing RTP stream simultaneously. Since copy mode does not change the bitrate, the outbound bandwidth requirement matches the inbound HLS bitrate. For a 1080p stream at 4 Mbps, plan for at least 8 Mbps of total bandwidth headroom to account for protocol overhead and network fluctuations.
Firewall configuration is critical. If you plan to send RTP over UDP, you must open the destination UDP port on the receiving end. RTP typically uses an even-numbered port, with RTCP using the next odd-numbered port. Ensure both ports are open. If your environment uses NAT, you may need port forwarding or a STUN/TURN server to ensure packets reach the destination. For TCP-based RTP, ensure the destination port accepts inbound TCP connections.
A quick checklist for environment readiness: FFmpeg version 4.4 or higher with HLS and RTP support, a server with adequate bandwidth, UDP or TCP ports opened on the receiving firewall, and a test HLS source URL you can access reliably.

Understanding HLS and RTP Fundamentals

What is HLS?

HTTP Live Streaming, or HLS, is an adaptive bitrate streaming protocol developed by Apple. It works by breaking the media stream into small sequential segments, typically 2 to 10 seconds each, and delivering them over standard HTTP. A playlist file with the extension .m3u8 indexes these segments and tells the player where to find each one. The playlist can be a simple static list for video-on-demand or a sliding-window live playlist that updates as new segments become available.
HLS supports adaptive bitrate streaming by offering multiple renditions of the same content at different resolutions and bitrates. The player monitors available bandwidth and switches between renditions to maintain smooth playback. This makes HLS excellent for large-scale content delivery over CDNs.
The trade-off is latency. Because HLS must wait for a complete segment to be encoded, uploaded, and downloaded before playback, end-to-end latency typically ranges from 6 to 30 seconds. Even with Low-Latency HLS (LL-HLS) extensions that reduce segment sizes and use partial segments, the latency floor remains higher than what real-time applications require. For a deeper comparison of streaming protocols and latency characteristics, the VideoSDK Interactive Live Streaming documentation covers how sub-second streaming differs from HLS delivery.

What is RTP?

The Real-time Transport Protocol, or RTP, is a network protocol designed for delivering audio and video over IP networks in real time. Unlike HLS, which delivers complete segments over HTTP, RTP packetizes media at the frame or access unit level and sends each packet immediately as it is produced. Each RTP packet carries a header containing a sequence number, a timestamp, a payload type identifier, and a synchronization source identifier.
The sequence numbers allow receivers to detect packet loss and reorder out-of-sequence packets. The timestamps enable proper playback timing and jitter buffer management. RTP typically runs over UDP, which avoids the head-of-line blocking delays that TCP introduces. This makes RTP suitable for real-time distribution where occasional packet loss is preferable to delayed delivery.
RTP is the standard transport layer for WebRTC media, SIP-based VoIP calls, and many broadcast IP workflows. When you convert HLS to RTP, you bridge the gap between HTTP-based adaptive streaming and real-time packet-based delivery. This bridge enables scenarios like feeding an HLS surveillance camera feed into a WebRTC gateway for browser-based monitoring, or relaying a live HLS broadcast into a media server for further distribution.

Architecture Overview

The FFmpeg copy HLS to RTP pipeline follows a straightforward linear architecture. An HLS source, typically a live .m3u8 playlist hosted on a CDN or origin server, serves as the input. FFmpeg reads this playlist, downloads segments as they appear, and demuxes the media streams from within those segments. In copy mode, FFmpeg does not decode the compressed media. Instead, it extracts the already-encoded H.264 and AAC frames and passes them to the RTP muxer.
The RTP muxer wraps each frame in RTP headers with appropriate payload type, timestamp, and sequence number fields. These packets are then sent over the network using either UDP or TCP as the transport protocol. Downstream receivers can include media servers like Wowza or Ant Media, WebRTC gateways, IP set-top boxes, or even another FFmpeg instance for further processing.
Architecture Diagram
This architecture is intentionally minimal. The fewer hops and processing stages between source and destination, the lower the end-to-end latency. Because FFmpeg copy mode does not touch the media payload, the pipeline acts as a protocol bridge rather than a transcoder. This is what makes it so efficient for low-latency relay scenarios.

Step-by-Step Guide to ffmpeg copy hls to rtp

1. Verify Source Compatibility

Before attempting to copy an HLS stream into RTP, you must confirm that the source codecs are compatible with RTP packetization. The most reliable way to inspect an HLS stream is to use FFprobe, the analysis tool bundled with FFmpeg. Point FFprobe at the HLS playlist URL and examine the output for video and audio codec information.
You are looking for H.264 or H.265 for video and AAC or Opus for audio. H.264 and AAC are the most universally supported codecs in RTP muxing. If the HLS source uses H.265 (HEVC), RTP packetization is possible but less widely supported by downstream receivers. If the source uses an unusual codec like VP9 or AV1 in an HLS container, RTP copy will likely fail because FFmpeg's RTP muxer does not have a standard payload format for those codecs in copy mode.
Also check the video resolution, frame rate, and bitrate. While copy mode does not care about these values for the copy operation itself, they matter for bandwidth planning and for ensuring the downstream receiver can handle the stream. Note the audio sample rate and channel layout as well, since RTP timestamps for audio depend on the sample rate.

2. Build the FFmpeg Copy Command

Constructing the FFmpeg command for HLS to RTP copy involves several logical components. The first component is the input specification, where you provide the URL of the HLS playlist. FFmpeg's HLS demuxer will handle playlist parsing, segment downloading, and segment concatenation automatically.
The second component is the codec specification. You instruct FFmpeg to use copy mode for both video and audio streams. This tells FFmpeg to pass the encoded frames through without decoding or re-encoding. The copy flag applies to all streams by default, but you can also specify it per-stream if the source contains multiple tracks.
The third component is the output URL. For RTP output, the URL follows the RTP scheme and includes the destination IP address and port number. You can specify separate RTP outputs for video and audio if needed, or use a single output that multiplexes both. The output format is automatically inferred from the URL scheme by FFmpeg.
The fourth component involves optional transport and packetization parameters. You can specify the RTP transport protocol, adjust payload sizes, and set flags for handling live sources. These parameters affect how FFmpeg packetizes the media and sends it over the network.

3. Choose UDP vs TCP for RTP

The choice between UDP and TCP for RTP transport is one of the most consequential decisions in this pipeline. UDP is the traditional and recommended transport for RTP. It sends packets without establishing a connection and without retransmission guarantees. This means packets arrive with minimal delay, but if the network drops a packet, it is gone. For real-time applications, this is the right trade-off because a retransmitted packet would arrive too late to be useful anyway.
TCP, on the other hand, guarantees delivery through retransmission. If a packet is lost, TCP holds subsequent packets in a buffer until the lost packet is re-sent and acknowledged. This causes head-of-line blocking, where a single lost packet delays all subsequent media. For real-time streaming, this behavior is usually undesirable. However, TCP may be necessary when traversing firewalls that block UDP traffic or when network reliability is poor enough that UDP packet loss becomes unacceptable.
MTU considerations also matter. The default network MTU on most Ethernet networks is 1500 bytes. If RTP packets exceed this size, the network layer will fragment them, increasing the chance of packet loss because losing any fragment loses the entire packet. FFmpeg allows you to configure the maximum RTP packet size to stay within the MTU. A common setting is 1400 bytes, which leaves room for IP, UDP, and RTP headers.

4. Launch the Process and Monitor

Once the command is constructed, launch the FFmpeg process. FFmpeg will begin reading the HLS playlist, downloading segments, and sending RTP packets to the destination. The first thing to watch for in the FFmpeg logs is the initial connection to the HLS source. If the playlist URL is incorrect or the server is unreachable, FFmpeg will report a connection error immediately.
As the process runs, FFmpeg logs statistics including frame count, bitrate, and processing speed. In copy mode, the processing speed should be approximately real-time, meaning FFmpeg is not falling behind. If you see the speed drop below 1x, it may indicate network congestion on the HLS download side or issues with the source server.
To confirm that RTP packets are arriving at the destination, you can use a packet capture tool like tcpdump or Wireshark on the receiving machine. Filter for UDP traffic on the destination port and verify that packets are flowing. You can also use FFprobe on the receiving side to analyze the incoming RTP stream and check for codec information, packet loss indicators, and timing issues.
Watch for jitter warnings in the FFmpeg logs. High jitter can indicate network instability or insufficient buffering on the receiver side. If you see repeated packet loss warnings, investigate network conditions, MTU settings, and firewall rules.

5. Graceful Shutdown and Restart Strategies

Live HLS streams are dynamic. The playlist updates continuously as new segments are published, and old segments are removed from the sliding window. FFmpeg's HLS demuxer handles this automatically, but there are scenarios where the process may need to restart. For example, if the HLS source goes offline temporarily, FFmpeg may exhaust its retry attempts and exit.
To handle restarts gracefully, wrap the FFmpeg process in a supervisor system like systemd, supervisord, or a custom shell loop. The supervisor should detect when FFmpeg exits and restart it automatically. When FFmpeg restarts, it will re-read the playlist from the beginning and resume downloading the latest available segments.
One challenge with restarts is that the RTP timestamp sequence resets. Downstream receivers may interpret this as a discontinuity. Some receivers handle this gracefully by resetting their jitter buffers, while others may produce a brief audio or video glitch. To minimize this, consider using a persistent timestamp offset or configure the receiver to tolerate timestamp discontinuities.
For production deployments, consider running the FFmpeg process in a container with health checks. The health check can monitor the FFmpeg log output for errors and trigger a container restart if the process becomes unhealthy. This approach provides a clean, repeatable deployment pattern that integrates well with orchestration platforms like Kubernetes.

Common Pitfalls and Troubleshooting

Mismatched Codecs

The most common reason FFmpeg copy HLS to RTP fails is a codec mismatch between the HLS source and what the RTP muxer supports. If the HLS stream uses a video codec that does not have a defined RTP payload format, FFmpeg will report an error stating that the codec is not supported for RTP output. This typically happens with newer codecs like AV1 or with unusual configurations like H.264 with certain profiles that the RTP muxer cannot packetize.
When this happens, you have two options. The first is to fall back to transcoding. Instead of copy mode, instruct FFmpeg to decode the source and re-encode it into H.264 for video and AAC for audio before sending to RTP. This adds CPU overhead and latency but ensures compatibility. The second option is to change the source encoding if you control the HLS origin server. Switching the origin to output H.264 and AAC eliminates the problem at the source.

Network MTU and Fragmentation

Oversized RTP packets are a frequent source of packet loss in RTP pipelines. When FFmpeg packetizes a large video frame, such as a keyframe in high-bitrate 1080p content, the resulting RTP packet can exceed the network MTU. The IP layer then fragments the packet into smaller pieces. If any single fragment is lost in transit, the entire RTP packet is unrecoverable.
To prevent this, configure FFmpeg to limit the maximum RTP packet size. A safe value is 1400 bytes, which accounts for the 1500-byte Ethernet MTU minus the IP header (20 bytes), UDP header (8 bytes), and RTP header (12 bytes), with some additional margin for safety. This setting ensures each RTP packet fits within a single network frame and avoids fragmentation entirely.

Clock Drift and Synchronization

RTP relies on accurate timestamps for proper playback. The timestamps in RTP packets are relative to the sender's clock, and the receiver uses them to schedule playback and manage jitter buffers. If the sender's clock drifts relative to the receiver's clock, playback can speed up, slow down, or produce artifacts.
In an FFmpeg copy pipeline, the timestamps come from the HLS source segments. HLS segments are typically timestamped by the encoder at the origin, and these timestamps carry through to the RTP packets. If the origin encoder's clock is not synchronized, the RTP timestamps will reflect that drift. Using NTP (Network Time Protocol) on both the HLS origin server and the FFmpeg relay machine helps keep clocks aligned and reduces timestamp drift.
For multi-stream scenarios where audio and video travel as separate RTP streams, synchronization becomes even more important. RTCP (RTP Control Protocol) sender reports carry NTP timestamp information that receivers use to align audio and video playback. Ensure RTCP is not disabled in your FFmpeg configuration.

Firewall and NAT Issues

RTP over UDP is particularly susceptible to firewall and NAT problems. Many corporate and cloud firewalls block inbound UDP traffic by default. If the receiving machine is behind a firewall that blocks the RTP port, packets will be silently dropped and the receiver will see no media.
For simple deployments, the fix is to open the appropriate UDP port range on the receiving firewall. Remember that RTP uses an even port and RTCP uses the next odd port, so you need to open both. For NAT traversal, you may need port forwarding rules on the NAT device or a STUN/TURN server to relay traffic. The W3C WebRTC specification describes how STUN and TURN work in the context of real-time media, and the same principles apply to RTP streams.
If UDP traversal proves impossible, switching to TCP-based RTP is a practical fallback. While TCP introduces head-of-line blocking, it reliably traverses most firewalls because it uses a standard connection handshake that firewalls recognize and allow.

Performance Considerations

The performance difference between FFmpeg copy mode and full transcoding is significant. In copy mode, FFmpeg's CPU usage is dominated by I/O operations: downloading HLS segments over HTTP and sending RTP packets over the network. On a modern server, a single 1080p HLS to RTP copy stream typically uses less than 5 percent of one CPU core. The same stream transcoded to H.264 at the same resolution can consume 50 to 100 percent of a core depending on the encoder preset.
Latency measurements tell a similar story. Copy mode adds approximately 5 to 15 milliseconds of processing latency, which is the time required to demux the HLS segment and packetize frames into RTP. Transcoding adds 100 to 500 milliseconds per pass, depending on codec complexity and hardware acceleration availability. For pipelines that require multiple transcoding stages, the latency compounds at each stage.
Bandwidth consumption is identical between the source HLS stream and the output RTP stream in copy mode. The media payload is the same compressed data, just wrapped in different protocol headers. RTP headers add approximately 12 bytes per packet, which is negligible compared to the media payload size. This means you can accurately predict outbound bandwidth by measuring the inbound HLS bitrate.
For teams building real-time streaming applications that need to go beyond simple RTP relay, VideoSDK's real-time communication SDKs provide a higher-level abstraction with built-in network-adaptive streaming, participant management, and sub-second latency across 10+ platforms.

Alternatives to ffmpeg copy hls to rtp

FFmpeg is not the only tool capable of converting HLS to RTP. GStreamer is a powerful alternative that offers a pipeline-based multimedia framework. GStreamer's modular architecture allows you to construct processing pipelines from individual elements, giving you fine-grained control over each stage. For complex pipelines that require custom processing between demuxing and packetization, GStreamer may be a better fit. However, its learning curve is steeper than FFmpeg's command-line approach.
Wowza Streaming Engine is a commercial media server that can ingest HLS and output RTP among many other protocols. It provides a web-based management interface, transcoding capabilities, and robust clustering for large-scale deployments. If you need a full-featured media server rather than a lightweight relay, Wowza is worth evaluating.
The NGINX-RTMP module can handle some streaming protocol conversions, though its primary focus is on RTMP ingestion and HLS delivery. For HLS to RTP specifically, it is less capable than FFmpeg or GStreamer.
For developers building interactive live streaming or video calling features directly into applications, using a dedicated SDK like VideoSDK eliminates the need to manage protocol conversion manually. VideoSDK handles media transport, adaptive streaming, and participant management through a simple API across React, React Native, Flutter, Android, iOS, and JavaScript.

Best Practices Checklist

  • Verify source codecs with FFprobe before attempting copy mode to ensure H.264 video and AAC audio compatibility with RTP packetization.
  • Use UDP as the RTP transport protocol for lowest latency, and switch to TCP only if firewall traversal demands it.
  • Set the maximum RTP packet size to 1400 bytes to avoid IP fragmentation and reduce packet loss on standard Ethernet networks.
  • Monitor FFmpeg logs continuously for speed drops, jitter warnings, and packet loss indicators during live operation.
  • Open both the RTP even-numbered port and the RTCP odd-numbered port on the receiving firewall.
  • Synchronize clocks using NTP on both the HLS origin server and the FFmpeg relay machine to prevent timestamp drift.
  • Wrap the FFmpeg process in a supervisor system like systemd for automatic restart on failure.
  • Test the complete pipeline end to end with a real receiver before deploying to production.
  • Keep FFmpeg updated to the latest stable release to benefit from HLS demuxer and RTP muxer improvements.
  • Consider VideoSDK for application-level real-time streaming if you need participant management, adaptive streaming, and multi-platform SDK support beyond simple RTP relay.

Definitions Glossary

HLS (HTTP Live Streaming): A segment-based adaptive streaming protocol where media is broken into small files delivered over HTTP, indexed by a .m3u8 playlist. Typical latency ranges from 6 to 30 seconds.
RTP (Real-time Transport Protocol): A network protocol for delivering audio and video over IP in real time, packetizing media at the frame level with sequence numbers and timestamps for proper playback timing.
FFmpeg Copy Mode: A mode where FFmpeg passes encoded media streams through without decoding or re-encoding, preserving original quality and minimizing CPU usage and latency.
RTCP (RTP Control Protocol): A companion protocol to RTP that provides feedback on stream quality, packet loss statistics, and NTP timestamp information for audio-video synchronization.
MTU (Maximum Transmission Unit): The largest packet size that can be transmitted over a network without fragmentation, typically 1500 bytes on standard Ethernet networks.
Jitter Buffer: A buffer on the receiving side that compensates for variation in packet arrival times, ensuring smooth playback despite network jitter.

Key Takeaways

  • FFmpeg copy mode is the most efficient way to convert HLS to RTP because it skips decoding and encoding entirely, preserving original codec quality with near-zero CPU overhead.
  • Copy mode works only when the HLS source uses codecs compatible with RTP packetization, primarily H.264 for video and AAC for audio.
  • UDP is the recommended transport for RTP due to its low latency, but TCP is a viable fallback when firewalls block UDP traffic.
  • Setting the maximum RTP packet size to 1400 bytes prevents IP fragmentation and significantly reduces packet loss on standard networks.
  • Clock synchronization via NTP and RTCP sender reports is essential for maintaining proper playback timing and audio-video sync in RTP pipelines.
  • For application-level real-time streaming with participant management and multi-platform support, VideoSDK's SDKs provide a higher-level alternative to manual protocol conversion.

Conclusion

Copying HLS to RTP with FFmpeg is one of the most lightweight and effective ways to bridge HTTP-based adaptive streaming into real-time packet-based delivery. By leveraging copy mode, you avoid the CPU cost and latency of transcoding while preserving the original stream quality. The pipeline is simple to set up, easy to monitor, and scalable enough for production workloads on modest hardware. Whether you are relaying surveillance camera feeds into a monitoring dashboard, feeding live sports into a WebRTC gateway, or building a custom media pipeline, the ffmpeg copy hls to rtp workflow provides a reliable foundation. For developers who need to go beyond protocol relay and build interactive streaming or video calling into their applications, VideoSDK offers comprehensive SDKs with built-in network-adaptive streaming and sub-second latency. You can start building for free at app.videosdk.live/login. What are you building with real-time streaming? Drop a comment below, I would love to hear about your HLS to RTP use case.

Free $20 Balance for AI Voice Agents & Video Calls

FAQ