Advanced audio coding (AAC) is a lossy audio codec standardized by the Moving Picture Experts Group (MPEG) for MPEG-2 and MPEG-4. Its encoder uses a psychoacoustic model to discard sound a listener is unlikely to hear. In live streaming, AAC is the usual audio codec for broadcast delivery, while real-time calls use Opus.
This guide is for developers who meet AAC in an encoder, an HTTP Live Streaming (HLS) playlist or a Real-Time Messaging Protocol (RTMP) ingest spec. It covers how the codec works, its profiles and codec strings, what an .aac file holds, which protocols carry AAC, and how to choose an encoder and bitrate.
How does advanced audio coding work?
Advanced audio coding is a perceptual codec: it splits the signal into frequency bands, estimates what the ear will notice, and quantizes the rest coarsely or drops it. According to the Open University's OpenLearn section on MPEG-4 AAC, AAC uses either 2048 or 256 sub-bands, against 32 for MP3 (MPEG-1 Audio Layer III), which gives the psychoacoustic model finer frequency resolution.
OpenLearn names the deciding step perceptual masking: a louder sound hides quieter ones close to it in frequency, so the encoder codes the hidden parts with fewer bits, or none. Three coding tools extend this at low bitrates: perceptual noise substitution (PNS), spectral band replication (SBR) and parametric stereo (PS).
Because AAC is lossy, each decode and re-encode step loses more quality. AAC also adds delay. Fraunhofer IIS notes on its AAC-ELD family page that conventional AAC's encoder and decoder chain exceeds 100 ms of algorithmic delay, which it calls not acceptable for two-way communication. That is one reason real-time calls use a different codec.
What is the AAC audio format?
The AAC audio format is a codec, not a file layout. The short answer to what is AAC format, or what is AAC music format, is that the name describes the compression; a container decides how the compressed frames are stored (see the difference between a codec and a container). MDN's web audio codec guide lists three containers for AAC, MP4, ADTS (Audio Data Transport Stream) and 3GP, with bitrates up to 512 kbps and sample rates of 8 to 96 kHz.
Inside a playlist or a media type, AAC is identified by a codec string. RFC 6381 defines the syntax: in mp4a.40.2, 40 means MPEG-4 audio and 2 is the audio object type for AAC-LC (Low Complexity). An HLS player reads this string from the CODECS attribute, and RFC 8216, section 4.3.4.2, requires that attribute to list every sample format in the variant, for example ac-3,mp4a.40.2.
AAC formats compared: AAC-LC, HE-AAC and xHE-AAC
AAC formats, more precisely profiles or object types, add coding tools to the core codec for lower bitrates or lower delay. The table compares five.
| Profile | Codec string | What it adds | Where it fits | Source |
|---|---|---|---|---|
| AAC-LC | mp4a.40.2 | The core codec: up to 256 kbit/s per channel, 8 to 96 kHz | FFmpeg's default and most compatible profile | Fraunhofer AAC-LC page; RFC 6381 |
| HE-AAC v1 (High Efficiency) | mp4a.40.5 | SBR, a bandwidth extension for high frequencies | Low-bitrate stereo | MDN codecs parameter guide; Via LA |
| HE-AAC v2 | mp4a.40.29 | PS on top of SBR | Very low-bitrate stereo | MDN; Via LA |
| xHE-AAC (Extended HE-AAC) | Not in MDN's object-type table | Speech coding plus mandatory MPEG-D loudness and dynamic range control | Mixed speech and music at low rates; fragmented MP4 (fMP4) only, in Apple HLS | Via LA; Apple rule 2.25 |
| AAC-LD and AAC-ELD (Low Delay, Enhanced Low Delay) | mp4a.40.23, mp4a.40.39 | Low delay; ELD reaches 7.5 ms at 48 kHz in delay-reduced mode | Two-way calls, such as FaceTime | Fraunhofer AAC-ELD page; MDN |
The key reading is the bitrate boundary. Apple's HLS authoring specification for Apple devices says you should not use HE-AAC above 64 kbit/s (rule 2.4), and rule 2.3 requires stereo audio in AAC, HE-AAC v1 or HE-AAC v2. Above 64 kbit/s, AAC-LC is the profile to use.
What is an AAC file?
An AAC file is a .aac file holding an ADTS stream. MDN's media container guide describes ADTS as an almost bare stream of AAC data, made of frames with a minimal header, served as audio/aac and intended for streamed audio such as internet radio.
The same MDN guide lists .m4a and .mp4 under the MP4 container, audio/mp4. So the answer to what is an AAC audio file depends on the extension: raw ADTS in a .aac file, AAC inside MP4 in an .m4a file.
The distinction matters in HLS. RFC 8216, section 3.4, defines packed audio segments of AAC with ADTS framing, each starting with an ID3 tag that holds its timestamp. HLS can also carry AAC inside MPEG transport stream (TS) or fMP4 segments (sections 3.2 and 3.3).
Where AAC audio fits in HLS, RTMP and WebRTC
AAC audio is the codec both broadcast protocols name, and one that Web Real-Time Communication (WebRTC) does not require. The table lists the audio codecs each specification or ingest guide names.
| Protocol | Audio codecs named | Carries AAC? | Source |
|---|---|---|---|
| HLS | AAC, MP3, AC-3, E-AC-3 as packed audio; Apple requires stereo AAC or HE-AAC | Yes, required by Apple | RFC 8216 section 3.4; Apple rule 2.3 |
| RTMP, YouTube ingest | AAC or MP3; 5.1 surround only as AAC | Yes | YouTube live encoder settings, as of 29 Sep 2026 |
| Enhanced RTMP v2 | Adds AC-3, E-AC-3, Opus and FLAC to legacy RTMP | Yes | Enhanced RTMP repository |
| WebRTC | Endpoints must implement Opus and G.711 | Not required | RFC 7874 section 3 |
The reading: an app with a live call and a broadcast audience runs two audio paths, and the bridge between them is where Opus becomes AAC. For the protocols themselves, see how HLS segments and playlists work and how RTMP carries audio and video to an ingest server. The video track is a separate choice; see how AV1 and H.264 compare for the video track.
A room carries Opus end to end; the HLS rendition needs stereo AAC for Apple devices, so the bridge from room to HLS re-encodes the audio once.
How to choose an AAC encoder and bitrate for a live stream
An AAC encoder choice depends on the profiles you need and what your build can include. FFmpeg's codec documentation (development version, page dated 28 September 2026) describes two options:
- The native
aacencoder is FFmpeg's default AAC codec. Setting a bitrate with-b:aswitches it to constant bit rate (CBR), and without one it uses 128 kbps. Its profiles,aac_low(the default and most compatible),mpeg2_aac_lowandaac_ltp, include no HE-AAC. libfdk_aacwraps the Fraunhofer FDK library and addsaac_he,aac_he_v2,aac_ldandaac_eld. The build must be configured with--enable-libfdk-aac, and because the library is incompatible with the GNU General Public License (GPL), a GPL build also needs--enable-nonfree.
For bitrate, Apple's HLS specification gives 32 to 160 kbit/s for stereo AAC. YouTube recommends 128 kbps stereo at 44.1 kHz. MDN gives 96 kbps at 48 kHz as the recommended minimum for stereo AAC. For rate-control modes, see constant vs variable bitrate modes.
A decision rule that follows from these sources:
- Opus on the real-time leg, because RFC 7874 requires it in every WebRTC endpoint.
- AAC-LC stereo at 96 to 160 kbit/s for HLS and RTMP delivery: MDN's minimum up to Apple's upper bound.
- HE-AAC only for renditions at 64 kbit/s or below, such as an audio-only fallback (Apple rule 2.4).
- AAC-ELD only for two-way calling, the use Fraunhofer describes.
Handling AAC and Opus when you build on a live streaming SDK
A platform serving an interactive room and an HLS audience runs both paths in the diagram. Hosts talk over WebRTC, where Opus is mandatory, and the platform must produce AAC for the HLS rendition, because Apple's rule 2.3 requires stereo AAC. The HLS playlist's codec string shows what the platform chose.
VideoSDK splits the same job into interactive live streaming (ILS) and HLS. In an ILS room, hosts join with SEND_AND_RECV and the audience with RECV_ONLY. To reach an HLS audience, a participant calls startHls from useMeeting, whose mode option is "video-and-audio" or "audio", per the Start HLS guide. The docs do not state the audio codec, profile or bitrate of that HLS output
WHIP (WebRTC-HTTP Ingestion Protocol) lets an encoder such as OBS push a stream into a room, where it appears as a participant, per the WHIP ingest guide. Because WHIP carries a WebRTC session, the encoder offers WebRTC audio codecs, not the AAC it would send over RTMP. RTMP out pushes the room to platforms such as YouTube, which accepts AAC or MP3.
Glossary
ADTS (Audio Data Transport Stream): The frame-by-frame AAC stream format used by .aac files and HLS packed audio. RFC 8216 cites ISO/IEC 13818-7, the MPEG-2 AAC standard, for its definition.
SBR (spectral band replication): A bandwidth extension tool that codes the high frequencies of a signal with little data. It is the tool HE-AAC v1 adds to AAC-LC.
Parametric stereo (PS): A tool that analyzes the spatial relationship between the left and right channels and exploits what they share. HE-AAC v2 adds it to SBR.
Perceptual noise substitution (PNS): A tool that replaces noise-like high-frequency bands with generated noise at the decoder. FFmpeg's native encoder enables it by default.
Opus: An open audio codec standardized by the Internet Engineering Task Force (IETF) as RFC 6716 and required in every WebRTC endpoint. See why Opus is the default codec for real-time audio.
Key takeaways
- Advanced audio coding is MPEG's lossy perceptual audio codec, and AAC-LC (
mp4a.40.2) is its most compatible profile. - AAC is a codec, not a container: a .aac file holds raw ADTS frames, while .m4a and .mp4 files hold AAC inside MP4.
- HLS and RTMP ingest name AAC for audio, while WebRTC requires Opus, so a room re-broadcast over HLS is re-encoded once at the bridge.
- Use AAC-LC stereo at 96 to 160 kbit/s for delivery, and HE-AAC only at 64 kbit/s or below, per Apple's HLS rules.
Next, check the CODECS strings your pipeline produces and your encoder's audio settings against these rules. To build a live stream with an interactive room and an HLS audience, start with the VideoSDK interactive live streaming docs; a new account comes with a $20 credit.
Frequently asked questions
What is advanced audio coding?
Advanced audio coding is a family of lossy audio compression formats defined in ISO/IEC 13818-7 (MPEG-2) and ISO/IEC 14496-3 (MPEG-4). It codes what the ear is likely to notice and discards much of the rest. Its profiles range from AAC-LC for general delivery to HE-AAC for low bitrates and AAC-ELD for calls.
Is AAC better than MP3?
Generally yes, at the same bitrate. OpenLearn reports a broad consensus that AAC gives better subjective quality than MP3 at the same bit rate, and MDN notes that AAC was designed to give more compression with higher fidelity than MP3. At low bitrates, OpenLearn says HE-AAC delivers near-CD quality at 64 kbit/s.
Do you need a license to stream AAC audio?
No patent license fee is due for streaming AAC content. The Via LA AAC program's FAQ states that fees apply to the sale of encoders and decoders only, not to distributing AAC bitstreams by broadcast, network streaming or physical media. MDN agrees, adding that codec developers license through Via.