How does HLS adaptive streaming work, why do stalls happen, and what does FairPlay add?

18.4.1 · L3 · Interview

Predict first

A master playlist has three variants. Which does a simple 80% rule pick at 3.5 Mbps and at 0.7 Mbps?

let master = """
#EXTM3U
#EXT-X-STREAM-INF:BANDWIDTH=800000,RESOLUTION=640x360
low.m3u8
#EXT-X-STREAM-INF:BANDWIDTH=2500000,RESOLUTION=1280x720
mid.m3u8
#EXT-X-STREAM-INF:BANDWIDTH=6000000,RESOLUTION=1920x1080
high.m3u8
"""
func variants(_ text: String) -> [(bandwidth: Int, uri: String)] {
    var out: [(Int, String)] = []
    var pending: Int?
    for line in text.split(separator: "\n") {
        if line.hasPrefix("#EXT-X-STREAM-INF:"), let r = line.range(of: "BANDWIDTH=") {
            pending = Int(line[r.upperBound...].prefix { $0.isNumber })
        } else if !line.hasPrefix("#"), let bw = pending {
            out.append((bw, String(line)))
            pending = nil
        }
    }
    return out.sorted { $0.0 < $1.0 }
}
func pick(_ vs: [(bandwidth: Int, uri: String)], measured: Int) -> String {
    let budget = Double(measured) * 0.8
    return vs.last { Double($0.bandwidth) <= budget }?.uri ?? vs.first!.uri
}
let vs = variants(master)
print(pick(vs, measured: 3_500_000), pick(vs, measured: 700_000))

mid.m3u8 low.m3u8
high.m3u8 low.m3u8
mid.m3u8 none

Mental model. HLS is a delivery of a film in many small parcels, in several sizes. The player (courier) looks at how fast parcels are arriving and picks the next parcel size accordingly, and a stall is the screen going blank because the next parcel has not arrived.

HTTP Live Streaming (HLS) delivers media as ordinary HTTP downloads of short segments (a few seconds each), described by text playlists (.m3u8). A master playlist lists variants: the same content encoded at several bitrates and resolutions, each line #EXT-X-STREAM-INF:BANDWIDTH=2500000,RESOLUTION=1280x720 followed by the URI of that variant’s media playlist, which lists the segments in order (#EXTINF:6.0, then the segment URI), with #EXT-X-ENDLIST for on-demand video and none for live, where the playlist keeps growing. Because it is just HTTP, HLS rides existing CDNs and caches and works through firewalls. Adaptive bitrate (ABR) is the client’s job: AVPlayer measures recent throughput and the buffer level, and switches between variants at segment boundaries, preferring the highest bandwidth that fits with headroom (a simplified rule: pick the largest variant whose declared BANDWIDTH is at most about 80% of the measured throughput, and fall back to the lowest). Players start conservatively (low quality to start fast) and then ramp up. You influence it with preferredPeakBitRate, preferredMaximumResolution, and by ordering variants sensibly; AVPlayerItem.accessLog() and errorLog() report the variants played, bitrate switches, stalls and errors.

Stalls (rebuffering) happen when the buffer drains faster than the network refills it. Causes: throughput lower than the lowest variant, a variant ladder that jumps too far, large segments, slow CDN or DNS, a cold cache, radio changes on cellular, and a player that is too aggressive at upswitching. Symptoms in code: timeControlStatus == .waitingToPlayAtSpecifiedRate with reasonForWaitingToPlay == .toMinimizeStalls (a buffer-filling wait) or .evaluatingBufferingRate. Mitigations: include a low-bitrate variant (and an audio-only one), keep segments short (about 6 s), use HTTP/2 or 3 and a good CDN, preload with preferredForwardBufferDuration when appropriate (the default of 0 lets the system choose), allow automaticallyWaitsToMinimizeStalling (the default), and measure it. FairPlay Streaming is Apple’s DRM for HLS: segments are encrypted, the playlist references a key (#EXT-X-KEY with the skd:// scheme), and the app, through AVContentKeySession, requests a content key from your key server (which verifies entitlement and returns a key wrapped for that device), so only authorised devices can decrypt, with offline keys for downloads; it needs an agreement and certificate from Apple and a key-server (KSM) implementation, and playback of protected content has restrictions (screen capture, external displays). Other HLS features: subtitles and alternate audio via AVMediaSelectionGroup, low-latency HLS for live, and downloads with AVAssetDownloadURLSession.

80% of 3.5 Mbps is 2.8 Mbps, so the 2.5 Mbps variant fits. At 0.7 Mbps nothing fits, so the player falls back to the lowest variant.

Common traps

Offering only high-bitrate variants. Users on slow networks stall; include a low variant and an audio-only one.
Forcing a high peak bit rate to “improve quality”. It increases stalls on weaker networks; let ABR adapt within sensible caps.

They might follow up with

What does a master playlist list? The variants, each with a declared bandwidth and resolution, pointing to media playlists.
What does FairPlay add to HLS? Encrypted segments with keys delivered per device by your key server.

How does HLS adaptive streaming work, why do stalls happen, and what does FairPlay add?

Predict first

Common traps

They might follow up with

Related questions