Design the client side of a video player: startup, adaptive bitrate selection, buffer targets and why switching rules need smoothing and hysteresis.

26.5.2 · L5 · Interview

Predict first

A link flips between about 5 and 1 Mbit/s. How do “last sample” and “smoothed with hysteresis” compare?

let ladder: [Double] = [0.4, 0.8, 1.6, 3.2, 6.0]            // Mbit/s
let samples: [Double] = [5, 1.2, 5, 0.9, 5, 1.2, 5, 0.9, 5, 1.2, 5, 0.9, 5, 1.2, 5, 5, 5, 5, 5, 5]   // throughput during each 4 s segment
func simulate(smooth: Bool) -> (levels: [Int], stalls: Int) {
    var est = samples[0], level = 0, buffer = 8.0, upStreak = 0, stalls = 0
    var out: [Int] = []
    for s in samples {
        if smooth {
            est = 0.7 * est + 0.3 * s                                  // EWMA
            let target = ladder.lastIndex { $0 <= est * 0.8 } ?? 0     // 20% safety margin
            if target < level { level = target; upStreak = 0 }         // down at once
            else if target > level { upStreak += 1; if upStreak >= 3 && buffer > 10 { level += 1; upStreak = 0 } }
            else { upStreak = 0 }
        } else {
            level = ladder.lastIndex { $0 <= est } ?? 0                // pick from the previous sample only
            est = s
        }
        buffer += 4 - ladder[level] * 4 / s                            // seconds fetched minus seconds played
        if buffer < 0 { stalls += 1; buffer = 0 }
        buffer = min(buffer, 30)
        out.append(level)
    }
    return (out, stalls)
}
func switches(_ l: [Int]) -> Int { zip(l, l.dropFirst()).filter { $0 != $1 }.count }
for (name, smooth) in [("last sample", false), ("smoothed  ", true)] {
    let r = simulate(smooth: smooth)
    print(name, r.levels.map(String.init).joined(), "switches", switches(r.levels), "stalls", r.stalls)
}

Last sample: 14 switches and 6 stalls; smoothed: 3 switches and no stalls
Both switch the same number of times
Smoothed stalls more because it reacts slowly

Mental model. Driving in fog: you do not floor the pedal because the last ten metres were clear. You smooth your guess, leave a margin, brake at once and speed up only when the road has stayed clear for a while.

Playback quality is three numbers: startup time, rebuffering (stalls) and picture quality, and they trade against each other. Start with a low-to-middle rendition so the first frames arrive quickly, then climb. The player holds a buffer of seconds ahead of the playhead; a low buffer means stall risk, a very large one wastes data if the user leaves. Adaptive bitrate (ABR) picks, per segment, a rendition from a ladder (here 0.4 to 6 Mbit/s) using measured throughput and buffer level. With HLS, AVPlayer already does this for you; you tune it with the authored ladder and the player item’s preferredPeakBitRate, preferredMaximumResolution and preferredForwardBufferDuration, and you observe it through the access log and your own telemetry (Q400). A custom player or a downloader needs its own ABR, and the logic is the same.

The check simulates 20 four-second segments over a link that flips between 5 and about 1 Mbit/s. Picking a rendition from the last sample flaps 14 times and stalls 6 times, because each fast sample sends it to a rendition the next slow sample cannot sustain. Three rules fix it: smooth the estimate (an exponentially weighted average), apply a safety margin (use about 80 percent of the estimate), and make switching asymmetric: go down at once when the buffer or estimate says so, but go up only after several good segments and a healthy buffer (hysteresis). The same link gives 3 switches and no stalls. Real systems also use buffer-based rules (below a floor choose the lowest rendition, regardless of throughput), per-network ceilings (Low Data Mode, cellular, allowsConstrainedNetworkAccess), and metrics: time to first frame, rebuffer ratio, average bitrate, switch count, and exit-before-start. The simulation is a model with made-up numbers; the point is the shape of the rule, not the constants, which you tune from field data.

Smoothing, a margin and asymmetric switching trade a little quality for far fewer switches and stalls.

Common traps

Choosing a rendition from the latest throughput sample. It oscillates and causes stalls on variable links.
Optimising bitrate only. Viewers forgive a softer picture far more than a stall or a long start.

They might follow up with

What do you log? Time to first frame, rebuffer count and duration, average and peak bitrate, switches, errors and startup failures, by network type and device (Q400).
Why start low? The first segment sets time to first frame; you have no throughput estimate yet.

Design the client side of a video player: startup, adaptive bitrate selection, buffer targets and why switching rules need smoothing and hysteresis.

Predict first

Common traps

They might follow up with

Related questions