Scroll-driven video — where scroll position drives video.currentTime — is one of the most cinematic effects on the modern web. It's also one of the easiest to get catastrophically wrong. The difference between buttery and broken comes down to two things: how the video is encoded, and how the seek is driven.
The encoding secret: GOP = 1
Video codecs store most frames as diffs from a previous keyframe. When you seek, the browser has to walk back to the nearest keyframe and decode forward — for a typical web encode that can mean decoding dozens of frames for a single seek. Do that sixty times per second while scrolling and you get a slideshow.
The fix is re-encoding with a keyframe on every frame: ffmpeg -g 1. File size grows, but every seek becomes a single-frame decode. For a short hero clip the trade is worth it every time.
Driving the seek
Don't write currentTime directly in a scroll handler. Use a tween proxy: animate a plain object's time property with GSAP ScrollTrigger scrub, and copy it to the video in onUpdate only when the delta is meaningful. You get GSAP's interpolation and scrub smoothing for free, and you avoid spamming the decoder with sub-frame seeks.
Respect the escape hatches
Honour prefers-reduced-motion with a static poster. Pause the whole system when the section is off-screen. And measure on a mid-range Android phone, not your MacBook — that's where jank lives.