Learn · Developers

Variable frame rate video: how to detect and convert it

Contents

A variable frame rate (VFR) video doesn’t show its frames at even intervals. Some frames stay on screen longer than others. Players handle that well, but editing tools and video pipelines often don’t, and the sound drifts away from the picture.

This guide explains what variable frame rate is and how to detect it with ffprobe and FFmpeg. It then shows two ways to convert it to a constant frame rate (CFR): the FFmpeg fps filter and a render with Shotstack’s video editing API. We tested each step on clips we built for the purpose.

TL;DR

  • In a VFR video, every frame carries its own timestamp and the gaps between frames change.
  • Compare r_frame_rate and avg_frame_rate in ffprobe for a quick check. Count the frame durations for a sure one.
  • FFmpeg’s vfrdet filter counts changes in frame duration, so a small number can still mean a variable file.
  • ffmpeg -vf fps=25 converts to a constant frame rate. In our test it kept every frame in its place.
  • A tool that ignores the timestamps played our 12 second test clip in 9.33 seconds, with the sound left 2.67 seconds behind.

What is variable frame rate video?

In a constant frame rate video, every frame is on screen for the same time. At 25 frames per second (fps), that is 40 milliseconds each.

In a variable frame rate video, each frame has its own timestamp and the gaps between them change. Phones are the most common source. Wikipedia’s entry on variable frame rate notes that “many smartphone cameras record their videos in VFR”.

Recorders do it for two reasons. A lower frame rate in the dark gives each frame a longer exposure. Skipping frames that don’t change, such as a still screen, makes the file smaller.

Why variable frame rate breaks audio sync

Sound runs on a steady clock. A VFR picture stays in step with it only when each frame is shown at its own timestamp.

Editing tools work on a fixed grid of frames, like the frame rates you choose for an edit. A tool that places VFR frames on that grid one after another, without reading the timestamps, plays the slow parts too fast.

We measured it with a 12 second test clip at 30 fps, where each frame carries its own number. From 4 to 8 seconds it keeps only every third frame, so that part runs at 10 fps.

Read as a plain 30 fps file, the 280 frames played in 9.33 seconds while the sound still ran for 12.

How to detect variable frame rate video

There are three checks, from quick to thorough. The sample outputs below come from a 10 second VFR clip we made from 25 fps footage. It drops to a third of its frame rate between seconds 4 and 7.

Compare the two frame rates in ffprobe

ffprobe reports a base frame rate and an average frame rate for a video stream:

Bash
ffprobe -v error -select_streams v:0 -show_entries stream=r_frame_rate,avg_frame_rate,nb_frames -of default=noprint_wrappers=1 sample-vfr.mp4
Text
r_frame_rate=25/1
avg_frame_rate=20/1
nb_frames=200

The base rate is 25 fps, but 200 frames in 10 seconds average 20 fps. A clear difference like this means the frame rate varies. A difference of a few hundredths doesn’t: one constant 25 fps clip we checked reports an average of 25.04.

Count the frame durations

For a sure answer, list how long each frame lasts and count the values:

Bash
ffprobe -v error -select_streams v:0 -show_entries frame=duration_time -of default=noprint_wrappers=1:nokey=1 sample-vfr.mp4 | sort | uniq -c
Text
    175 0.040000
     25 0.120000

A constant file has one value. This one has 175 frames of 40 milliseconds and 25 frames of 120 milliseconds, so it is variable.

Run FFmpeg’s vfrdet filter

FFmpeg’s vfrdet filter reads the whole stream and prints a summary when it ends:

Bash
ffmpeg -i sample-vfr.mp4 -vf vfrdet -an -f null -
Text
[Parsed_vfrdet_0 @ 00000223c6a638c0] VFR:0.010050 (2/197) min: 512 max: 1536 avg: 1024

The two numbers in brackets are frames with a variable gap and frames with a constant gap. The count is low because the filter counts changes: this clip changes pace twice, at 4 and 7 seconds. Any number above zero means the file is variable.

min and max are the shortest and longest gaps in the stream’s time base. Here that is 12,800 ticks a second, so 512 is 40 milliseconds and 1536 is 120.

Check a file by URL

For a file that is already online, Shotstack’s Probe endpoint returns the same ffprobe fields as JSON, so you can compare the two rates without downloading the file. Our guide to inspecting media metadata covers the full response. The script later in this article uses it.

Convert variable frame rate to constant with FFmpeg

FFmpeg’s fps filter converts a video to a constant frame rate. It reads each frame’s timestamp, then repeats or drops frames to fill an even grid:

Bash
ffmpeg -i sample-vfr.mp4 -vf fps=25 -c:a copy sample-cfr.mp4

Use the base rate from ffprobe (r_frame_rate) as the target, so no real frame is dropped. The sound is copied as it is. After the conversion, the same checks report a constant file:

Text
r_frame_rate=25/1
avg_frame_rate=25/1
nb_frames=250
[Parsed_vfrdet_0 @ 000002c39d410e00] VFR:0.000000 (0/249)

The 25 long frames became 75 frames of normal length, which brings the total from 200 to 250.

Render at a constant frame rate with the Shotstack API

Shotstack’s Edit API renders every video at one constant frame rate, set by output.fps (25 by default). When your footage goes into an edit anyway, the render gives you a constant frame rate file.

This Python script probes a video URL, renders it at a constant 25 fps, then probes the result. It uses only the standard library. Set SHOTSTACK_API_KEY to the API key from your Shotstack dashboard, then run it with Python 3.

Python
import json
import os
import sys
import time
import urllib.error
import urllib.parse
import urllib.request

API = "https://api.shotstack.io/edit/v1"
SOURCE = "https://d2jn8jtjz02j0j.cloudfront.net/sample_vfr_946ef404f8.mp4"
FPS = 25  # the constant frame rate to render at


def call(url, body=None):
    data = json.dumps(body).encode() if body else None
    headers = {
        "x-api-key": os.environ["SHOTSTACK_API_KEY"],
        "Content-Type": "application/json",
    }
    try:
        request = urllib.request.Request(url, data, headers)
        with urllib.request.urlopen(request) as response:
            return json.load(response)["response"]
    except urllib.error.HTTPError as error:
        sys.exit(f"HTTP {error.code}: {error.read().decode()[:300]}")


def rate(fraction):
    """Turn an ffprobe rate such as "30000/1001" into a number."""
    top, _, bottom = fraction.partition("/")
    return float(top) / float(bottom or 1) if float(bottom or 1) else 0.0


def frame_rates(url):
    """Probe a media URL and return its base and average frame rates."""
    probe = call(f"{API}/probe/{urllib.parse.quote(url, safe='')}")
    streams = probe["metadata"]["streams"]
    video = next(s for s in streams if s["codec_type"] == "video")
    return rate(video["r_frame_rate"]), rate(video["avg_frame_rate"])


def report(label, url):
    base, average = frame_rates(url)
    kind = "variable" if abs(base - average) > 0.01 * base else "constant"
    print(f"{label}: base {base:.2f} fps, average {average:.2f} fps -> {kind}")


if __name__ == "__main__":
    if not os.environ.get("SHOTSTACK_API_KEY"):
        sys.exit("Set SHOTSTACK_API_KEY first.")
    report("Source", SOURCE)
    clip = {"asset": {"type": "video", "src": SOURCE}, "start": 0, "length": "auto"}
    edit = {
        "timeline": {"tracks": [{"clips": [clip]}]},
        "output": {"format": "mp4", "resolution": "hd", "fps": FPS},
    }
    render_id = call(f"{API}/render", edit)["id"]
    print("Render queued:", render_id)
    while True:
        time.sleep(5)
        status = call(f"{API}/render/{render_id}")
        if status["status"] == "done":
            break
        if status["status"] == "failed":
            sys.exit(f"Render failed: {status.get('error')}")
    report("Output", status["url"])
    print(status["url"])

Run python vfr_to_cfr.py. It prints the frame rates before and after, then the URL of the rendered video:

Text
Source: base 25.00 fps, average 20.00 fps -> variable
Render queued: c1a74e98-d797-4243-9205-868948ed3f8a
Output: base 25.00 fps, average 25.00 fps -> constant
https://shotstack-api-v1-output.s3-ap-southeast-2.amazonaws.com/35tqpmb0ya/c1a74e98-d797-4243-9205-868948ed3f8a.mp4

This is the rendered video, 250 evenly spaced frames:

  • frame_rates() calls the Probe endpoint and reads r_frame_rate and avg_frame_rate from the video stream.
  • report() calls a file variable when the two rates differ by more than 1%.
  • The clip’s length is "auto", so the edit plays the whole source.
  • Set FPS to the source’s base rate, or to the rate your project needs.

What we measured

We ran the numbered test clip through each method and read the number back from every output frame. That shows which source frame is on screen at each moment.

MethodOutputLargest gap from the source’s timing
Read as plain 30 fps, timestamps ignored280 frames, 9.33 s of picture, 12 s of sound80 frames (2.67 s)
FFmpeg fps=30360 frames, 12 s, constant0 frames
Edit API render, output.fps 30360 frames, 12 s, constant7 frames (0.23 s), at the very end
  • Both conversions kept the picture the same length as the sound, with every frame 33 milliseconds apart.
  • The FFmpeg fps filter put every source frame exactly where its timestamp said. Convert with it first when frame accuracy matters.
  • The beeps in the test clip’s sound stayed within 0.04 seconds of their place in every output.

Make frame rate checks part of your video pipeline with Shotstack

Variable frame rate video is normal for phone footage, and it only causes trouble when a tool ignores the timestamps. Check the two frame rates before you edit, count the frame durations when you need to be sure, and convert to a constant frame rate with FFmpeg or with a render.

Create a free Shotstack account to get your API key, probe your own footage and render it at a constant frame rate.

Frequently asked questions (FAQs)

Does converting to a constant frame rate lower the quality?

The conversion re-encodes the picture, so set the quality you need, as you would for any encode. It adds no new motion. Frames that lasted longer in the source are repeated, so a part recorded at 10 fps still looks like 10 fps.

Which constant frame rate should I choose?

Use the source’s base rate, the r_frame_rate that ffprobe reports, when you want to keep every frame. Use your project’s rate when the clip joins other footage. Going from 30 to 25 fps drops one frame in six.

Is variable frame rate bad for playback?

No. Players read the timestamps and show each frame at the right time, so a VFR file plays correctly. The trouble starts in editing and processing tools that expect an even grid of frames.

Keep reading

Start free

  • Unlimited developer sandbox
  • No credit card required