Mute a video or replace its audio with Python and the Shotstack API

Summarize this article in Claude ChatGPT Perplexity Google AI

Muting a video sounds like one job. In practice it is three different jobs: silence the original sound, remove the audio track from the file, or swap the sound for something else. Each one is a different setting in the Edit API, and each one leaves a different file behind.

This guide covers all three, plus a fourth that comes up just as often, mixing music under the original audio. One Python script renders each version, and the probe endpoint reads back what the finished file contains. The numbers in the tables come from those renders.

TL;DR

• Set volume: 0 on the video asset to silence it. The output keeps a silent audio track.
• Set output.mute: true to remove the audio track from the file entirely.
• To replace the sound, silence the video and add an audio asset on its own track.
• To mix, leave the video at full volume and add the music at a low volume, 0.2 in this test.
• The script below does all four with the standard library, no SDK needed.

Where the sound lives in a render

An Edit API render is a JSON timeline of tracks and clips. Sound comes from two places, and there is a third setting that acts on the finished file.

• The video asset’s own audio. Every video asset has a volume between 0 and 1, where 0 is muted and 1 is full volume. The API reference gives 1 as the default.
• Audio assets. An audio asset is a clip like any other. It takes a src URL, a trim point, a volume, a speed and a fade effect. Put it on its own track so it does not compete for time with the video clips.
• output.mute. A boolean on the output block that mutes the audio track of the whole output video.

The older timeline.soundtrack field still appears in some tutorials. The API reference marks it deprecated and points to an audio asset on its own track instead, so none of the examples below use it.

Mute a video

There are two ways to do it, and they are not the same.

Setting volume: 0 on the video asset silences that clip. The output still has an audio track, it is just silent. Setting output.mute: true goes further and drops the audio track from the file. The probe endpoint shows the difference below.

Use volume: 0 when other clips on the timeline still need their sound, or when you plan to add music. Use output.mute when the file must have no audio at all, for example a background loop on a web page.

For feed video the choice costs little either way, because most of the audience never hears it. In LinkedIn’s 2017 survey of its members, 56% of people watching video on a laptop or desktop typically listen to it as well, but only 37% of those watching on mobile do.

Replace the audio

Replacing means two changes in one edit: the video asset at volume: 0, and an audio asset on a second track starting at the same time. In the script this is the replace mode.

Two details matter:

• Length. Audio plays until the file ends or the clip length is reached. Set the audio clip’s length to match the video, or pass "end" to run it to the end of the timeline. A music file shorter than the clip leaves silence after it.
• Fades. The audio asset’s effect takes fadeIn, fadeOut or fadeInFadeOut. The test uses fadeOut so the music does not stop dead at ten seconds.

Replacing is the right call when the original sound carries the meaning and you are swapping in a voiceover or a narration track.

Meta’s guidance for video ads makes the point: in one study of Facebook video ads, 41% of videos were basically meaningless without sound, and captioned ads increased view time by an average of 12%. If that describes your video, replace the audio or burn in captions rather than muting it.

Mix music under the original

Keep the video asset at volume: 1 and add the music at a low volume. The test uses 0.2 and the dialogue stays clear over it.

Watch the headroom. The mixed output in this test peaked at 0.0 dB, which is full scale. If your source is already loud, bring the video volume down to 0.8 or the music to 0.15 before the peaks clip. The audio multiple volumes template shows the same idea with several tracks at different levels.

The script

The script renders a ten second section of the Sintel trailer, an open movie by the Blender Foundation under CC BY 3.0, with a music track from the Shotstack sample library. It needs an API key, which you can get by signing up for a free account, set as SHOTSTACK_API_KEY. It runs against the sandbox by default. Set SHOTSTACK_ENV=v1 for production.

"""Mute a video, replace its audio, or mix music under it with the Shotstack Edit API.

Usage: SHOTSTACK_API_KEY=... python audio_modes.py <mode>     mode = mute | output-mute | replace | mix | all
Renders a 10 second clip of the Sintel trailer (Blender Foundation, CC BY 3.0) and prints the output URL.
Sandbox by default; set SHOTSTACK_ENV=v1 for production.
"""
import json
import os
import sys
import time
import urllib.parse
import urllib.request

API_KEY = os.environ.get("SHOTSTACK_API_KEY") or sys.exit("Set SHOTSTACK_API_KEY first.")
ENV = os.environ.get("SHOTSTACK_ENV", "stage")
BASE = f"https://api.shotstack.io/edit/{ENV}"
VIDEO = "https://download.blender.org/durian/trailer/sintel_trailer-720p.mp4"
MUSIC = "https://shotstack-assets.s3-ap-southeast-2.amazonaws.com/music/unminus/lit.mp3"
START_IN_SOURCE = 20  # seconds into the trailer, where the dialogue is
LENGTH = 10


def call(method, path, body=None):
    data = json.dumps(body).encode() if body else None
    req = urllib.request.Request(BASE + path, data=data, method=method, headers={
        "x-api-key": API_KEY, "Content-Type": "application/json", "Accept": "application/json"})
    with urllib.request.urlopen(req, timeout=30) as r:
        return json.load(r)["response"]


def video_clip(volume):
    return {"asset": {"type": "video", "src": VIDEO, "trim": START_IN_SOURCE, "volume": volume}, "start": 0, "length": LENGTH}


def music_clip(volume):
    return {"asset": {"type": "audio", "src": MUSIC, "volume": volume, "effect": "fadeOut"}, "start": 0, "length": LENGTH}


def edit(mode):
    tracks = {
        "mute": [[video_clip(0)]],                                   # original sound off, nothing added
        "output-mute": [[video_clip(1)]],                            # same, done at output level (see below)
        "replace": [[video_clip(0)], [music_clip(1)]],               # original off, music on its own track
        "mix": [[video_clip(1)], [music_clip(0.2)]],                 # keep the dialogue, music under it at 20 per cent
    }[mode]
    output = {"format": "mp4", "resolution": "hd"}
    if mode == "output-mute":
        output["mute"] = True
    return {"timeline": {"tracks": [{"clips": clips} for clips in tracks]}, "output": output}


def render(mode):
    render_id = call("POST", "/render", edit(mode))["id"]
    while True:
        r = call("GET", f"/render/{render_id}")
        if r["status"] == "done":
            return render_id, r["url"]
        if r["status"] == "failed":
            sys.exit(f"{mode}: render {render_id} failed: {r.get('error')}")
        time.sleep(3)


def audio_streams(url):
    """What the finished file actually contains, read back through the API's probe endpoint (ffprobe, no download)."""
    meta = call("GET", "/probe/" + urllib.parse.quote(url, safe=""))["metadata"]
    return [s for s in meta["streams"] if s["codec_type"] == "audio"]


modes = ["mute", "output-mute", "replace", "mix"] if sys.argv[1:] == ["all"] else sys.argv[1:2]
for mode in modes:
    render_id, url = render(mode)
    audio = audio_streams(url)
    summary = ", ".join(f"{s['codec_name']} {int(s.get('bit_rate', 0)) // 1000} kbps" for s in audio) or "no audio stream"
    print(f"{mode:<12} {render_id}  audio: {summary}")
    print(f"{'':<12} {url}")

Three things the script does that are worth copying into your own code:

1. One function per clip type. video_clip and music_clip take a volume and return a clip, so each mode is one line in the edit function.
2. One track per simultaneous clip. The video and the music start at the same time, so they sit on separate tracks. Two clips that overlap in time on one track do not render the way you expect.
3. Read the file back. The probe endpoint runs ffprobe on the hosted output and returns its streams, so the script can say what the file contains without downloading it. The media metadata guide covers the rest of that response.

What each mode left in the file

This is the output of python audio_modes.py all against the sandbox:

mute         390652c3-82de-4d07-aab3-669dc6defd4f  audio: aac 2 kbps
             https://shotstack-api-stage-output.s3-ap-southeast-2.amazonaws.com/qh9aqzj6v6/390652c3-82de-4d07-aab3-669dc6defd4f.mp4
output-mute  c8d108fb-87df-4f4a-b3b8-509e5d248b1f  audio: no audio stream
             https://shotstack-api-stage-output.s3-ap-southeast-2.amazonaws.com/qh9aqzj6v6/c8d108fb-87df-4f4a-b3b8-509e5d248b1f.mp4
replace      7dbc6f8a-4a23-4bcf-b663-4c555f469440  audio: aac 128 kbps
             https://shotstack-api-stage-output.s3-ap-southeast-2.amazonaws.com/qh9aqzj6v6/7dbc6f8a-4a23-4bcf-b663-4c555f469440.mp4
mix          a7bb915d-0d7f-4855-b224-098458812e14  audio: aac 128 kbps
             https://shotstack-api-stage-output.s3-ap-southeast-2.amazonaws.com/qh9aqzj6v6/a7bb915d-0d7f-4855-b224-098458812e14.mp4

We then downloaded the four files and measured their loudness with FFmpeg’s volumedetect filter:

Two findings worth knowing before you pick a mode.

• A muted clip still carries a track. The mute output has a stereo AAC stream at 2 kbps. Players show an audio track, and some platforms’ checks will see one. Only output.mute produces a file with no audio stream, and it saves 8 KB on a ten second clip, so size is not the reason to choose it.
• Mixing can hit full scale. The mix output peaked at 0.0 dB with the music at 0.2. The replace mode, music alone, peaked at -0.1 dB. If you mix over a loud source, pull one of the two volumes down.

The sandbox adds a watermark to the picture, which has no effect on the audio. The production render of the mix mode, embedded below, measured the same -13.5 dB.

One edit, the sound you want, nothing to re-encode locally

Muting, stripping, replacing and mixing are four settings on the same timeline. Set them in the edit, submit it, and the API returns a file with exactly the audio you asked for. The probe endpoint confirms it without a download.

Sign up for a free API key and run the script on your own footage. If this is your first render, the first video guide walks through the submit and poll loop.

Frequently asked questions (FAQs)

Can I fade the original sound out instead of cutting it?

Yes. The video asset has a volumeEffect field that takes fadeIn, fadeOut or fadeInFadeOut. For a custom curve, such as ducking the original under a voiceover for a few seconds, volume also accepts an array of tween objects that change the level over time.

Does the replacement audio have to be an MP3?

The API reference describes the audio asset’s src as the URL of an MP3 file, and it must be publicly accessible or include credentials. Convert other formats to MP3 first.

Can I mute only part of a clip?

Split it into two clips on the same track, the first with volume: 1 and the second with volume: 0, using trim and start so they run back to back. Or keep one clip and use a volume tween to drop the level at the point you want.

What happens if the music is shorter than the video?

The audio asset plays until its file ends or the clip length is reached, whichever comes first. A short file leaves silence for the rest of the clip. Either loop it with a second audio clip that starts where the first ends, or trim the video to the music.

Does muting change the picture or the file size?

No. All four outputs in the test were 1280 × 720 from the same edit, and the sizes differed by under 5 per cent. The audio settings change only the audio stream.