Edit Videos Using Code

Shotstack is a cloud based video editing API. Every video starts as an edit: JSON that describes how the video should be put together. You can write the edit by hand, build it in your code or ask an AI agent to write it, and Studio shows the same edit on a visual timeline.

The edit follows the principles of desktop editing software such as a timeline, tracks and clips. If you have used Adobe Premiere or After Effects, these concepts should be familiar.

These core concepts and principles are described below:

Edit

An edit encompasses everything about the video that you want to create and is made of two components - the timeline which describes the assets (video, images, text, audio and more) that you want to use and how they are arranged (edited), and the output format (mp4, gif, mp3, jpg, png or bmp) and resolution (a preset from preview to 4K, or a custom size).

{
    "timeline": {...},   // the main video edit description
    "output": {...}      // the output format and resolution
}

info

The Edit is the JSON payload that you POST to the render endpoint to create your video.

Timeline

The timeline represents the time that the video will play for, using seconds and starting from 0. A timeline is made up of tracks which are like layers that contain clips.

{
    "timeline": {
        "tracks": [{...}]   // array of tracks
    },
    ...
}

Tracks

Tracks are like layers within the timeline which span the entire length of the timeline, a track contains one or more clips Tracks can be visualized as layers that can be placed one on top of another. Clips on the top most track will obscure those on tracks below it. This can be useful for when you want to layer a title over a video clip or you want to fade between clips.

Tracks also help to logically organize different types of content, for example a title track which plays over the top of a video and a watermark track that plays over the top of everything.

Tracks are an array within the timeline, the first element is the top track and the last element is the bottom most track:

{
    "timeline": {
        "tracks": [
            {
                ...   // top track
            },
            {
                ...   // bottom track
            }
        ]
    },
    ...
}

Clips

Clips are where you select the asset (video, image, rich text and more) you want to display within a track on the timeline, when it starts, how long it plays for and what effects, transitions and filters you want to apply.

Clips are placed on a track at a specific start time, for example a clip with a start time of 3 will appear in the video after three seconds have played. Clips also have a length which determines how long the clip will play for in seconds.

You can choose to set the start or length property automatically by setting it to auto or end. See the smart clips section for all options.

Each asset type has a specific set of options and features but you must specify the type of asset and also the content of the asset such as a URL src to a video or image file, or a text string for rich text.

{
    "timeline": {
        "tracks": [
            {
                "clips": [
                   {
                       // place an image at the very start of the track/timeline that plays for 4 seconds.
                       "asset":{
                           "type": "image",
                           "src": "https://s3-ap-southeast-2.amazonaws.com/my-bucket/photo.jpg",
                       },
                       "start": 0,
                       "length": 4
                    },
                    {
                        // place a video that starts on the fourth second of the track/timeline ("start": 4) that has
                        // the first two seconds trimmed ("trim": 2) and plays for 6 seconds ("length": 6).
                       "asset":{
                           "type": "video",
                           "src": "https://s3-ap-southeast-2.amazonaws.com/my-bucket/footage.mp4",
                           "trim": 2
                       },
                       "start": 4,
                       "length": 6
                    }
                ]
            },
        ]
    },
    ...
}

danger

Do not place clips on a track so that their start or end times overlap. This will result in the clips flickering because the API does not know which clip to display.

A clip can also have transitions, effects and filters applied to them. A clip may look something like this when more options are applied:

{
    "asset":{
        "type": "video",
        "src": "https://s3-ap-southeast-2.amazonaws.com/my-bucket/footage.mp4",
        "trim": 2,
        "volume": 0.5
    },
    "start": 0,
    "length": 4,
    "transition": {
        "in": "fade",
        "out": "fade"
    },
    "filter": "boost",
    "effect": "slideRight"
}

From these basic building blocks you can assemble a sophisticated, reusable template. Replace assets and text with merge fields, or add clips with loops, conditions and other code, to build a range of video based applications.