Frame sequences: 360° orbits as timelines

Spin a product on a shop page and you are scrubbing a video that is not a video. Behind the drag handler there is usually a folder of stills, thirty-six of them if the turntable moved ten degrees at a time, and a function that maps the pointer to a number. The pointer position is an index. The picture is whichever file that index names.

That shape turns up everywhere once you look for it. Generated image sets are the same. A storyboard is the same. We added it to @elah/core in 0.6.0 as a small, React-free abstraction, and the interesting part is not the abstraction. It is how little work it takes to turn it back into an ordinary timeline.

What a frame sequence is

A FrameSequence is an ordered list of frames plus a rule for moving through them. Each Frame has an id, an index that always equals its position in the array, a base src, and optionally a list of responsive sources (a width and a MIME type each, the same information a srcset needs). The sequence carries a playback fps, a loop mode, and one flag worth explaining: seamless.

seamless is true only when the last frame joins the first without a visible jump, which is to say a genuine closed 360° orbit. It is a fact about the asset, not a preference about the interface, which is why it lives on the sequence. A camera arc that stops at 200 degrees looks broken when wrapped, because the subject snaps back. So loop defaults from it: a seamless sequence wraps, and anything else bounces.

import { createFrameSequence, frameAt, frameCount } from '@elah/core'

const orbit = createFrameSequence({
  frames: ['orbit-000.webp', 'orbit-010.webp', 'orbit-020.webp' /* ... */],
  seamless: true, // closed orbit, so loop defaults to 'wrap'
  label: 'Chair, 360 degrees',
})

frameCount(orbit)  // 3 here
frameAt(orbit, -1) // wraps to the last frame

Frames can be bare strings, as above, or partial frames with their own id and sources; createFrameSequence fills in the ids and indices. normalizeFrameIndex(sequence, index) is the single place that decides what an out-of-range index means. Under none it clamps. Under wrap it takes a true modulo, which matters because the JavaScript remainder operator returns negatives for negative input. Under pingpong it reflects at each end over a period of twice the frame count minus two, so the end frames are each visited once and there is no stutter at the turn.

The controller

FrameSequenceController is what you attach a viewer to. It owns the current index, a clock, and drag-to-scrub, and it is deliberately per-instance: nothing in it is module-scoped, so a page can run several sequences independently. That is the property that lets several orbit viewers coexist on one page where only one EditorProvider can.

The clock is not new code. The controller owns a PlaybackEngine, the same anchor-and-integrate clock the editor's transport uses, so autoplay inherits the integer-frame notification guard and the tab-visibility handling for free, and behaves the way the timeline does. The clock counts linearly and its output is mapped through normalizeFrameIndex, which is how wrap and pingpong both ride one counter without the controller knowing the difference.

Drag is where the design has an opinion. The distance the pointer must travel per frame is derived from the viewer's width, and dragSensitivity is how many sequence-lengths one full-width drag covers, defaulting to 1, so by default one full-width drag sweeps through the whole sequence once and the gesture feels the same on a phone and on a wide monitor. A fixed pixels-per-frame rule makes a wide viewer sluggish and a narrow one twitchy. The fractional position is accumulated as a float during the drag and rounded only when emitting, so a slow drag still advances instead of rounding to zero on every move.

import { FrameSequenceController } from '@elah/core'

const controller = new FrameSequenceController({ sequence: orbit })

controller.setViewportWidth(viewer.clientWidth)
viewer.addEventListener('pointerdown', (e) => controller.beginDrag(e.clientX))
window.addEventListener('pointermove', (e) => controller.drag(e.clientX))
window.addEventListener('pointerup', () => controller.endDrag())

controller.subscribe(({ index }) => showFrame(index))

The snapshot the controller exposes is reference-stable between changes, which is what lets React bind it with useSyncExternalStore without an adapter. And because a React effect cleanup is not proof of unmount, there is a reversible detach() and reattach() pair alongside the final destroy(). A StrictMode remount runs setup, cleanup, setup against one controller; tearing the clock down irreversibly there left a transport that claimed to be playing while the index never moved.

Preloading without starving the frame you are looking at

Thirty-six frames is thirty-six network requests. Fire them all at once and the frame the user is looking at competes with the one on the far side of the orbit. Load them lazily and every drag lands on an undecoded image. createFramePreloader(sequence, options) loads outward from the focus: the current frame first, then one either side, then two, under a concurrency cap. You call focus(index) as the index changes and it re-prioritises. Under wrap it measures distance the short way round, so focusing frame 0 of a closed orbit treats the last frame as its neighbour.

What it warms is the subtle part. It does not warm frame.src. It warms whatever pickFrameSource(frame, sizeHint) resolves, which is the URL the browser will actually paint: the first source whose MIME type the browser supports, then the narrowest candidate that still meets the target width, falling back to frame.src when nothing applies. supportsImageType(type) probes AVIF and WebP support once and memoizes the answer for the session. Warming a different URL than the one that will be painted downloads bytes nobody sees and delays the ones somebody needs, and that was the bug this module exists to close.

Decoding goes through core's existing image cache rather than a second one, so a frame preloaded here is the same decoded object the renderer picks up if the sequence is later dropped on a timeline.

From a sequence to a project

Here is the bridge. frameSequenceToProject(sequence, options) is a pure function that builds a Project: one video track, and one image clip per frame laid end to end. Nothing about the result is special. You hand it to engine.loadProject() and it is a composition like any other.

import { frameSequenceToProject } from '@elah/core'

// One timeline frame per sequence frame: timeline time and index are the same number.
engine.loadProject(frameSequenceToProject(orbit))

// Hold each angle for 6 timeline frames instead, for a slower, slideshow-like pass.
engine.loadProject(frameSequenceToProject(orbit, { holdFrames: 6, fps: 30 }))

The defaults are chosen to keep that identity. holdFrames is 1, so index and timeline frame coincide, and the stage is 1920 by 1080, landscape on purpose, because orbits and storyboards usually are and it matches the CLI's spec default. The moment you want to trim the orbit, caption it, reorder it, put a transition between two angles or export it as an MP4, you are not doing anything special. It is a project. The editor did not need to learn what a sequence is.

Why not just a video

You could render the orbit to a video and seek it. Plenty of viewers do. We did not, for reasons we can defend from how the pieces work.

A sequence is addressed by index. Frame 17 is the seventeenth image, full stop, with no keyframe to find and no decoder to warm; the cost of asking for a frame does not depend on which frame it is or where you were a moment ago. A video is a different structure: compressed frames depend on earlier ones, and arriving at an arbitrary frame means starting from the nearest keyframe before it. For a drag that jumps backwards and forwards at the pointer's whim, that is the wrong access pattern.

And every frame in a sequence is a still image in its own right, encoded on its own. Nothing in frame 17 was produced by compressing it against frame 16, so there is no inter-frame compression to smear detail while you are scrubbing. We are not claiming that stills are better for every use. A sequence is a poor fit for continuous motion with sound, which is what a video is for. For a product on a turntable, where the only thing that moves is the viewer's hand, indexed stills are the honest representation of what the data is.

If the thing you are scrubbing is really a list of images, model it as a list of images. Then, when you want a video, you can have one.