<SYSTEM>This is the full developer documentation for framefields</SYSTEM>

# framefields docs

> What framefields is, a complete first composition, and where to go next.

framefields is a TypeScript library that renders video on the GPU, frame by frame, straight to MP4. A video is a `Composition`: a tree of layers with keyframed animation. It renders natively on WebGPU (Dawn → Metal / Vulkan) in Node.js — no browser, no Puppeteer, no screenshots.

It is built to be driven by coding agents. Add the skills to your agent:

```bash
npx skills add gatewai-dev/framefields
```

Works with Claude, Codex, Cursor, Hermes, Gemini CLI, GitHub Copilot, and 70+ others. More in [framefields for agents](/docs/agents/). Or write it yourself — this is a complete film:

src/film.ts

```ts
import { Composition, FontManager, Layer, LayerAnimation } from "framefields";


export async function buildFilm() {
  await FontManager.register({ family: "Inter", source: "assets/fonts/Inter.ttf" });


  const film = new Composition({
    width: 1920,
    height: 1080,
    fps: 30,
    durationMs: 3000,
    backgroundColor: "#0e0d0c",
  });


  film.add(
    Layer.text("Hello, framefields", {
      position: "absolute",
      x: 0, y: 480, width: 1920, height: 120,
      fontFamily: "Inter", fontSize: 96, fontWeight: 700,
      fill: "#eee9e0", align: "center",
    }).animate(
      LayerAnimation.create()
        .fadeIn(0, 20, "power2.out")
        .fromTo("y", 520, 480, { start: 0, end: 30, ease: "expo.out" }),
    ),
  );


  return film;
}


// elsewhere: await (await buildFilm()).renderVideo({ outputPath: "output/hello.mp4" });
```

## What it covers

[Section titled “What it covers”](#what-it-covers)

| Area          | You get                                                                                                             |
| ------------- | ------------------------------------------------------------------------------------------------------------------- |
| Layout & type | Flexbox (Yoga) boxes, GPU vector text (Slug), rich text, RTL, text animators                                        |
| Motion        | Keyframes, 13 easing families, springs, wiggles, signals, scene transitions                                         |
| Effects       | 54 GPU effects: grading, grain, blur, LUTs, keying, glass, trails, datamosh, relighting, audio DSP                  |
| 3D            | Cameras, lights, PBR environment, refracting glass/acrylic, shadows, bloom, primitives, glTF/OBJ/FBX/STL/PLY models |
| Physics       | Deterministic 2D & 3D rigid bodies (Rapier), live worlds, performer deflection, GPU compute particles               |
| Audio         | Soundtracks, procedural SFX, audio-reactive signals                                                                 |
| Data          | Animated line, area, bar, scatter, candlestick, pie and donut charts                                                |
| Vision        | On-device detection, segmentation, pose and mattes driving layout and effects                                       |
| Delivery      | PNG frames, contact-sheet grids, MP4 with built-in QA, live browser preview                                         |

## Examples & showcase

[Section titled “Examples & showcase”](#examples--showcase)

Explore open-source reference films in the [GitHub repository](https://github.com/gatewai-dev/framefields/tree/main/examples):

* [**Physics dancer**](https://youtu.be/POC7rKnuoE4) (`examples/28_physics_dancer`) — 3D glass word cards, live Rapier rigid-body collisions against a tracked dancer’s limbs, and silhouette-deflected GPU embers.
* [**Mac Keyboard**](https://youtu.be/k2lgbdL6_TA) (`examples/26_mac_keyboard_from_3d_primitives`) — Mechanical keyboard built entirely from 3D primitives, PBR materials, and camera rack focus.
* [**Framefields launch**](https://youtu.be/FVXTqAozAv8) (`examples/22_framefields_launch`) — Lyric-driven launch film with tight BeatGrid audio synchronization.
* [**Full Circle**](https://youtu.be/9BF1snfmg3o) (`examples/21_full_circle`) — Kinetic typography, 3D scenes, and camera trajectories.
* [**Dancer showcase**](https://youtu.be/RQ3yi_lzPqI) (`examples/19_framefields_film`) — Multi-scene brand film with on-device vision mattes, audio reactive signals, and LUT grading.

## Start here

[Section titled “Start here”](#start-here)

[Getting started](/docs/getting-started/)Install, scaffold, and render your first MP4.

[framefields for agents](/docs/agents/)Install the skills, the agent workflow, example prompts.

[Core concepts](/docs/core-concepts/)Compositions, frames, layer trees, timing and layout.

[Authoring rules](/docs/reference/rules/)The short list of rules that prevent most broken renders.

## For agents

[Section titled “For agents”](#for-agents)

Machine-readable docs

See [Docs for agents](/docs/agents/llms/). Every page has a Markdown twin: swap the trailing `/` for `.md` (`/docs/guides/animation/` → `/docs/guides/animation.md`; this page is [`/docs/index.md`](/docs/index.md)). The whole site is in [`/docs/llms-full.txt`](/docs/llms-full.txt), a compact version in [`/docs/llms-small.txt`](/docs/llms-small.txt), and an index in [`/docs/llms.txt`](/docs/llms.txt).

Requirements: **Node.js ≥ 22**, an ES module project, and a **WebGPU-capable GPU** (Metal on macOS, Vulkan on Linux/Windows). Without a GPU, Linux falls back to a much slower CPU driver. framefields is Apache-2.0 and in beta (v2.2).

# Getting started

> Create a framefields project with one command (or by hand), and verify a first render from the cheapest check to the final MP4.

## The quick way: `framefields init`

[Section titled “The quick way: framefields init”](#the-quick-way-framefields-init)

One command sets up everything this page describes: `package.json` (with the install-script approvals below, scripts, dependencies and `packageManager`), `tsconfig.json`, `src/theme.ts`, `film.ts` and `render.ts`, `assets/` with Git LFS rules, `.gitignore`, and the Inter and JetBrains Mono fonts.

```bash
pnpm dlx framefields init my-film --install   # or: npx framefields init my-film --install
cd my-film && pnpm frames -- 0 30 60          # first frames in output/
```

* **Nothing to answer.** `init` asks no questions, so it runs the same in CI and on remote servers.
* **Existing projects are safe.** Run it inside one to add framefields there. Files you have are left alone, and `package.json`, `.gitignore` and `.gitattributes` only gain what they lack. A `"type": "commonjs"` package is refused, because framefields needs ES modules.
* **Fonts are pinned.** Inter and JetBrains Mono are downloaded from Google Fonts, checked by SHA-256 and saved in `assets/fonts/` with their OFL licenses. The generated `src/components/fonts.ts` registers them, and `theme.ts` calls it. The npm package ships no fonts.
* **The AI gateway is optional.** To generate images, video, music, voice and 3D from code, create a key at [framefields.dev/dashboard](https://framefields.dev/dashboard), copy `.env.example` to `.env` and set `FRAMEFIELDS_API_KEY`. `render.ts` loads `.env` when it exists. See [Generative AI](/docs/guides/ai/).

| Option                      | Effect                                                        |
| --------------------------- | ------------------------------------------------------------- |
| `--install`                 | Install dependencies afterwards                               |
| `--pm npm\|pnpm\|yarn\|bun` | Package manager (default: the one running the command)        |
| `--emoji`                   | Also install Noto Color Emoji, so emoji in text draw in color |
| `--no-fonts`                | Skip Inter and JetBrains Mono                                 |
| `--no-git`                  | Don’t run `git init`                                          |
| `--name <name>`             | Package name (default: the folder’s name)                     |

Then jump to [4. Verify](#4-verify-cheapest-step-first). The steps below set up the same project by hand, for when you want to see each piece or merge into an unusual setup.

### More from the CLI

[Section titled “More from the CLI”](#more-from-the-cli)

```bash
npx framefields list              # ready-made components, one line each
npx framefields add chat-thread   # copies a component's source into src/components/
npx framefields fonts add emoji   # fonts on their own: emoji, sans (Inter), mono (JetBrains Mono)
```

Components that need a font bring it along: `chat-thread` and `captions` install the emoji font, `terminal` and `code-block` JetBrains Mono. Text with emoji but no emoji font draws nothing, so `comp.lint()` reports `missing_emoji_font` and the first render prints how to fix it.

## 1. Check the machine

[Section titled “1. Check the machine”](#1-check-the-machine)

```bash
node --version   # must be v22 or later
```

* **A GPU, or a CPU renderer.** Rendering uses WebGPU via Dawn: Metal on macOS (every Mac has a Metal GPU), Vulkan on Linux, Direct3D 12 or Vulkan on Windows. Without a GPU it renders on the CPU, about 12× slower: on Linux install Mesa lavapipe (`apt-get install -y libvulkan1 mesa-vulkan-drivers`); Windows has WARP built in. See [Rendering without a GPU](/docs/guides/rendering/#rendering-without-a-gpu).
* **ES modules.** `package.json` needs `"type": "module"`.

## 2. Write `package.json`, then install

[Section titled “2. Write package.json, then install”](#2-write-packagejson-then-install)

framefields needs exactly one dependency install script: `skia-canvas` downloads its native binary. npm 11, pnpm 10 and bun block install scripts unless approved, so approve that one and deny the rest **before** installing.

package.json

```json
{
  "name": "my-film",
  "private": true,
  "type": "module",
  "allowScripts": {
    "skia-canvas": true,
    "sharp": false,
    "webgpu": false,
    "onnxruntime-node": false,
    "node-av": false,
    "esbuild": false,
    "protobufjs": false
  },
  "pnpm": {
    "onlyBuiltDependencies": ["skia-canvas"],
    "ignoredBuiltDependencies": ["sharp", "webgpu", "onnxruntime-node", "node-av", "esbuild", "protobufjs"]
  },
  "trustedDependencies": ["skia-canvas"],
  "scripts": {
    "typecheck": "tsc --noEmit -p .",
    "frames": "tsx src/render.ts frames",
    "grid": "tsx src/render.ts grid",
    "preview": "tsx src/render.ts preview",
    "render": "tsx src/render.ts"
  }
}
```

`allowScripts` is read by npm, `pnpm` by pnpm, `trustedDependencies` by bun. Yarn needs none of them. Deny the rest: they fetch an ffmpeg CLI and CUDA libraries framefields doesn’t use, or do nothing it needs.

Add `"packageManager"` with the manager and version you install with (e.g. `"pnpm@10.13.1"`; `pnpm --version` prints it), and commit the lockfile it writes. A [render worker](/docs/guides/render-service/) installs with exactly that version and lockfile.

```bash
npm install framefields
npm install --save-dev tsx typescript @types/node
```

Caution

If you see `Cannot find module '.../skia.node'`, the `skia-canvas` script was skipped. Approve it as above, then run `npm rebuild skia-canvas` (or `pnpm rebuild skia-canvas`).

tsconfig.json

```json
{
  "compilerOptions": {
    "module": "NodeNext",
    "moduleResolution": "NodeNext",
    "target": "ES2022",
    "strict": true,
    "skipLibCheck": true,
    "noEmit": true,
    "types": ["node"]
  },
  "include": ["src/**/*"]
}
```

## 3. Scaffold

[Section titled “3. Scaffold”](#3-scaffold)

Keep the composition separate from the script that renders it, so tests, preview and export all build the same film.

```plaintext
src/
  theme.ts    sizes, fps, colors, fonts — the only place for magic numbers
  film.ts     buildFilm(): returns the Composition
  render.ts   frames | grid | preview | video
assets/       every input file, stored with Git LFS
  fonts/      .ttf / .otf (text needs a real font file)
  footage/    .mp4 / .mov / .webm
  audio/      .mp3 / .wav
  images/     .png / .jpg / .webp
output/       renders (gitignored, never committed)
.gitattributes  LFS rules for assets/
```

src/theme.ts

```ts
import path from "node:path";
import { fileURLToPath } from "node:url";
import { FontManager } from "framefields";


const HERE = path.dirname(fileURLToPath(import.meta.url));


export const W = 1920;
export const H = 1080;
export const FPS = 30;
export const INK = "#0e0d0c";
export const PAPER = "#eee9e0";
export const SANS = "Inter";


export async function registerFonts(): Promise<void> {
  // Resolve from the module URL so the browser preview finds the file too.
  await FontManager.register({ family: SANS, source: path.resolve(HERE, "../assets/fonts/Inter.ttf") });
}
```

src/film.ts

```ts
import { Composition, Layer, LayerAnimation } from "framefields";
import { FPS, H, INK, PAPER, registerFonts, SANS, W } from "./theme.js";


const DURATION = 3 * FPS; // frames


export async function buildFilm(): Promise<Composition> {
  await registerFonts(); // before any text layer is built


  const film = new Composition({
    width: W,
    height: H,
    fps: FPS,
    durationMs: (DURATION / FPS) * 1000,
    backgroundColor: INK,
  });


  film.add(
    Layer.text("Hello, framefields", {
      id: "title",
      position: "absolute",
      x: 0, y: H / 2 - 60, width: W, height: 120,
      fontFamily: SANS, fontSize: 96, fontWeight: 700,
      fill: PAPER, align: "center",
    }).animate(
      LayerAnimation.create()
        .fadeIn(0, 20, "power2.out")
        .fromTo("y", H / 2 - 20, H / 2 - 60, { start: 0, end: 30, ease: "expo.out" }),
    ),
  );


  return film;
}
```

src/render.ts

```ts
import fs from "node:fs/promises";
import path from "node:path";
import { HeadlessMediaRenderer, startPreview } from "framefields";
import { buildFilm } from "./film.js";


const OUT = path.resolve(import.meta.dirname, "../output");
const [mode = "video", ...args] = process.argv.slice(2);


if (mode === "preview") {
  const session = await startPreview({ entry: new URL("./film.ts", import.meta.url), export: "buildFilm" });
  console.log(`Preview at ${session.url}`);
  await session.closed;
  process.exit(0);
}


const film = await buildFilm();
await fs.mkdir(OUT, { recursive: true });


if (mode === "frames") {
  const renderer = new HeadlessMediaRenderer();
  for (const frame of args.map(Number)) {
    const png = await film.renderFrame({ frame, renderer });
    await fs.writeFile(path.join(OUT, `f${String(frame).padStart(4, "0")}.png`), png);
  }
} else if (mode === "grid") {
  const png = await film.renderFrameGrid({
    frames: args.length ? args.map(Number) : [0, 10, 20, 30, 45, 60, 75, 89],
    columns: 4,
    cellWidth: 480,
  });
  await fs.writeFile(path.join(OUT, "grid.png"), png);
} else {
  const result = await film.renderVideo({ outputPath: path.join(OUT, "output.mp4"), quality: "high", qa: true });
  console.log(`Exported ${result.filePath}`);
}


// Native GPU and encoder threads keep Node alive: always exit explicitly.
process.exit(0);
```

## 4. Verify, cheapest step first

[Section titled “4. Verify, cheapest step first”](#4-verify-cheapest-step-first)

Each step catches in seconds what a full render takes minutes to show.

| Step | Command                     | Look for                                                                                                  |
| ---- | --------------------------- | --------------------------------------------------------------------------------------------------------- |
| 1    | `npm run typecheck`         | type errors                                                                                               |
| 2    | `npm run frames -- 0 30 60` | PNGs in `output/`; a black frame usually means a missing font, an off-canvas layer, or opacity stuck at 0 |
| 3    | `npm run grid`              | smooth motion across the contact sheet                                                                    |
| 4    | `npm run preview`           | a person watches it with sound ([Live preview](/docs/guides/preview/))                                    |
| 5    | `npm run render`            | the MP4, plus QA findings ([Rendering](/docs/guides/rendering/))                                          |

## 5. Store assets with Git LFS

[Section titled “5. Store assets with Git LFS”](#5-store-assets-with-git-lfs)

Keep every input file under `assets/` and store it with [Git LFS](https://git-lfs.com). Set it up before adding any media:

```bash
git lfs install
git lfs track "assets/fonts/**" "assets/footage/**" "assets/audio/**" "assets/images/**" \
  "assets/luts/**" "assets/models/**" "assets/lottie/**"
echo "output/" >> .gitignore
git add .gitattributes .gitignore && git commit -m "Track assets with Git LFS"
```

**Push:** `git add assets/…`, commit and `git push` as usual; the media upload with the push. Run `git lfs ls-files` before committing to confirm each new file is listed.

**Pull:** in a fresh clone, container or CI runner, run `git lfs pull` before rendering (in GitHub Actions, `actions/checkout` with `lfs: true`).

Caution

Until pulled, LFS files are \~130-byte pointers. framefields cannot decode them: fonts fail to register and media renders blank. Ignore `output/`, not `*.mp4`, or your footage disappears from Git too.

Details, including fixing a file committed without LFS: [Assets and Git LFS](/docs/agents/workflow/#assets-and-git-lfs).

## Next

[Section titled “Next”](#next)

* [Core concepts](/docs/core-concepts/) — how frames, layers and timing fit together.
* [framefields for agents](/docs/agents/) — install the skills so your agent knows all of this.
* [Render on a server](/docs/guides/render-service/) — render the project from git on a GPU worker.

# framefields for agents

> Why framefields is built for coding agents, how to give your agent the skills in one command, and the loop it follows to ship a video.

framefields is made to be driven by a coding agent. A film is a TypeScript file, every check is a function that returns data, and every review step produces a file the agent can open and look at. You describe the video; the agent writes, renders, inspects and fixes it.

## Install the skills

[Section titled “Install the skills”](#install-the-skills)

Add the skills to your agent:

```bash
npx skills add gatewai-dev/framefields
```

Works with Claude, Codex, Cursor, Hermes, Gemini CLI, GitHub Copilot, Windsurf, OpenCode, Goose, Cline and 70+ others.

Then ask for a video:

```text
Set up framefields in this project and render a 3-second title card that says "Hello".
```

The skills are instructions only: no MCP servers, hooks or executables, and they send no data anywhere. Details per agent are on [Install the skills](/docs/agents/install/).

## Why it suits agents

[Section titled “Why it suits agents”](#why-it-suits-agents)

|                        |                                                                                                                 |
| ---------------------- | --------------------------------------------------------------------------------------------------------------- |
| **Plain code**         | A video is a `Composition` in TypeScript. Diffs, reviews, tests and git all work.                               |
| **Deterministic**      | `(composition, frame) → pixels`. The same frame renders the same way every time, so checks are repeatable.      |
| **Checks return data** | `comp.lint()` and `renderVideo({ qa })` return `{ severity, code, message }` objects, not logs to parse.        |
| **Cheap feedback**     | One PNG frame or a contact-sheet grid in seconds, long before a full render.                                    |
| **People in the loop** | The live preview saves notes a person pins on the picture to `notes.json` — exact feedback an agent can act on. |
| **No creative suite**  | One npm package; no browser, no desktop apps, no licenses.                                                      |

## The loop

[Section titled “The loop”](#the-loop)

1. **Write** `film.ts` exporting `buildFilm()`.
2. **Check** — `tsc --noEmit`, then `comp.lint()`.
3. **Look** — render key frames to PNG and open them.
4. **Look at motion** — render a frame grid.
5. **Show** — start the live preview; read the person’s pinned notes.
6. **Ship** — `renderVideo({ qa: true })`; stop if `qa.passed` is false.

Full detail in [The agent workflow](/docs/agents/workflow/).

[Install the skills](/docs/agents/install/)Claude Code, Codex, Cursor, Gemini CLI, Copilot and more; project or global; team setup.

[The skills](/docs/agents/skills/)What each of the four skills teaches and when it loads.

[The agent workflow](/docs/agents/workflow/)Write → check → look → show → ship, with code.

[Example prompts](/docs/agents/prompts/)Prompts that work well, from first render to vision effects.

[Docs for agents](/docs/agents/llms/)llms.txt, Markdown pages, and an AGENTS.md snippet.

# Core concepts

> Compositions, the frame clock, layer trees, local timing, layout and coordinates — the model everything else builds on.

## Every frame is a function of its number

[Section titled “Every frame is a function of its number”](#every-frame-is-a-function-of-its-number)

A composition compiles to a deterministic program: `(composition, frame) → pixels`. There are no timers and no wall clock. Rendering frame 45 always produces the same image, in any order, on any run. This is what makes frames testable and renders parallel.

## Composition

[Section titled “Composition”](#composition)

```ts
import { Composition } from "framefields";


const comp = new Composition({
  width: 1920,
  height: 1080,
  fps: 30,               // default 60
  durationMs: 10_000,    // or durationFrames; omitted → computed from media
  backgroundColor: "#0e0d0c", // default "#000000"
  antialias3d: true,     // 4x MSAA for 3D meshes; false saves GPU memory
});
```

| Method                                            | Does                                                               |
| ------------------------------------------------- | ------------------------------------------------------------------ |
| `add(layer \| layer[])`                           | adds layers (or arrays, e.g. from `Lighting.studio()`) to the root |
| `addAudio(Layer.audio(...))`                      | adds a soundtrack                                                  |
| `apply(effect)`                                   | applies an effect to the whole frame                               |
| `addSignal(name, signal)`                         | registers a named signal layers can bind by name                   |
| `withVision(options)`                             | runs vision on the composition, returns reactive signals           |
| `vision(source, options)`                         | on-demand clip vision for live physics and particles               |
| `lint()`                                          | document checks, no GPU                                            |
| `toSpec()`                                        | the plain-JSON program (validate with `CompositorProgramSchema`)   |
| `renderFrame` / `renderFrameGrid` / `renderVideo` | output, see [Rendering](/docs/guides/rendering/)                   |
| `computeDuration()`                               | duration in ms, from options or media                              |

## Layers

[Section titled “Layers”](#layers)

Every `Layer.*` builder (boxes, shapes, text, media, 3D primitives, and `Layer.particles`) returns a node you can chain: `.animate()`, `.withEffect()`, `.withMatte()`, `.withVision()`, `.pinToObject()`. Layers nest through `children`.

```ts
Layer.box({
  width: 720, height: 380, padding: 32,
  background: "#141721", borderRadius: 24,
  borderColor: "#262b3d", borderWidth: 1.5,
  children: [
    Layer.flex({
      dir: "column", gap: 16,
      children: [
        Layer.text("ENGINE", { fontSize: 16, fontWeight: 700, fill: "#6366f1", letterSpacing: 2 }),
        Layer.text("WebGPU motion", { fontSize: 48, fontWeight: 700, fill: "#f8fafc" }),
      ],
    }),
  ],
});
```

## Time

[Section titled “Time”](#time)

* `startFrame` and `durationFrames` place a layer on its parent’s timeline.
* **Keyframes are local**: frame 0 is the layer’s first visible frame. Never add the parent’s `startFrame` to keyframe times.
* **Frames are integers.** Keyframe frames, start frames and durations must be whole numbers; `.keys()` rounds for you, other APIs reject decimals.
* Keyframes past a layer’s `durationFrames` never play (lint/QA: `animation_truncated`).
* Signals are sampled at the **composition** frame, not the layer’s local frame.

A beat grid keeps timing musical and readable:

```ts
export const FPS = 30;
export const BPM = 100;
export const BEAT = Math.round((FPS * 60) / BPM); // 18 frames
export const beats = (n: number) => Math.round(n * BEAT);
```

## Layout and position

[Section titled “Layout and position”](#layout-and-position)

* Containers lay children out with flexbox (Yoga): `Layer.flex({ dir, gap, justify, align, wrap, padding })`.
* Setting `x` or `y` without `position: "relative"` makes a layer **absolute** — removed from flex flow, positioned in its parent.
* **Animating `x`/`y` overrides layout.** A layer placed at `y: 400` animates with `fromTo("y", 440, 400)`, not `fromTo("y", 40, 0)`.
* Borders are `borderColor` + `borderWidth`. A CSS shorthand like `border: "1px solid red"` is ignored.

## Coordinates

[Section titled “Coordinates”](#coordinates)

* 2D: pixels, origin top-left, y down.
* 3D: the camera is calibrated so `z = 0` maps 1:1 onto canvas pixels. y is still down, so “up” is −y. See [3D](/docs/guides/3d/).

## Colors

[Section titled “Colors”](#colors)

Every color prop takes hex, `rgb()`, `hsl()`, `oklch()`, `oklab()` or a CSS name. Color keyframes blend in OKLab, so red → green passes through yellow. Gradients are CSS strings (`linear-`, `radial-`, `conic-gradient()`) usable as any `background` or shape `fill`.

## Project shape

[Section titled “Project shape”](#project-shape)

```plaintext
src/theme.ts     tokens: size, fps, colors, fonts, beat grid
src/scenes/*.ts  one function per scene, each returns a full-frame box
src/film.ts      buildFilm(): fonts, Composition, Layer.sequence(scenes), grade
src/render.ts    frames | grid | preview | video
assets/          fonts, footage, audio, images, LUTs, models — in Git LFS
output/          renders, gitignored
```

Media in `assets/` is stored with Git LFS: push it with a normal `git push`, and run `git lfs pull` in any fresh checkout before rendering. See [Assets and Git LFS](/docs/agents/workflow/#assets-and-git-lfs).

# Authoring rules

> The numbered rules that prevent most broken framefields renders. Read before writing a composition.

1. **Register fonts first.** Call `FontManager.register({ family, source })` for every family before building any text layer. There are no system fonts. Emoji need `FontManager.registerEmojiFont(...)` with a CBDT/CBLC color font.
2. **Keyframes are local.** Frame 0 is the layer’s first frame. Never add the parent’s `startFrame`.
3. **Frames are integers.** Keyframe frames, `startFrame`, `durationFrames` — whole numbers only.
4. **Animating `x`/`y` overrides layout.** Animate to the placed value: placed at `y: 400` → `fromTo("y", 440, 400)`.
5. **`x`/`y` make a layer absolute** unless `position: "relative"` is set.
6. **Borders are `borderColor` + `borderWidth`.** CSS `border` strings are ignored.
7. **Give text a width** when it can wrap; otherwise it may clip.
8. **Every `Layer.audio()` needs `durationFrames`.**
9. **Images are raster.** `.png/.jpg/.webp` → `Layer.image`; `.json/.lottie` → `Layer.lottie`.
10. **Effect strengths are often 0–100**, not 0–1 (`Vignette`, `FilmGrain`, `Blur`). Check the [Effects reference](/docs/reference/effects/).
11. **Grade, then optics, then grain.** `FilmGrain` is the outermost pass.
12. **PBR needs an environment.** `roughness`/`metallic` do nothing without `Layer.environment()` or a `Lighting` rig.
13. **One shadow light**, directional or spot.
14. **Signals sample the composition frame**, not the layer’s local frame.
15. **Vision trails by one frame.** Verify vision with the MP4 or consecutive frames, never with a frame grid.
16. **Resolve assets from `import.meta.url`** so the browser preview can load them.
17. **Assets live in `assets/`, in Git LFS.** `git lfs pull` before rendering in a fresh clone; confirm new media appear in `git lfs ls-files` before committing; never commit `output/`.
18. **Verify cheapest first:** typecheck → `lint()` → `renderFrame` → `renderFrameGrid` → `renderVideo({ qa: true })`. Look at every image you render.
19. **End render scripts with `process.exit()`.**
20. **Tokens live in `theme.ts`.** No magic colors, sizes or timings in scenes.
21. **Ease with intent.** `back.out(1.4–1.7)` entrances, `expo.out`/`spring` decelerations, `power2.in` exits, `linear` only for loops and counters.
22. **Custom shaders output premultiplied alpha** and blend with `srcFactor: "one"`, never `"src-alpha"`.

# Install the skills

> Install the framefields agent skills in Claude Code, Codex, Cursor, Gemini CLI, GitHub Copilot and 70+ other agents — per project, per user, or for a whole team.

The skills live in [`plugins/framefields`](https://github.com/gatewai-dev/framefields/tree/main/plugins/framefields) in the framefields repo. They are Markdown instructions only. The engine itself is the `framefields` npm package, which the skills tell your agent to install into your project.

## Install

[Section titled “Install”](#install)

* Claude Code

  Inside an interactive `claude` session:

  ```text
  /plugin install framefields
  ```

  Or from a shell:

  ```bash
  claude plugin install framefields@claude-plugins-official
  ```

  It comes from Anthropic’s official plugin directory, which Claude Code adds for you, and updates automatically. Restart Claude Code or run `/reload-plugins` afterwards. In the desktop app’s Code tab, use the shell form or **+ → Plugins → Add plugin → Framefields**.

  **Track `main` instead of the directory release:**

  ```text
  /plugin marketplace add gatewai-dev/framefields
  /plugin install framefields@framefields-plugins
  ```

* Codex

  ```bash
  npx skills add gatewai-dev/framefields -a codex
  ```

* Cursor

  ```bash
  npx skills add gatewai-dev/framefields -a cursor
  ```

* Gemini CLI

  ```bash
  npx skills add gatewai-dev/framefields -a gemini-cli
  ```

* Copilot

  ```bash
  npx skills add gatewai-dev/framefields -a github-copilot
  ```

  VS Code / Copilot can also load the portable [`plugin.json`](https://github.com/gatewai-dev/framefields/blob/main/plugins/framefields/plugin.json) through its plugin UI.

* Other

  ```bash
  npx skills add gatewai-dev/framefields
  ```

  The [`skills`](https://skills.sh) CLI detects the agents on your machine and asks where to install. Agent ids include `windsurf`, `opencode`, `goose`, `cline`, `roo`, `amp`, `kiro-cli` and `hermes-agent`.

## Choose the scope

[Section titled “Choose the scope”](#choose-the-scope)

| Goal                                                | Command                                                     |
| --------------------------------------------------- | ----------------------------------------------------------- |
| This project only (default)                         | `npx skills add gatewai-dev/framefields`                    |
| Every project, for your user                        | `npx skills add gatewai-dev/framefields -g`                 |
| Several agents at once                              | `npx skills add gatewai-dev/framefields -a codex -a cursor` |
| No prompts (CI, scripts, agents installing for you) | add `-y`                                                    |
| One skill only                                      | add `-s framefields-render`                                 |
| See what’s in the package first                     | `npx skills add gatewai-dev/framefields --list`             |
| Copy instead of symlink                             | add `--copy`                                                |

## Set it up for a team

[Section titled “Set it up for a team”](#set-it-up-for-a-team)

**Claude Code.** Commit this to `.claude/settings.json`; Claude Code offers the plugin to teammates when they trust the folder:

.claude/settings.json

```json
{
  "enabledPlugins": {
    "framefields@claude-plugins-official": true
  }
}
```

Or install with `claude plugin install framefields@claude-plugins-official --scope project`, which writes the same entry.

**Other agents.** Install per project (no `-g`) and commit the skill folders the CLI creates (for example `.agents/skills/`), so everyone who clones the repo gets them.

## Keep them current

[Section titled “Keep them current”](#keep-them-current)

```bash
npx skills update        # update installed skills
npx skills ls            # list what's installed
npx skills remove        # remove them
```

Claude Code updates plugins from the official directory automatically. The skills’ version follows the `framefields` npm package.

## Install by hand

[Section titled “Install by hand”](#install-by-hand)

Copy each folder from [`plugins/framefields/skills/`](https://github.com/gatewai-dev/framefields/tree/main/plugins/framefields/skills) into one of:

* `.claude/skills/` (Claude Code, project)
* `.agents/skills/` (project, most other agents)
* `~/.agents/skills/` (your user)

## Check it worked

[Section titled “Check it worked”](#check-it-worked)

Ask your agent: *“Which framefields skills do you have?”* It should list `framefields`, `framefields-compose`, `framefields-effects` and `framefields-render`. Skills load on their own when a task matches, so you never need to name them.

## Your project still needs the package

[Section titled “Your project still needs the package”](#your-project-still-needs-the-package)

The skills don’t install the engine. On the first task, the `framefields` skill has your agent add it to your project — Node.js 22+, an ES module `package.json`, and approval for the one install script it needs. You can do it yourself with [Getting started](/docs/getting-started/).

# Docs for agents

> Machine-readable versions of these docs (llms.txt, full and compact text, per-page Markdown) and an AGENTS.md snippet for your project.

These docs are written to be read by agents as much as people. Every page is short, example-first, and available as plain text.

## URLs

[Section titled “URLs”](#urls)

| URL                                            | Contents                                                                                                              |
| ---------------------------------------------- | --------------------------------------------------------------------------------------------------------------------- |
| [`/llms.txt`](/llms.txt)                       | site-level pointer for agents                                                                                         |
| [`/docs/llms.txt`](/docs/llms.txt)             | index: summary, requirements, and every page with its Markdown link                                                   |
| [`/docs/llms-full.txt`](/docs/llms-full.txt)   | all pages in one file                                                                                                 |
| [`/docs/llms-small.txt`](/docs/llms-small.txt) | all pages, compact                                                                                                    |
| `/docs/<page>.md`                              | one page as Markdown — swap the trailing `/` for `.md`, e.g. [`/docs/guides/animation.md`](/docs/guides/animation.md) |

Each page also has a **Copy as Markdown** button under its title, for pasting into a chat.

## Tell your agent about them

[Section titled “Tell your agent about them”](#tell-your-agent-about-them)

Add this to your project’s `AGENTS.md` (or `CLAUDE.md`):

AGENTS.md

```md
## Video (framefields)


This project renders video with the `framefields` npm package.


- Docs index: https://framefields.dev/docs/llms.txt — fetch a page's `.md` URL from it when you need details.
- Rules to follow: https://framefields.dev/docs/reference/rules.md
- Film entry: `src/film.ts` exports `buildFilm()`. Tokens live in `src/theme.ts`.
- Input files live in `assets/` and are stored with Git LFS. Run `git lfs pull` before rendering in a fresh checkout; a ~130-byte file starting with `version https://git-lfs` is an un-pulled pointer. Check `git lfs ls-files` lists new media before committing. Renders go to `output/` (gitignored).
- Verify cheapest first: `npm run typecheck`, `comp.lint()`, `npm run frames -- <n…>`, `npm run grid`, then `npm run render` (QA must pass).
- Open every PNG you render before claiming it looks right.
- Preview notes from people are in `.framefields/preview-notes/notes.json`.
```

## Pick the right file

[Section titled “Pick the right file”](#pick-the-right-file)

* **Starting cold?** `llms-small.txt` fits most context windows and covers everything.
* **Working on one area?** Fetch just that page’s `.md` from the index.
* **Need effect props?** [`/docs/reference/effects.md`](/docs/reference/effects.md) lists all 52 effects; the skills’ generated catalog has types, ranges and defaults.

# Example prompts

> Prompts that work well with framefields agents, from a first title card to data stories, 3D product shots and vision effects.

Good prompts say **what** (content, length, size), **how it should feel** (pace, palette, type), and **how to check** (which frames to show you). The skills handle the API.

## Getting started

[Section titled “Getting started”](#getting-started)

```text
Set up framefields in this project and render a 3-second 1080p title card that says "Launch day".
Show me frames 0, 30 and 89 before exporting.
```

```text
Add framefields to this repo with a theme.ts, film.ts and render.ts. Use the Inter font in assets/fonts.
```

```text
Set up Git LFS for assets/ (fonts, footage, audio, images, LUTs, models, Lottie), gitignore output/,
then add and push the files I just dropped into assets/footage. Confirm they are in git lfs ls-files.
```

```text
This is a fresh clone: pull the LFS assets, check none are still pointers, then render frame 0.
```

## Motion and type

[Section titled “Motion and type”](#motion-and-type)

```text
Make a 10-second 1920×1080 intro at 30 fps, 100 BPM. Four scenes on the beat: the headline rises
glyph by glyph, three feature cards pop in, a stat counts up, the logo settles. Push transitions
between scenes. Render a frame grid of each scene for me.
```

```text
The second card enters too early and feels stiff. Move it one beat later and give it back.out(1.5).
```

## Data

[Section titled “Data”](#data)

```text
Turn data/revenue.csv into a 6-second animated bar chart, quarters on x, revenue in $ on y,
bars growing in one after another. Use the colors in theme.ts.
```

## 3D

[Section titled “3D”](#3d)

```text
Make a product shot: the word "framefields" extruded in yellow, studio lighting with a soft shadow,
a slow turn over 4 seconds, light bloom on the edges. Show me the first and last frame.
```

## Grading and effects

[Section titled “Grading and effects”](#grading-and-effects)

```text
Give the whole film a warm filmic grade with subtle grain and vignette. Keep it restrained.
```

```text
Key out the green screen in assets/footage/talent.mp4 and put her over the gradient background.
```

## Vision

[Section titled “Vision”](#vision)

```text
Put the headline "MOVE" behind the dancer in assets/footage/dancer.mp4. Check it with the exported video,
not a frame grid.
```

```text
Reframe assets/footage/interview.mp4 from 16:9 to 9:16, following the speaker. Blur any other faces.
```

## Sound

[Section titled “Sound”](#sound)

```text
Add assets/audio/score.mp3 with a 2-second fade in and out, plus a whoosh on every scene change.
Target -14 LUFS and fail if QA doesn't pass.
```

## Review with a person

[Section titled “Review with a person”](#review-with-a-person)

```text
Start the preview and give me the link. I'll leave notes on the picture — read them and fix each one,
then restart the preview so I can tick them off.
```

## Ship

[Section titled “Ship”](#ship)

```text
Lint the film, fix any warnings, then export output/final.mp4 at high quality with QA and show me the report.
```

# The skills

> What each of the four framefields skills teaches an agent, and the kind of request that loads it.

Four skills, each loaded automatically when a request matches its description. They hand off to each other, so an agent starting a project ends up with all four.

| Skill                 | Teaches                                                                                                                                                                                     | Loads on requests like                                                                         |
| --------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ---------------------------------------------------------------------------------------------- |
| `framefields`         | Checking Node/GPU, writing `package.json` with the right install-script approvals, installing, scaffolding `theme.ts` / `film.ts` / `render.ts`, Git LFS for media, a first verified render | *“Make a video with framefields”*, *“set up framefields in this repo”*                         |
| `framefields-compose` | Compositions and layer trees, flex layout, typography and fonts, keyframes and easing, text animators, scenes on a beat grid, transitions, mattes, 3D looks, charts                         | *“Add a title sequence”*, *“animate these cards in on the beat”*, *“make the logo spin in 3D”* |
| `framefields-effects` | Attaching effects to layers, whole films and regions; grading, grain, blur, LUTs, keying; signals on effect props; on-device vision (tracking, cutouts, pose)                               | *“Add film grain and a vignette”*, *“put the headline behind the dancer”*, *“blur the faces”*  |
| `framefields-render`  | The verification ladder: lint, single frames, frame grids, pixel tests, live preview and pinned notes, MP4 export with QA, diagnosing blank frames                                          | *“Render a frame grid of the intro”*, *“why is frame 40 black?”*, *“export the final MP4”*     |

## What they make an agent do differently

[Section titled “What they make an agent do differently”](#what-they-make-an-agent-do-differently)

* **Verify cheapest first.** Typecheck and `comp.lint()` before any GPU work; frames and grids before a full render.
* **Look at its own output.** Render a PNG, open it, and compare against intent — not assume.
* **Keep tokens in one place.** Colors, sizes, fonts and the beat grid live in `theme.ts`.
* **Avoid the classic breakages.** Fonts registered first, local integer keyframes, absolute vs flex positioning, `durationFrames` on audio, `process.exit()` after rendering. See [Authoring rules](/docs/reference/rules/).
* **Bring a person in before exporting.** Start the preview, share the URL, and act on pinned notes.

## Where they come from

[Section titled “Where they come from”](#where-they-come-from)

The skills are versioned with the `framefields` package and live in the framefields repo at [`plugins/framefields/skills/`](https://github.com/gatewai-dev/framefields/tree/main/plugins/framefields/skills). `framefields-effects` includes a generated catalog of every effect and prop. If a skill gets something wrong, [open an issue](https://github.com/gatewai-dev/framefields/issues).

# The agent workflow

> The write → check → look → show → ship loop an agent should follow with framefields, with the code for each step.

Every step returns a file or data an agent can inspect. Go in order: each step catches in seconds what the next takes minutes to show.

## 0. Project shape

[Section titled “0. Project shape”](#0-project-shape)

```plaintext
src/theme.ts      tokens: size, fps, colors, fonts, beat grid
src/film.ts       export async function buildFilm(): Promise<Composition>
src/render.ts     frames | grid | preview | video
assets/           every input file, tracked with Git LFS
  fonts/          .ttf .otf
  footage/        .mp4 .mov .webm
  audio/          .mp3 .wav
  images/         .png .jpg .webp
  luts/           .cube
  models/         .glb .gltf .obj .fbx
  lottie/         .json .lottie
  captions/       .srt (plain text, no LFS needed)
output/           renders (gitignored, never committed)
.gitattributes    LFS rules for assets/
```

Also commit the lockfile and pin `packageManager`. Keep `film.ts` free of side effects: it builds the film and renders nothing. That way the same repo renders on a [render worker](/docs/guides/render-service/#project-checklist) unchanged.

`render.ts` is in [Getting started](/docs/getting-started/#3-scaffold). Keeping `buildFilm()` separate means tests, preview and export all build the same film. Reference assets from the module URL (`path.resolve(HERE, "../assets/…")`) so the [preview](/docs/guides/preview/) can load them too.

## Assets and Git LFS

[Section titled “Assets and Git LFS”](#assets-and-git-lfs)

Source media goes in `assets/` and is stored with [Git LFS](https://git-lfs.com), so the repo stays small and clones stay fast. Renders go in `output/` and are never committed.

### Set up once per repo

[Section titled “Set up once per repo”](#set-up-once-per-repo)

```bash
git lfs install                       # once per machine
git lfs track "assets/fonts/**" "assets/footage/**" "assets/audio/**" \
  "assets/images/**" "assets/luts/**" "assets/models/**" "assets/lottie/**"
echo "output/" >> .gitignore
git add .gitattributes .gitignore
git commit -m "Track assets with Git LFS"
```

`git lfs track` writes the rules to `.gitattributes` (`assets/footage/** filter=lfs diff=lfs merge=lfs -text`, and so on). Commit that file before adding any media. Ignore `output/`, never `*.mp4` — that would also hide your footage.

### Push assets

[Section titled “Push assets”](#push-assets)

Add and push as usual; Git uploads the LFS objects with the push.

```bash
git add assets/footage/interview.mp4
git lfs ls-files                      # the new file must be listed — otherwise it is not in LFS
git commit -m "Add interview footage"
git push
```

If a file was committed before its rule existed, it went into normal Git. Fix it with `git rm --cached <file>`, `git add <file>` and commit again.

### Pull assets

[Section titled “Pull assets”](#pull-assets)

A clone with LFS installed downloads assets automatically. Anywhere else — a fresh container, a CI runner, an agent sandbox, or a clone made with `GIT_LFS_SKIP_SMUDGE=1` — fetch them before rendering:

```bash
git lfs install
git lfs pull                          # all assets for the current commit
git lfs pull --include="assets/fonts/**,assets/footage/intro.mp4"   # or just what this render needs
```

Check nothing is still a pointer: in `git lfs ls-files`, `*` means the file is downloaded and `-` means it is still a pointer.

Pointers render blank

An un-pulled LFS file is a \~130-byte text pointer starting with `version https://git-lfs.github.com/spec/v1`. framefields cannot decode it: fonts fail to register, and videos, images and audio come out blank or throw. If a render shows missing media in a fresh checkout, run `git lfs pull` before changing any code.

In GitHub Actions, check out with `lfs: true` (`actions/checkout`) so the runner has the real files.

## 1. Check without the GPU

[Section titled “1. Check without the GPU”](#1-check-without-the-gpu)

```bash
npx tsc --noEmit -p .
```

```ts
const film = await buildFilm();
for (const d of film.lint()) console.log(`${d.severity} ${d.code}: ${d.message}${d.layerId ? ` [${d.layerId}]` : ""}`);
```

Fix every error and warning before rendering. Codes and fixes: [Lint & QA codes](/docs/reference/qa-codes/).

## 2. Look at key frames

[Section titled “2. Look at key frames”](#2-look-at-key-frames)

```bash
npm run frames -- 0 30 60 89
```

Open each PNG in `output/` and compare it to what was asked. A black frame usually means a font that isn’t registered, a layer off canvas, opacity stuck at 0, or an asset that is still a Git LFS pointer (`git lfs pull`).

## 3. Look at motion

[Section titled “3. Look at motion”](#3-look-at-motion)

```bash
npm run grid -- 0 6 12 18 24 30 36 42
```

One image of the whole move: easing hitches, text that re-wraps, wrong z-order, missing cuts. Not for vision layers (they need frames in order).

## 4. Show a person

[Section titled “4. Show a person”](#4-show-a-person)

```bash
npm run preview   # run in the background; prints the URL
```

* Give the person the URL, or open it in your own browser pane.
* Read the command’s output: page errors and lifecycle lines show up there.
* When they say they left notes, read `.framefields/preview-notes/notes.json` and open each open note’s `note-<id>.jpg` before editing. Each note has the frame, the spot (`x`, `y` as fractions) and the text.
* Rerun the preview after changes; the open tab reloads at the same moment. Mark notes `"done": true` when addressed.

## 5. Ship with QA

[Section titled “5. Ship with QA”](#5-ship-with-qa)

```ts
import { formatQaReport } from "framefields";


const result = await film.renderVideo({ outputPath: "output/film.mp4", quality: "high", qa: { targetLufs: -14 } });
console.log(formatQaReport(result.qa!));
process.exit(result.qa!.passed ? 0 : 1);
```

`qa.passed` is false on any error or warning: dropped, black or frozen frames, clipping, loudness off target, plus every lint finding. Treat it as the gate.

## 6. Render on a server

[Section titled “6. Render on a server”](#6-render-on-a-server)

To render on a GPU machine instead of locally, push the commit and its LFS assets, then submit a job naming the repo, the ref, `src/film.ts` and `buildFilm`. Go through the [project checklist](/docs/guides/render-service/#project-checklist) first. When a job fails, read its `code` and `hint`: resubmit unchanged only when `retryable` is true.

## In CI

[Section titled “In CI”](#in-ci)

A runner without a GPU renders on the CPU: on Linux once Mesa lavapipe is installed (`apt-get install -y libvulkan1 mesa-vulkan-drivers`), on Windows through the built-in WARP. That is enough for frames and short exports; run long films on a GPU runner or the [render worker](/docs/guides/render-service/). On any runner you can also gate on types and `lint()` without rendering:

scripts/check.ts

```ts
import { CompositorProgramSchema } from "framefields";
import { buildFilm } from "../src/film.js";


const film = await buildFilm();
const bad = film.lint().filter((d) => d.severity !== "info");
const schema = CompositorProgramSchema.safeParse(film.toSpec());
if (bad.length || !schema.success) {
  for (const d of bad) console.error(`${d.code}: ${d.message}`);
  process.exit(1);
}
process.exit(0);
```

## Habits that keep agents fast

[Section titled “Habits that keep agents fast”](#habits-that-keep-agents-fast)

* Change one scene at a time and re-render only its frames.
* Keep all magic numbers in `theme.ts`; a tweak is then one line.
* Name layers with `id` — lint, QA and chart children report by id.
* Put asset paths through `import.meta.url` so the preview loads them too.

# Why framefields for your AI agents

Coding agents have fundamentally changed how software gets built. An agent can draft an entire backend service, run unit tests, inspect stack traces, fix regressions, and submit a pull request in minutes.

Yet until now, asking an AI agent to produce a video—whether a product teaser, an onboarding motion graphic, a dynamic data chart, or a localized social clip—has meant hitting an architectural brick wall.

The existing options were never designed for an autonomous loop:

1. **Desktop GUI Suites (After Effects, Premiere, Final Cut)**: Built around human eye-hand coordination, complex mouse interactions, proprietary binary project files, and zero headless programmatic inspection.
2. **Headless Browser Capture (Puppeteer, Playwright, Remotion)**: Wrapping a full Chromium browser to render HTML/CSS and taking screenshots frame by frame.

While headless browsers brought code into video production, they were designed to display interactive web pages to humans—not to be high-performance, deterministic video rendering engines.

**framefields was built to give coding agents a native, deterministic, GPU-accelerated video engine.** Here is why that matters.

***

## 1. The Headless Browser Problem

[Section titled “1. The Headless Browser Problem”](#1-the-headless-browser-problem)

Headless Chromium is an extraordinary tool for testing web apps. But when used as a video engine driven by an autonomous agent, its limitations compound quickly:

| Bottleneck              | Headless Chromium                                              | framefields                                                        |
| ----------------------- | -------------------------------------------------------------- | ------------------------------------------------------------------ |
| **Frame Capture**       | Lay out DOM tree $\to$ composite layers $\to$ read back pixels | Rendered straight into GPU textures via WebGPU (Dawn)              |
| **Clock Model**         | Wall-clock timers (`setTimeout`, `rAF`) prone to drift         | Pure deterministic function: `(comp, frame) → pixels`              |
| **Typography**          | CPU rasterization; scales blur or re-rasterize                 | Vector outlines solved per pixel on the GPU; crisp at any scale    |
| **Memory Footprint**    | 2–4 GB per worker process                                      | 200–400 MB per worker                                              |
| **3D & Shaders**        | Separate canvas contexts, difficult layer compositing          | Calibrated 3D camera lining up 1:1 with 2D pixels; 50+ GPU effects |
| **On-device AI Vision** | External pipelines or complex WebGL hacks                      | Embedded ONNX runtime for pose, mattes, and tracking               |

When an agent renders a 30-second 4K video using a browser screenshot loop, workers consume gigabytes of RAM, wall-clock timing fluctuates across CI runs, and frame drops ruin motion smoothness.

Agents need a framework that treats video not as a series of web page screenshots, but as a pure, compiled GPU pipeline.

***

## 2. Determinism: Every Frame is a Function of its Number

[Section titled “2. Determinism: Every Frame is a Function of its Number”](#2-determinism-every-frame-is-a-function-of-its-number)

In framefields, there are no timers, no `Date.now()`, and no asynchronous rendering race conditions.

```ts
(composition, frameNumber) => pixels
```

Frame 45 of your video is mathematically guaranteed to render the exact same pixels on an Apple Silicon laptop, a Linux server with Vulkan, or a Cloudflare Worker container.

This determinism unlocks three critical agent capabilities:

* **Parallel rendering without order dependencies**: Render frames 0–30 on GPU worker A and frames 31–60 on GPU worker B.
* **Repeatable regression testing**: An agent can assert on exact pixel values or frame-to-frame diffs in Vitest.
* **Cheapest-first feedback**: The agent doesn’t need to render an entire MP4 file just to know if a transition worked. It can render frame 45 directly to PNG in 8 milliseconds.

***

## 3. The “Cheapest First” Agent Feedback Loop

[Section titled “3. The “Cheapest First” Agent Feedback Loop”](#3-the-cheapest-first-agent-feedback-loop)

AI agents work best when the feedback loop is fast, structured, and cheap. When feedback takes five minutes or yields unparsable error logs, agents hallucinate and thrash.

framefields structures the video development process into six progressive tiers:

```plaintext
[1. Typecheck] ──> [2. Static Lint] ──> [3. Keyframe PNGs] ──> [4. Frame Grid] ──> [5. Human Preview] ──> [6. Full QA Encode]
   (0.2 sec)          (0.05 sec)            (0.1 sec)             (0.8 sec)         (Interactive)          (GPU render)
```

### Tier 1: Type Safety (`tsc --noEmit`)

[Section titled “Tier 1: Type Safety (tsc --noEmit)”](#tier-1-type-safety-tsc---noemit)

Because every layer, effect, transition, and keyframe in framefields is strongly typed in TypeScript, syntax errors, mismatched property names, and invalid color formats are caught by the language server before anything touches the GPU.

### Tier 2: Static Document Lint (`comp.lint()`)

[Section titled “Tier 2: Static Document Lint (comp.lint())”](#tier-2-static-document-lint-complint)

Before allocating GPU memory, the agent runs `comp.lint()`. This executes a document-level static analysis on the composition tree and returns **structured JSON diagnostics**, not unstructured logs:

```ts
const diagnostics = comp.lint();
// [
//   {
//     severity: "warning",
//     code: "animation_truncated",
//     message: "Keyframe at frame 120 exceeds layer duration of 90 frames",
//     layerId: "hero-title"
//   }
// ]
```

Each code maps to an exact entry in the framefields troubleshooting catalog, giving the agent a direct path to self-correction without human intervention.

### Tier 3 & 4: Keyframe Inspection and Contact-Sheet Grids

[Section titled “Tier 3 & 4: Keyframe Inspection and Contact-Sheet Grids”](#tier-3--4-keyframe-inspection-and-contact-sheet-grids)

Once linting passes, the agent renders only the critical inflection points to PNG:

```bash
npm run frames -- 0 30 60 90
```

With multimodal vision capabilities, coding agents can open the resulting PNG images, verify layout alignment, check color contrast, and examine easing. By rendering a contact-sheet grid (`renderFrameGrid`), an agent evaluates transitions and camera moves across time in a single image.

### Tier 5 & 6: Pinned Notes & Automated QA

[Section titled “Tier 5 & 6: Pinned Notes & Automated QA”](#tier-5--6-pinned-notes--automated-qa)

During interactive review, a human can scrub the WebGPU live preview and drop a note: *“Make the logo enter 10 frames earlier.”* The preview saves this to a local `notes.json` file. The agent reads `notes.json`, identifies the layer, adjusts the keyframe frames, and commits the fix.

Finally, `renderVideo({ qa: true })` validates black frames, audio sync, contrast ratios, and playback bitrate before signing off on `qa.passed: true`.

***

## 4. Typography and 3D Built for Motion

[Section titled “4. Typography and 3D Built for Motion”](#4-typography-and-3d-built-for-motion)

When an agent creates motion design, text and spatial depth are paramount.

### Resolution-Independent Typography

[Section titled “Resolution-Independent Typography”](#resolution-independent-typography)

In framefields, glyph outlines are not rasterized to bitmaps at build time. Their vector curves are evaluated per pixel on the GPU. Whether an agent scales text by 10× during a camera zoom, animates letter-by-letter with After Effects–style range selectors, or renders at 8K resolution, text never pixelates or exhibits texture filtering artifacts.

### Pixels That Line Up in 3D

[Section titled “Pixels That Line Up in 3D”](#pixels-that-line-up-in-3d)

Typical engines require switching between a 2D canvas and a 3D WebGL context with mismatched coordinate systems.

framefields calibrates its 3D camera so that **$z = 0$ corresponds 1:1 to canvas pixels**:

```ts
import { Composition, Layer, Lighting, Camera } from "framefields";


const comp = new Composition({ width: 1920, height: 1080 });


// 2D title card
comp.add(Layer.text("CYBERNETIC", { x: 960, y: 300, fontSize: 72 }));


// 3D model positioned directly on the 2D grid
comp.add(
  Layer.model("assets/models/device.glb", {
    x: 960,
    y: 600,
    z: 0, // matches 2D coordinates perfectly
    scale: 1.5,
  }).animate((k) => [
    k.fromTo("rotationY", 0, Math.PI * 2, { durationFrames: 120 })
  ])
);


comp.add(Lighting.studio());
```

***

## 5. On-Device Vision: Video That Reacts to Media

[Section titled “5. On-Device Vision: Video That Reacts to Media”](#5-on-device-vision-video-that-reacts-to-media)

One of the most complex tasks for video editors is rotoscoping, object tracking, and reframing. framefields embeds an ONNX runtime directly in the rendering pipeline.

Without writing external Python scripts or making slow API calls to cloud services, your agent can write video code that:

* Detects faces and human poses frame by frame.
* Automatically isolates a subject with a real-time person matte.
* Pins graphical callouts or 3D annotations to a tracked bounding box.
* Automatically reframes a 16:9 landscape clip into 9:16 vertical video for mobile platforms.

```ts
// Extract person matte and pin a dynamic graphics ring to their wrist
const { personMatte, keypoints } = comp.withVision({
  model: "segmentation",
  source: "assets/footage/presenter.mp4",
});


comp.add(
  Layer.box({ width: 60, height: 60, borderRadius: 30, borderColor: "#f5e300" })
    .pinToObject(keypoints.rightWrist)
);
```

***

## 6. Giving Your Agent the Skills

[Section titled “6. Giving Your Agent the Skills”](#6-giving-your-agent-the-skills)

We designed framefields so any coding agent can start building production-ready videos immediately. The instructions teach the agent project structure, timing conventions, layer composition, and the step-by-step review loop.

### Claude Code

[Section titled “Claude Code”](#claude-code)

```text
/plugin install framefields
```

### Cursor, Codex, Gemini CLI, Windsurf, Copilot

[Section titled “Cursor, Codex, Gemini CLI, Windsurf, Copilot”](#cursor-codex-gemini-cli-windsurf-copilot)

```bash
npx skills add gatewai-dev/framefields
```

Then prompt your agent:

```text
Create a 5-second product teaser for a developer analytics platform in 1080x1920 60fps.
Include an animated bar chart, kinetic headline typography on a beat grid,
and inspect keyframes before encoding the final MP4.
```

Your agent will scaffold the project, write the composition, lint the layer tree, review the generated PNG frames, and render the MP4 on the GPU.

***

## The Video Engine for the Agentic Era

[Section titled “The Video Engine for the Agentic Era”](#the-video-engine-for-the-agentic-era)

When coding agents have direct access to a deterministic, high-performance GPU engine, video ceases to be a manual bottleneck. Video becomes code: version-controlled, diffable, automated, and testable.

Explore the [Documentation](/docs/) to see the full API, or browse [framefields on GitHub](https://github.com/gatewai-dev/framefields).

# 3D

> Cameras and camera moves, lights and lighting rigs, physically based environments, primitives, extruded text and models.

3D and 2D share one depth buffer. The camera is calibrated so **`z = 0` maps 1:1 onto canvas pixels**: a box at `x: 960, y: 540, z: 0` sits in the centre of a 1920×1080 frame. y points down, so “up” is −y.

## The fast path: a lighting rig

[Section titled “The fast path: a lighting rig”](#the-fast-path-a-lighting-rig)

```ts
import { Composition, Layer3D, LayerAnimation, Lighting } from "framefields";


const W = 1920, H = 1080;
const comp = new Composition({ width: W, height: H, fps: 30, durationMs: 4000, backgroundColor: "#0b0b0c" });


comp.add(Lighting.studio({ target: [W / 2, H / 2, 0] }));          // environment + key/rim/fill, soft shadow


comp.add(
  Layer3D.text({
    text: "framefields", fontFamily: "Archivo", fontWeight: 800, fontSize: 280,
    extrusionDepth: 26, fill: "#f5e300", bevelColor: "#6b6300", roughness: 0.35,
    x: W / 2, y: H / 2, z: 0,
  }).animate(LayerAnimation.create().keys("rotateY", [[0, -20], [120, 6, "sine.inOut"]])),
);
```

Rigs: `Lighting.studio`, `soft`, `sunset`, `dramatic`. Options: `target`, `distance`, `rotation`, `intensity`, `exposure` (stops), `toneMapping`, `shadows` (default on), `shadowSoftness`, `fog`, `bloom`. A rig returns an array; `comp.add` takes it, or spread it into `children`.

## Camera

[Section titled “Camera”](#camera)

```ts
import { CameraAnimation, Layer } from "framefields";


comp.add(
  Layer.camera({ x: 960, y: 540, z: -1000, targetX: 960, targetY: 540, targetZ: 0 }).animate(
    CameraAnimation.camera()
      .orbit({ azimuth: { from: -30, to: 30 }, elevation: { from: 15, to: 15 }, radius: { to: 1200 }, start: 0, end: 300, ease: "sine.inOut" })
      .rackFocus({ fromDistance: 2000, toDistance: 1200, fromFStop: 1.8, toFStop: 4, start: 20, end: 60 }),
  ),
);
```

`CameraAnimation.camera()` moves: `orbit`, `dolly`, `truck`, `pedestal`, `pan`, `tilt`, `roll`, `rackFocus`, `handheld({ translationAmplitude, rotationAmplitude })`. Camera `mode` is `"lookAt"` (default), `"free"` or `"orbit"`.

## Lights by hand

[Section titled “Lights by hand”](#lights-by-hand)

```ts
import { Layer, Light } from "framefields";


comp.add([
  Layer.environment("studio", {   // presets: studio | soft | sunset | night | neutral
    exposure: 0.3,
    toneMapping: "neutral",       // default; "agx" filmic, "aces" punchy, "none"
    rotation: 30,
    fog: { color: "#0b0b0c", start: 2000, end: 5000 },
    bloom: true,                  // or { intensity, threshold, radius }
  }),
  Light.directional({ x: 400, y: -1200, z: -600, targetX: 960, targetY: 540, castShadow: true }),
  Light.ambient("#ffffff", 0.3),
]);
```

`Light.ambient`, `Light.directional`, `Light.point`, `Light.spot` (also `Layer.ambientLight`, `Layer.directionalLight`, `Layer.pointLight`, `Layer.spotLight`).

| Rule                                                                       | Why                                                                                               |
| -------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------- |
| Add `Layer.environment(...)` for `roughness` / `metallic` to mean anything | Without it the scene uses classic Blinn-Phong (lint: `pbr_without_environment`)                   |
| One `castShadow` light, directional or spot                                | Only the first casts (`multiple_shadow_lights`); point/ambient can’t (`shadow_unsupported_light`) |
| Tone mapping `"neutral"` keeps brand colors as authored                    | `"agx"` rolls highlights off filmically                                                           |
| Bloom only takes light well past white                                     | Use `emissiveIntensity` ≳ 3 to make something glow                                                |
| Set `fog.color` to the background                                          | Surfaces fade into it seamlessly                                                                  |

## Primitives

[Section titled “Primitives”](#primitives)

```ts
Layer3D.roundedBox({ x: 960, y: 540, z: 0, width: 300, height: 80, depth: 300, radius: 18, taper: 0.9, color: "#ff5a1f", roughness: 0.45 }); // keycap
Layer3D.sphere({ x: 700, y: 540, z: 0, radius: 90, color: "#e8e8e8", metallic: 1, roughness: 0.15 });                                // chrome
Layer3D.cylinder({ x: 1200, y: 540, z: 0, radius: 12, height: 400, color: "#111", emissive: "#4da3ff", emissiveIntensity: 3 });       // neon tube
```

| Builder                                          | Makes                                                                             |
| ------------------------------------------------ | --------------------------------------------------------------------------------- |
| `Layer3D.roundedBox / sphere / cylinder`         | solid smooth-shaded meshes; equal shapes share GPU buffers, so hundreds are cheap |
| `Layer3D.cube({ size, faces })`                  | a cube whose faces are 2D layers                                                  |
| `Layer3D.carousel({ radius, items })`            | 2D items on a ring                                                                |
| `Layer3D.prism`, `Layer3D.plane`, `Layer3D.grid` | prisms, flat cards, ground grids                                                  |
| `Layer3D.text` / `Layer3D.extrudedText`          | extruded, bevelled type from your registered fonts                                |

### Materials & glass refraction

[Section titled “Materials & glass refraction”](#materials--glass-refraction)

Meshes take a `material`: `"pbr"` (default), `"unlit"`, `"glass"`, `"acrylic"` or `"frosted"`.

`glass`, `acrylic` and `frosted` materials physically refract what is behind them on screen — with chromatic dispersion, slab thickness, tint, and studio reflections — instead of rendering as simple translucent plastic:

```ts
Layer3D.roundedBox({
  x: 960, y: 540, z: 0,
  width: 320, height: 200, depth: 36, radius: 18,
  material: "glass",
  color: "#eef5ff",
  roughness: 0.03, // smooth clear glass; higher roughness gives frosted glass
});
```

Surface props on meshes: `color`, `tint` (animates as `color`), `roughness`, `metallic`, `emissive`, `emissiveIntensity`, `castShadow`. Flat layers cast shadows only with `castShadow: true`; meshes cast by default.

## Models

[Section titled “Models”](#models)

```ts
comp.add(Layer.glb("assets/models/character.glb", { x: 960, y: 640, z: 0, scale: 2.5, loop: true }));
```

Loaders: `Layer.glb`, `gltf`, `obj`, `fbx`, `stl`, `ply`, `vox`, `threeDS`, `off` (also on `Layer3D`). glTF brings base color, normal, metallic-roughness, occlusion and emissive maps, and skeletal animation (up to 128 bones). Animate playback with `animationProgress`, `animationTime` or `animationSpeed`.

## 3D rigid-body physics

[Section titled “3D rigid-body physics”](#3d-rigid-body-physics)

Animate primitives and models with deterministic 3D physics (Rapier) using `Physics.world3d`: collision responses, joints, and forces. See the [Physics & particles guide](/docs/guides/physics/#3d-rigid-bodies).

## Animatable 3D props

[Section titled “Animatable 3D props”](#animatable-3d-props)

`z`, `rotateX/Y/Z`, `scaleX/Y/Z`, `roughness`, `metallic`, `emissive`, `emissiveIntensity`, `exposure`, `intensity`, plus camera props (`cameraX/Y/Z`, `targetX/Y/Z`, `cameraFov`, `focusDistance`, `fStop`, `orbitRadius`, `orbitAzimuth`, `orbitElevation`).

# Generative AI (ai)

> Generate images, videos, sound and assets directly within framefields compositions using the AI gateway.

`framefields` includes built-in generative AI capabilities via the `ai` module. One API key unlocks every supported generative model with unified credit billing and direct WebGPU composition integration.

```ts
import { ai, Composition, Layer } from "framefields";


// Subscribe to a queue-based generation
const { data } = await ai.subscribe("seedream/5.0-pro", {
  input: {
    prompt: "An atmospheric 35mm film still of a cinematic neon street",
    size: "2K",
  },
});


console.log("Image URL:", data.images[0].url);
```

## Setup & API Keys

[Section titled “Setup & API Keys”](#setup--api-keys)

You don’t need separate provider accounts or keys.

1. Sign in at [framefields.dev/dashboard](https://framefields.dev/dashboard).
2. Top up credits on the [Billing page](https://framefields.dev/dashboard/billing) with Dodo Payments.
3. Create an API key on the [API Keys page](https://framefields.dev/dashboard/keys).
4. Add the key to your project `.env`:

```bash
FRAMEFIELDS_API_KEY=ff_live_...
```

A project made with `framefields init` already has a `.env.example` (commit it) and ignores `.env` (never commit that). Its `render.ts` loads `.env` when the file exists, so renders and scripts pick up the key without extra setup. Without a key everything else still works, including in CI: only `ai.*` calls need it.

Or configure it in code:

```ts
import { ai } from "framefields";


ai.config({
  apiKey: process.env.FRAMEFIELDS_API_KEY,
});
```

## Model Catalog

[Section titled “Model Catalog”](#model-catalog)

Explore available models in the [Model Catalog](https://framefields.dev/models), including text-to-image, image-to-video, voice synthesis, and upscaling models. Each model page lists its input and output fields.

| Kind                                                       | Model IDs                                                                     |
| ---------------------------------------------------------- | ----------------------------------------------------------------------------- |
| Video (text, image, audio and video to video, with sound)  | `seedance/2.5`, `seedance/2.0`, `seedance/2.0-fast`, `seedance/2.0-mini`      |
| Images (generation, editing, batches, layer decomposition) | `seedream/5.0-pro`, `seedream/5.0-flash`, `seedream/5.0-lite`, `seedream/4.5` |
| Speech, music and sound effects (with word timestamps)     | `seed-audio/1.0`                                                              |

You can also fetch the catalog programmatically for agentic workflows:

```bash
curl https://api.framefields.dev/v1/models
```

## API Methods

[Section titled “API Methods”](#api-methods)

### `ai.subscribe` (Recommended)

[Section titled “ai.subscribe (Recommended)”](#aisubscribe-recommended)

Submits a request to the async processing queue, waits for completion, and returns the final output. Ideal for video and high-resolution rendering models.

```ts
const { data } = await ai.subscribe("seedance/2.5", {
  input: {
    prompt: "A neon geometric sculpture rotating in a void, slow orbit",
    resolution: "1080p",
    ratio: "16:9",
    duration: 5,
  },
  pollInterval: 2000, // poll status every 2 s
});


console.log(data.video.url);
```

### `ai.run`

[Section titled “ai.run”](#airun)

Executes over a single HTTP connection and returns the output. Fine for images and audio; video models only run through the queue (`ai.subscribe` or `ai.queue`).

```ts
const { data } = await ai.run("seed-audio/1.0", {
  input: {
    text_prompt: "Warm analog synth bed, then a calm narrator says: Welcome to framefields.",
    audio_config: { format: "mp3", enable_subtitle: true },
  },
});


console.log(data.audio.url, data.subtitle); // subtitle has sentence and word timestamps
```

### `ai.stream`

[Section titled “ai.stream”](#aistream)

Streams generation events (tokens or partial image steps) over Server-Sent Events (SSE), for models that stream.

```ts
const stream = await ai.stream("flux/dev", {
  input: { prompt: "Cyberpunk cityscape" },
});


for await (const event of stream) {
  console.log("Progress:", event);
}
```

### `ai.storage.upload`

[Section titled “ai.storage.upload”](#aistorageupload)

Uploads local buffers or files to framefields storage through signed upload URLs and returns a CDN URL any model accepts as input (e.g. image-to-video or style transfer). `Blob`s inside `input` upload automatically.

```ts
import fs from "node:fs";


const file = fs.readFileSync("input.png");
const fileUrl = await ai.storage.upload(new Blob([file], { type: "image/png" }));


const { data } = await ai.subscribe("seedance/2.5", {
  input: {
    prompt: "Slow cinematic camera push forward",
    content: [{ type: "image_url", image_url: { url: fileUrl }, role: "first_frame" }],
  },
});
```

## Using Generated Assets in Compositions

[Section titled “Using Generated Assets in Compositions”](#using-generated-assets-in-compositions)

Download or stream generated assets directly into your `framefields` layer hierarchy:

```ts
import { ai, Composition, Layer } from "framefields";
import fs from "node:fs/promises";


// 1. Generate an image asset
const { data } = await ai.subscribe("seedream/5.0-pro", {
  input: { prompt: "Abstract glowing glass caustics, dark mode", size: "2K" },
});


// 2. Fetch image buffer
const res = await fetch(data.images[0].url);
const imageBuffer = Buffer.from(await res.arrayBuffer());
await fs.writeFile("assets/generated-bg.png", imageBuffer);


// 3. Compose in WebGPU layer
export default Composition.create({
  width: 1920,
  height: 1080,
  fps: 60,
  durationInFrames: 300,
  children: [
    Layer.image({
      source: "assets/generated-bg.png",
      width: 1920,
      height: 1080,
    }),
  ],
});
```

## Credits & Error Handling

[Section titled “Credits & Error Handling”](#credits--error-handling)

Credits are charged in micro-USD ($0.000001 resolution) based on model usage plus margin.

When credits are depleted, requests will return HTTP `402 Payment Required`:

```ts
try {
  await ai.subscribe("seedream/5.0-pro", { input: { prompt: "..." } });
} catch (err: any) {
  if (err.status === 402) {
    console.error("Credits depleted. Top up at https://framefields.dev/dashboard/billing");
  }
}
```

| HTTP Status | Error Type             | Solution                                                                         |
| ----------- | ---------------------- | -------------------------------------------------------------------------------- |
| `401`       | `invalid_api_key`      | Verify `FRAMEFIELDS_API_KEY` in `.env`                                           |
| `402`       | `insufficient_credits` | Add credits in [Billing](https://framefields.dev/dashboard/billing)              |
| `402`       | `key_limit_reached`    | Increase key monthly limit in [API Keys](https://framefields.dev/dashboard/keys) |
| `429`       | `rate_limited`         | Respect retry-after backoff (60 submits/min, 600 status/min)                     |

# Animation

> Keyframes with LayerAnimation, easing, composing animations, procedural drivers, kinetic typography and 3D transforms on 2D layers.

Animations are data. Build one with `LayerAnimation`, attach it with `.animate()`. All frames are **local to the layer** and **integers**.

## Two keyframes: `fromTo`

[Section titled “Two keyframes: fromTo”](#two-keyframes-fromto)

```ts
import { LayerAnimation } from "framefields";


const entrance = LayerAnimation.create()
  .fadeIn(0, 20, "power2.out")                                    // opacity 0 → 1, frames 0–20
  .fromTo("y", 460, 400, { start: 0, end: 24, ease: "expo.out" })  // to the layout y, not 0
  .fromTo("scale", 0.95, 1, { start: 0, end: 30, ease: "back.out(1.4)" });


card.animate(entrance);
```

Shortcuts: `fadeIn(start, end, ease)`, `fadeOut(start, end, ease)`, `slideInX/slideInY(from, to, start, end, ease)`, `rotateX/rotateY/rotateZ`, `translateZ`, `perspective`, `letterSpacing`, `typewriter(start, end)`, `kineticSweep(fromOffset, toOffset, start, end, ease)`.

## Many keyframes: `keys`

[Section titled “Many keyframes: keys”](#many-keyframes-keys)

Each tuple is `[frame, value, ease?]`; the ease shapes motion **into** that key. Frames are rounded, so beat math can go straight in.

```ts
const pop = LayerAnimation.create()
  .keys("scale", [[0, 0], [10, 1.12, "back.out(1.6)"], [16, 1, "power2.out"]])
  .keys("opacity", [[0, 0], [6, 1]]);


LayerAnimation.keys("rotation", [[0, -8], [20, 0, "expo.out"]]); // static form
```

Color props (`fill`, `color`, `borderColor`, …) keyframe too, blended in OKLab.

## Easing

[Section titled “Easing”](#easing)

`<name>.<dir>` with optional parameters: `"expo.out"`, `"back.out(1.7)"`, `"power2.inOut"`.

| Names      | `none` `power1` `power2` `power3` `sine` `circ` `expo` `back` `elastic` `bounce` `spring` `cubic` `hold` |
| ---------- | -------------------------------------------------------------------------------------------------------- |
| Directions | `in` `out` `inOut` (`"linear"` is an alias for `none`)                                                   |

| Use                       | Ease                               |
| ------------------------- | ---------------------------------- |
| Snap-overshoot entrances  | `back.out(1.4–1.7)`                |
| Decelerating moves        | `expo.out`, `power3.out`, `spring` |
| Exits                     | `power2.in`                        |
| Loops, counters, spinners | `linear` (and nothing else)        |

## Composing animations

[Section titled “Composing animations”](#composing-animations)

`.animate()` **adds** to what a node has. Tracks on the same prop merge into one sorted track; a later key wins on a shared frame.

```ts
const rainIn = LayerAnimation.keys("y", [[0, -300], [d, -300], [d + 12, 0, "back.out(1.5)"]]);
const press  = LayerAnimation.keys("y", [[40, 0], [42, 6, "power2.in"], [47, 0, "power2.out"]]);
const glow   = LayerAnimation.keys("color", [[42, "#ffd400"], [60, "#2a2b2d", "sine.inOut"]]);


key.animate(rainIn).animate(press, glow);      // or LayerAnimation.merge(rainIn, press, glow)
```

## Procedural drivers

[Section titled “Procedural drivers”](#procedural-drivers)

```ts
LayerAnimation.create()
  .wiggle("x", { frequency: 1.5, amplitude: 25, seed: 3 })  // seeded noise, deterministic
  .spring("scale", { damping: 14, stiffness: 190, mass: 1 });
```

Bind any prop to a signal with `.signal(prop, signal, { multiplier, offset, smoothing })` — see [Audio & signals](/docs/guides/audio-and-signals/) and [Physics & particles](/docs/guides/physics/).

## Kinetic typography

[Section titled “Kinetic typography”](#kinetic-typography)

Text animators work per glyph, word or line on the GPU. Add one to the text, then sweep its selection with `kineticSweep`.

```ts
import { Layer, LayerAnimation, TextAnimator } from "framefields";


Layer.text("TypeScript in.", {
  fontFamily: "Archivo", fontWeight: 800, fontSize: 200, fill: "#e6e8e6",
  animators: [TextAnimator.waveRise({ y: 90, rotationX: 70 })],
}).animate(LayerAnimation.create().kineticSweep(-1, 1, 0, 40));
```

| Preset                                                         | Effect                           |
| -------------------------------------------------------------- | -------------------------------- |
| `TextAnimator.waveRise({ y, rotationX, opacity, blur, unit })` | glyphs rise and untilt in a wave |
| `TextAnimator.wordPop({ scale, y })`                           | words pop in                     |
| `TextAnimator.blurIn(...)`                                     | glyphs resolve from blur         |
| `TextAnimator.flip3D(...)`                                     | glyphs flip in around X          |
| `TextAnimator.colorSweep(...)`                                 | color passes through the text    |

`unit` is `"character"` (default), `"word"` or `"line"`. For terminal-style typing use `.typewriter(start, end)`.

## 3D transforms on 2D layers

[Section titled “3D transforms on 2D layers”](#3d-transforms-on-2d-layers)

Any layer can tilt in perspective.

```ts
Layer.box({
  width: 600, height: 400, background: "#181715", borderRadius: 24,
  transformStyle: "preserve-3d", perspective: 1200,
}).animate(
  LayerAnimation.create()
    .fromTo("rotateX", 25, 0, { start: 0, end: 30, ease: "expo.out" })
    .fromTo("rotateY", -35, 0, { start: 0, end: 30, ease: "expo.out" })
    .fromTo("translateZ", -200, 0, { start: 0, end: 30, ease: "expo.out" }),
);
```

For cameras, lights and meshes, see [3D](/docs/guides/3d/).

## Gotchas

[Section titled “Gotchas”](#gotchas)

* Keyframes past `durationFrames` never play (`animation_truncated`).
* Animating `x`/`y` overrides the layout position.
* A track whose keys all hold one value does nothing (`static_track`).

# Audio & signals

> Soundtracks, audio effects, deterministic procedural SFX, and signals that drive any prop or effect from time, data or sound.

## Soundtrack

[Section titled “Soundtrack”](#soundtrack)

```ts
import { Effect, Layer } from "framefields";


comp.addAudio(
  Layer.audio("assets/audio/score.mp3", {
    durationFrames: totalFrames, // required: without it audio can cut short
    volume: 0.9,
  }).apply(Effect.audioFade({ fadeInDuration: 2, fadeOutDuration: 2, fadeOutCurve: "scurve" })),
);
```

`Layer.audio` takes `volume`, `startFrame`, `durationFrames`, `muted`. Video layers carry their own sound (`muted`, `volume`, `trimStartSec`).

Audio effects attach with `.apply()`: `Effect.audioFade`, `audioCompressor`, `audioParametricEq`, `audioReverb`, `audioDelay`, `audioNoiseGate`, `stereoPanning` (or the classes `AudioFade`, `AudioCompressor`, …). Props are in the [Effects reference](/docs/reference/effects/).

## Procedural SFX

[Section titled “Procedural SFX”](#procedural-sfx)

Whooshes, risers, impacts, downshifters and glitches are synthesized on the CPU, seeded, and placed on a bar grid — the same input always gives the same samples.

```ts
import fs from "node:fs/promises";
import { encodeStereoWav, Layer, mixSfxInto, renderSfx, softLimit } from "framefields";


const sampleRate = 48_000;
const seconds = totalFrames / FPS;
const bed: [Float32Array, Float32Array] = [
  new Float32Array(Math.ceil(seconds * sampleRate)),
  new Float32Array(Math.ceil(seconds * sampleRate)),
];
const ctx = { sampleRate, secondsPerBar: 2.4 }; // 100 BPM, 4/4


mixSfxInto(bed, [
  renderSfx({ type: "riser", atBar: 0.5, volume: 0.4 }, { ...ctx, seed: 1 }),
  renderSfx({ type: "impact", atBar: 1, volume: 0.8 }, { ...ctx, seed: 2 }),
  renderSfx({ type: "whoosh", atBar: 2.75, volume: 0.5 }, { ...ctx, seed: 3 }),
]);
softLimit(bed);


await fs.writeFile("output/sfx.wav", encodeStereoWav(bed, { sampleRate }));
comp.addAudio(Layer.audio("output/sfx.wav", { durationFrames: totalFrames }));
```

`atBar` is where the sound **starts** (fractional bars allowed); a riser that should land on a cut starts `durationSec` earlier. `mixAudioTracks` mixes several tracks headlessly.

## Signals

[Section titled “Signals”](#signals)

A signal is a value per frame. Bind one to any animatable prop, or pass it to any numeric effect prop.

```ts
import { LayerAnimation, Signal } from "framefields";


const pulse = Signal.builder({ type: "sawtooth", frequency: 2, amplitude: 0.08, offset: 1 }); // 120 BPM


card.animate(
  LayerAnimation.create().signal("scale", pulse, { multiplier: 1, offset: 0, smoothing: 2 }),
);
```

The prop becomes `value × multiplier + offset` each frame. Signals are sampled at the **composition** frame.

| Constructor                                                     | Value                                                                          |
| --------------------------------------------------------------- | ------------------------------------------------------------------------------ |
| `Signal.builder({ type, frequency, amplitude, offset })`        | `sine`, `triangle`, `sawtooth`, `square`, `bounce`, or `custom` with `fn(ctx)` |
| `Signal.sine({ frequency, min, max, phase })` / `Signal.cosine` | periodic wave; `frequency` in Hz                                               |
| `Signal.fromArray(values, fps)`                                 | one sample per frame, e.g. a precomputed kick envelope                         |
| `Signal.programmatic((ctx) => number)`                          | any function of `ctx.frame`, `ctx.time`, `ctx.progress`                        |
| `Signal.frame` / `Signal.time` / `Signal.progress`              | the clock itself, as signals (read `.value`)                                   |
| `Signal.state(v)` / `Signal.computed(() => …)`                  | reactive values, e.g. `Signal.computed(() => 55 - Signal.frame.value * 0.1)`   |

### Shared signals by name

[Section titled “Shared signals by name”](#shared-signals-by-name)

```ts
comp.addSignal("kick", Signal.fromArray(kickEnvelope, FPS));
LayerAnimation.create().signal("opacity", "kick", { multiplier: 0.5, offset: 0.5 });
```

A signal object is registered automatically; a string must match an `addSignal` name. Anything else throws.

### Colors

[Section titled “Colors”](#colors)

`.colorSignal(prop, signal, options)` drives a color prop; modes are `interpolate`, `hueRotate` and `threshold`.

## Audio-reactive visuals

[Section titled “Audio-reactive visuals”](#audio-reactive-visuals)

Extract an envelope from audio with the `AudioSignalExtractor` effect (modes, attack/release, beat detection), or precompute one in Node and feed it through `Signal.fromArray`. Precomputing keeps the film a pure function of the frame and is the easiest to test.

# Charts & media

> Video, image, SVG, Lottie and caption layers, the Media chain, and animated charts built from native nodes.

## Media layers

[Section titled “Media layers”](#media-layers)

```ts
Layer.video("assets/footage/clip.mp4", { width: 1920, height: 1080, fit: "cover", trimStartSec: 2, muted: true });
Layer.image("assets/images/still.webp", { width: 800, height: 600, fit: "contain" }); // .png .jpg .webp
Layer.svg("assets/images/logo.svg", { width: 400, height: 120, fit: "contain" });
Layer.lottie("assets/lottie/badge.json", { width: 400, height: 400, fit: "contain", loop: true }); // .json .lottie
Layer.caption("assets/captions/subs.srt", { /* bottom-anchored, auto-wrapping cues */ });
```

| Builder         | Notes                                                                                                    |
| --------------- | -------------------------------------------------------------------------------------------------------- |
| `Layer.video`   | decoded with mediabunny/WebCodecs; `fit`, `trimStartSec`, `muted`, `volume`                              |
| `Layer.image`   | raster only. Passing `.json`/`.lottie`/video/audio is an error (`unsupported_image_format`)              |
| `Layer.svg`     | vector image                                                                                             |
| `Layer.lottie`  | Lottie JSON and dotLottie via ThorVG; `fit: "contain" \| "cover" \| "fill"`, `loop`, `speed`, `autoplay` |
| `Layer.caption` | SRT-driven subtitles                                                                                     |

All of them position, animate, matte, take effects and tilt into 3D like any layer. Asset paths resolve from the working directory; build them from `import.meta.url` if the [preview](/docs/guides/preview/) must load them too.

## Media chains

[Section titled “Media chains”](#media-chains)

Process a single clip outside the layer tree, e.g. to grade footage or extract a still.

```ts
import { ApplyLUT, Media } from "framefields";


const graded = Media.video("assets/footage/raw_log.mp4").apply(new ApplyLUT({ lutUrl: "assets/luts/rec709.cube" }));
await graded.renderFrame({ atMs: 1500 });             // PNG Buffer
await graded.renderVideo({ outputPath: "output/graded.mp4" });
```

`Media.video`, `Media.image`, `Media.lottie`, `Media.audio`; each has `apply`, `renderFrame`, `renderFrameGrid`, `renderVideo`.

## Charts

[Section titled “Charts”](#charts)

`Layer.chart(options, boxProps)` builds a chart out of ordinary boxes, paths and text — d3 does the scales and geometry, framefields draws it with your fonts.

```ts
Layer.chart(
  {
    type: "bar",               // line | area | bar | scatter | candlestick | pie | donut
    width: 900, height: 480,
    categories: ["Q1", "Q2", "Q3", "Q4"],
    series: [
      { name: "Revenue", data: [12, 19, 24, 31] },
      { name: "Costs", data: [8, 11, 13, 15] },
    ],
    yAxis: { min: 0, ticks: 4, format: "$,.0f" },          // d3-format string or (v) => string
    fontFamily: "Inter", fontSize: 24, textColor: "#e6e8e6", gridColor: "#2a2b2d",
    animate: { start: 10, duration: 30, stagger: 6, ease: "power3.out" }, // or false
  },
  { id: "revenue", position: "absolute", x: 120, y: 200 },
);
```

| Option                                                                | Notes                                                                |
| --------------------------------------------------------------------- | -------------------------------------------------------------------- |
| `series: [{ name, data, color, type, curve, pointRadius }]` or `data` | `data` for pie/donut slices and candles                              |
| `stacked`                                                             | stack bars/areas                                                     |
| `xAxis`, `yAxis`                                                      | `min`, `max`, `ticks`, `grid`, `format`                              |
| `legend`, `valueLabels`                                               | booleans                                                             |
| `colors`, `textColor`, `gridColor`, `background`, `cornerRadius`      | style                                                                |
| `innerRadius`                                                         | donut hole                                                           |
| `upColor`, `downColor`                                                | candlesticks                                                         |
| `animate`                                                             | built-in reveal: lines draw on, bars grow, points and slices stagger |

Children get stable ids — `<id>-bar-<series>-<i>`, `<id>-line-<series>`, `<id>-slice-<i>` — so you can find them in lint output. Keep scatter plots to a few hundred points: each point is a node.

# Effects

> Attach GPU effects to layers, whole compositions, regions and media chains; order them; drive their props with signals.

Effects are typed classes. Construct one with props, attach it, and the GPU runs it each frame. The full list with every prop is in the [Effects reference](/docs/reference/effects/).

```ts
import { Blur, Curves, FilmGrain, Vignette } from "framefields";          // or "framefields/effects"
```

## Where to attach

[Section titled “Where to attach”](#where-to-attach)

```ts
// 1. One layer (chain as many as you like, in order)
Layer.video("assets/footage/plate.mp4", { fit: "cover" })
  .withEffect(new Curves({ master: [{ x: 0, y: 0.05 }, { x: 0.5, y: 0.52 }, { x: 1, y: 0.95 }] }))
  .withEffect(new Vignette({ strength: 40, radius: 0.85, softness: 0.6 }));


// 2. The whole frame
comp.apply(new FilmGrain({ strength: 12, size: 1.5, animated: true }));


// 3. A region of a plate (rectangle, rounded, or a tracked object)
comp.section({
  source: "assets/footage/interview.mp4",
  x: 400, y: 200, width: 300, height: 300, borderRadius: 150,
  effects: [new Blur({ strength: 25, blurType: "Gaussian" })],
});


// 4. A clip outside the layer tree
import { ApplyLUT, Crop, Media } from "framefields";
const clip = Media.video("assets/footage/raw_log.mp4")
  .apply(new ApplyLUT({ lutUrl: "assets/luts/cinematic.cube", intensity: 1 }))
  .apply(new Crop({ cropType: "rectangle", topPercentage: 10, heightPercentage: 80 }));
await clip.renderFrame({ atMs: 1500 });
```

`.withEffects([a, b])` attaches several at once. Lowercase factories work too: `Effect.blur({ strength: 9 })`.

## Value ranges

[Section titled “Value ranges”](#value-ranges)

Ranges come from each effect’s schema; out-of-range values warn (set `FRAMEFIELDS_STRICT_PROPS=1` to throw). Many “strength” props are **0–100**, not 0–1:

| Effect         | Prop                                      | Range      | Default  | Tasteful |
| -------------- | ----------------------------------------- | ---------- | -------- | -------- |
| `Vignette`     | `strength`                                | 0–100      | 50       | 20–45    |
| `FilmGrain`    | `strength`                                | 0–100      | 15       | 8–20     |
| `FilmGrain`    | `size`                                    | 0.5–4      | 1.5      | 1.2–1.8  |
| `Blur`         | `strength`                                | 0–100      | 5        | —        |
| `ColorBalance` | `cyanRed` / `magentaGreen` / `yellowBlue` | −100–100   | 0        | ±4–12    |
| `ColorKey`     | `similarity`, `smoothness`                | 0–1        | 0.4, 0.1 | —        |
| `Modulate`     | `brightness`, `contrast`, `saturation`    | multiplier | 1        | 0.9–1.2  |

## Order

[Section titled “Order”](#order)

1. Tonal grading first: `Curves`, `Levels`, `ColorBalance`, `SelectiveColor`, `ApplyLUT`.
2. Then optics: `Blur`, `DepthOfField`, `Vignette`.
3. Grain last, as the outermost pass: `FilmGrain`.

For a still export, set `animated: false` on `FilmGrain` so the frame is deterministic across renders of different lengths.

## Drive props with signals

[Section titled “Drive props with signals”](#drive-props-with-signals)

Any numeric effect prop accepts a signal instead of a number. The uniform updates per frame without rebuilding the pipeline.

```ts
import { Blur, Layer, Signal } from "framefields";


const breathe = Signal.sine({ frequency: 0.5, min: 0, max: 20 }); // Hz, sampled at the composition frame


Layer.image("assets/images/bg.jpg").withEffect(new Blur({ strength: breathe, blurType: "Gaussian" }));
```

Vision signals plug in the same way, e.g. `new Vignette({ strength: vision.segmentation.subject.coverage })`. See [Vision](/docs/guides/vision/).

## Temporal effects (`Trails`, `Datamosh`)

[Section titled “Temporal effects (Trails, Datamosh)”](#temporal-effects-trails-datamosh)

Temporal effects process frame history to create motion persistence, light painting, and optical flow artifacts:

```ts
import { Datamosh, Layer, Trails } from "framefields";


// Motion echoes, feedback tunnels, and light painting
Layer.video("assets/footage/dancer.mp4").withEffect(
  new Trails({
    decay: 0.92,       // history retained per frame (0.8 = short echoes, 0.97 = long light trails)
    mode: "blend",     // "blend" | "over" | "add" (glow accumulation) | "max"
    zoom: 1.01,        // history scale per frame (pulls past into a tunnel)
    rotation: 0.5,     // history rotation per frame in degrees
    hueShift: 2,       // psychedelic hue rotation per frame
  }),
);


// Pixel smearing along motion vectors
Layer.video("assets/footage/action.mp4").withEffect(
  new Datamosh({
    keyframes: [0, 60, 120], // clean I-frames; pixels drag along optical flow between them
    strength: 1.5,           // how far pixels are dragged along motion
    bleed: 0.04,             // clean picture mixed back in each frame
  }),
);
```

## Families

[Section titled “Families”](#families)

| Family             | Effects                                                                                                                                                            |
| ------------------ | ------------------------------------------------------------------------------------------------------------------------------------------------------------------ |
| Grading            | `Curves`, `Levels`, `ColorBalance`, `SelectiveColor`, `GradientMap`, `ShadowsHighlights`, `ApplyLUT`, `ExtractLUT`, `Modulate`, `ChannelSplitter`, `ChannelMerger` |
| Texture & lens     | `FilmGrain`, `Vignette`, `Blur`, `MotionBlur`, `UnsharpMask`, `HighPass`, `HalftoneScreen`, `TileOffset`, `TemporalDeflicker`                                      |
| Distortion         | `DisplacementMap`, `Liquify`, `MeshWarp`, `CornerPin`, `Rotate`, `Flip`, `KenBurns`, `ResizerScaler`                                                               |
| Temporal           | `Trails`, `Datamosh`                                                                                                                                               |
| 3D & light         | `Relight3D`, `SSAO`, `PBRGlass`, `DepthOfField`, `RefractionCaustics3D`, `CameraParallax3D`                                                                        |
| Mattes & keying    | `ColorKey`, `Crop`, `RefineEdge`, `MaskMath`, `PatchHeal`                                                                                                          |
| Generators & style | `NoiseGenerator`, `ProceduralVFX`, `Paint`, `LayerStyle`                                                                                                           |
| Audio              | `AudioFade`, `AudioCompressor`, `AudioParametricEq`, `AudioReverb`, `AudioDelay`, `AudioNoiseGate`, `StereoPanning`, `AudioSignalExtractor`                        |
| Vision             | `Vision` (prefer `withVision`, see [Vision](/docs/guides/vision/))                                                                                                 |

## Restraint

[Section titled “Restraint”](#restraint)

Grain and vignette should be felt, not seen. If a viewer notices the effect before the content, halve it.

# Layout & text

> Boxes, flex layout, shapes, fonts, GPU text, rich text, measurement, colors, gradients and track mattes.

## Boxes and flex

[Section titled “Boxes and flex”](#boxes-and-flex)

```ts
import { Layer } from "framefields";


Layer.flex({
  width: 1920, height: 1080,
  dir: "column",            // "row" | "column"
  justify: "center",        // start | center | end | space-between | space-around
  align: "center",          // start | center | end | stretch
  gap: 24, padding: 80,
  wrap: false,
  background: "#0d0f12",
  children: [
    Layer.box({ padding: 16, background: "#1e232d", borderRadius: 20, children: [/* … */] }),
  ],
});
```

| Prop                                         | Notes                                                                       |
| -------------------------------------------- | --------------------------------------------------------------------------- |
| `width`, `height`                            | px, `"auto"` (intrinsic), `"fit"` (shrink-wrap) or `"fill"` (parent extent) |
| `position: "absolute"` + `x`, `y`            | take the layer out of flow; `x`/`y` alone also make it absolute             |
| `background`                                 | a color or a CSS gradient string                                            |
| `borderRadius`, `borderColor`, `borderWidth` | no CSS `border` shorthand                                                   |
| `overflow: "hidden"`                         | clip children to the box                                                    |
| `opacity`, `scale`, `rotation`               | static or animated                                                          |
| `startFrame`, `durationFrames`               | when the layer is on screen                                                 |
| `id`                                         | stable name; used by lint/QA messages and chart children                    |

## Shapes

[Section titled “Shapes”](#shapes)

```ts
Layer.shape("circle", { width: 200, height: 200, fill: "#ff5a1f" });
Layer.shape("star", { width: 240, height: 240, fill: "radial-gradient(#fff, #ffb800)" });
Layer.shape("path", { d: "M0 0 L200 0 A100 100 0 0 1 0 0Z", strokeColor: "#fff", strokeWidth: 4, trimEnd: 0 })
  .animate(LayerAnimation.create().fromTo("trimEnd", 0, 1, { start: 0, end: 30 })); // draw-on
```

Types: `rect`, `circle`, `ellipse`, `polygon`, `star`, `arrow`, `path`. Animate `trimStart` / `trimEnd` / `trimOffset` for stroke draw-ons.

## Fonts

[Section titled “Fonts”](#fonts)

There are no system fonts in headless WebGPU. Register every family before building any text layer.

```ts
import { FontManager } from "framefields";


await FontManager.register({ family: "Inter", source: "assets/fonts/Inter.ttf" });
await FontManager.register({ family: "Inter", source: "assets/fonts/Inter-Bold.ttf", weight: 700 });
await FontManager.registerEmojiFont("assets/fonts/NotoColorEmoji.ttf"); // emoji need a CBDT/CBLC color font
```

## Text

[Section titled “Text”](#text)

```ts
Layer.text("PRECISION MOTION", {
  fontFamily: "Inter",
  fontSize: 96,
  fontWeight: 700,
  fill: "#ffffff",
  letterSpacing: 2,
  lineHeight: 104,
  width: 1400,          // wrap width; set it or text may clip
  align: "center",      // "start" | "center" | "end" (follows text direction)
  verticalAlign: "middle", // "top" | "middle" | "bottom" inside the box height
});
```

Text is drawn by the Slug algorithm: glyph curves are evaluated per pixel in the shader, so it stays sharp at any size and under 3D camera moves. Arabic, Hebrew and Persian shape right-to-left automatically (`direction: "auto"`; force with `"rtl"` / `"ltr"`).

### Rich text

[Section titled “Rich text”](#rich-text)

```ts
import { Layer, bold, color, mark, rich } from "framefields";


Layer.text(
  rich`Scale ${bold("10x", { fill: "#6366f1" })} faster with ${mark("120 FPS", { bg: "#10b98133" })}`,
  { fontFamily: "Inter", fontSize: 56, fill: "#f8fafc" },
);
```

Span helpers: `span`, `bold`, `italic`, `color`, `mark`, `size`, `sub`, `sup`. A span takes `fontFamily`, `fontSize`, `fontWeight`, `fill`, `letterSpacing`, `baselineShift`, `opacity`.

### Measure with the real font

[Section titled “Measure with the real font”](#measure-with-the-real-font)

```ts
const m = FontManager.measure("Hello, world", { fontFamily: "Inter", fontSize: 64 });
m.width; m.height; m.lines; m.caretX(5); m.glyphs[0].x; // px, relative to the text box
```

Use it to place carets, underline a word, or size a pill to its label instead of guessing.

## Color helpers

[Section titled “Color helpers”](#color-helpers)

Every helper returns a hex string.

```ts
import { Color } from "framefields";


Color.oklch(0.72, 0.16, 250);           // equal L = equal perceived brightness
Color.mix("#ff5a1f", "#1f6bff", 0.5);   // perceptual (OKLab) mix
Color.ramp(["#3b82f6", "#8b5cf6", "#ec4899"])(0.3);
Color.steps(["#0e0d0c", "#ff5a1f"], 5); // 5 swatches
Color.lighten(c, 0.1); Color.darken(c, 0.1); Color.alpha("#000", 0.4);
Color.contrast(fg, bg);                 // WCAG ratio: ≥ 4.5 body, ≥ 3 large
Color.readableOn(bg);                   // black or white, whichever reads
```

## Gradients

[Section titled “Gradients”](#gradients)

CSS strings, usable in any `background` and any shape `fill`. Linear, radial and conic, up to 8 stops, hard stops allowed.

```ts
import { Gradient } from "framefields";


Layer.box({ background: "linear-gradient(135deg, #ff5a1f, #ffb800 60%, #7c3aed)" });
Layer.shape("circle", { fill: Gradient.radial(["#fff", "transparent"]) });
Layer.box({ background: Gradient.conic(["red", "yellow", "lime", "aqua", "blue", "magenta", "red"]) });
```

A gradient that doesn’t parse draws nothing; `comp.lint()` reports it as `invalid_gradient`.

## Track mattes

[Section titled “Track mattes”](#track-mattes)

`node.withMatte(matte, mode)` shows `node` only where the matte is opaque (`"alpha"`, default), transparent (`"alpha-inverted"`), bright (`"luma"`) or dark (`"luma-inverted"`). The matte is laid out like a sibling and is never drawn itself.

```ts
// Text revealed by a sliding bar
Layer.text("LAUNCH", { position: "absolute", x: 560, y: 480, width: 800, height: 120, fontSize: 120 })
  .withMatte(
    Layer.box({ position: "absolute", x: 560, y: 480, width: 800, height: 120, background: "#fff" })
      .animate(LayerAnimation.keys("x", [[0, -240], [24, 560, "expo.out"]])),
  );


// Video inside type
Layer.video("assets/footage/surf.mp4", { position: "absolute", x: 0, y: 0, width: 1920, height: 1080, fit: "cover" })
  .withMatte(Layer.text("OCEAN", { position: "absolute", x: 0, y: 340, width: 1920, fontSize: 380, align: "center", fill: "#fff" }));
```

# Physics & particles

> Deterministic 2D and 3D rigid-body simulation with Rapier, live performer collision, force fields, and WebGPU compute particles.

framefields includes deterministic rigid-body physics powered by Rapier (compiled to WebAssembly) and high-density GPU particle systems simulated in WebGPU compute shaders.

Physics simulates while the composition is built (or steps frame-by-frame in live mode) and replays as per-frame signals. Frame order does not affect the outcome: frame 80 always renders identically on every machine.

## 2D rigid bodies

[Section titled “2D rigid bodies”](#2d-rigid-bodies)

```ts
import { Composition, Layer, LayerAnimation, Physics } from "framefields";


const W = 1920, H = 1080, FPS = 30;
const comp = new Composition({ width: W, height: H, fps: FPS, durationFrames: 120, backgroundColor: "#0b0c10" });


const world = await Physics.world2d({ fps: FPS, frames: 120, gravity: 2400 });
world.walls({ width: W, height: H });


const cardLayer = Layer.box({
  width: 320, height: 200, background: "#1f2937", borderRadius: 16,
  position: "absolute", x: W / 2 - 160, y: 100,
});


const card = world.body(cardLayer, { restitution: 0.4, friction: 0.6 });
world.simulate();


// Flash on impact
const flash = Layer.box({ width: W, height: H, background: "#ffffff", opacity: 0 });
flash.animate(LayerAnimation.create().signal("opacity", card.impact, { multiplier: 0.8 }));


comp.add([flash, cardLayer]);
```

`world.body(target, options)` creates a body from any layer with an absolute position (`x`, `y`) and size, or from a plain rectangle `{ x, y, width, height }` (useful for invisible funnels and deflectors).

`world.simulate()` runs the simulation once and automatically binds each body’s layer `x`, `y`, and `rotation` to the simulated signals.

### Collider shapes

[Section titled “Collider shapes”](#collider-shapes)

By default (`shape: "auto"`), the collider is inferred from the layer:

| Layer type                                  | Inferred collider                             |
| ------------------------------------------- | --------------------------------------------- |
| `Layer.shape("circle", ...)`                | Circle                                        |
| `Layer.box` with `borderRadius`             | Rounded box matching corner radius            |
| \`Layer.shape(“polygon”                     | “star”                                        |
| `Layer.text` without explicit size          | Box measured automatically with `FontManager` |
| `Layer.box` / `Layer.image` / `Layer.video` | Rectangular box                               |

Force a specific shape with `shape: "box" | "circle" | "hull"` or override dimensions with `size: [width, height]`.

### Body options

[Section titled “Body options”](#body-options)

| Option            | Type                    | Default     | Description                                                            |
| ----------------- | ----------------------- | ----------- | ---------------------------------------------------------------------- |
| `type`            | \`“dynamic”             | “fixed”     | “kinematic”\`                                                          |
| `restitution`     | `number`                | `0.3`       | Bounciness, 0–1                                                        |
| `friction`        | `number`                | `0.6`       | Surface friction, 0–1                                                  |
| `density`         | `number`                | `1.0`       | Mass per unit area. Heavier bodies push lighter ones                   |
| `velocity`        | `[vx, vy]`              | `[0, 0]`    | Initial launch velocity in px/s                                        |
| `angularVelocity` | `number`                | `0`         | Initial spin in degrees/second                                         |
| `linearDamping`   | `number`                | `0`         | Air resistance per second                                              |
| `angularDamping`  | `number`                | `0`         | Spin resistance per second                                             |
| `gravityScale`    | `number`                | `1.0`       | Multiplier for world gravity (0 makes body float)                      |
| `lockRotation`    | `boolean`               | `false`     | Disables rotation so the body remains upright                          |
| `ccd`             | `boolean`               | `false`     | Continuous collision detection for thin, fast-moving bodies            |
| `sensor`          | `boolean`               | `false`     | Passes through bodies without collision while still reporting contacts |
| `startFrame`      | `number`                | `0`         | Simulation frame the body wakes up; holds start pose before            |
| `follow`          | `{ x?, y?, rotation? }` | —           | Kinematic driver: position and rotation each frame                     |
| `output`          | \`“signals”             | “keyframes” | “none”\`                                                               |

## Signals & reactions

[Section titled “Signals & reactions”](#signals--reactions)

Every body produces frame-rate signals sampled at composition time:

```ts
card.x;               // layer top-left X every frame
card.y;               // layer top-left Y every frame
card.rotation;        // continuous degrees (no 360 wrap jump)
card.centerX;         // body center X in px
card.centerY;         // body center Y in px
card.vx;              // velocity X in px/s
card.vy;              // velocity Y in px/s
card.speed;           // speed hypot(vx, vy) in px/s
card.angularVelocity; // degrees/s
card.sleeping;        // 1 when settled, 0 when moving
card.impact;          // exponential decay envelope on collision
```

### Impact & collisions

[Section titled “Impact & collisions”](#impact--collisions)

`body.impact` jumps whenever the body suffers a collision and decays over `impactDecay` frames (default 8 frames). A 1500 px/s change in velocity produces an impact level of 1.0.

`world.impact` produces the loudest impact across all bodies on each frame — perfect for triggering camera shake, lens flares, or audio whooshes.

`world.collisions` and `body.collisions` return an array of first contact events:

```ts
for (const contact of world.collisions) {
  // contact: { frame, body, with, speed }
  console.log(`Hit at frame ${contact.frame} with speed ${contact.speed} px/s`);
}
```

Use `world.collisions` to place impact sound effects (`renderSfx({ type: "impact", ... })`) or particle explosions precisely on contact frames.

## Bounds & joints

[Section titled “Bounds & joints”](#bounds--joints)

Create boundaries with built-in helpers:

```ts
// Ground floor sitting at y: 1000
world.ground({ y: 1000, friction: 0.8, restitution: 0.2 });


// Surrounding walls around the canvas box
world.walls({ width: 1920, height: 1080, thickness: 400, ceiling: true });
```

Connect bodies with joints:

```ts
// Pin to swing around a world point (default: body center)
world.pin(signpost, [960, 200]);


// Hinge two bodies together at a world pivot
world.hinge(bodyA, bodyB, [pivotX, pivotY]);


// Weld bodies rigidly together
world.weld(bodyParent, bodyChild);


// Damped spring between two bodies or body and point
world.spring(pendulum, [960, 100], { frequency: 2, dampingRatio: 0.2, length: 300 });


// Distance rope (slack when closer, taut at max length)
world.rope(weight, [960, 100], { length: 450 });
```

## Driving the simulation

[Section titled “Driving the simulation”](#driving-the-simulation)

Beyond gravity and collisions, worlds can be influenced by forces, impulses, and spatial fields:

```ts
// Steady acceleration (px/s², mass-independent)
world.force(balloon, { y: -1200 }); // buoyant lift


// Signal-driven acceleration (e.g. wind gusts from audio)
world.force(banner, { x: audioSignal });


// Instant velocity kick at a specific frame
world.impulse(ball, 30, [400, -800]); // kick at frame 30
```

### Force fields

[Section titled “Force fields”](#force-fields)

`world.field(callback, options)` is called every solver step for each awake body. The callback receives a `FieldSample2D` (`{ x, y, vx, vy, frame, body }`) and returns an acceleration `[ax, ay]` in px/s²:

```ts
// Radial vortex pulling bodies into center
world.field((s) => {
  const dx = 960 - s.x;
  const dy = 540 - s.y;
  const dist = Math.hypot(dx, dy);
  if (dist < 50 || dist > 600) return null;
  const pull = 1500 / dist;
  // Tangential swirl + inward pull
  return [-dy * pull + (dx / dist) * 200, dx * pull + (dy / dist) * 200];
});
```

## 3D rigid bodies

[Section titled “3D rigid bodies”](#3d-rigid-bodies)

`Physics.world3d` simulates rigid bodies in 3D for `Layer3D` primitives (`roundedBox`, `sphere`, `cylinder`), extruded text, and 3D models (`Layer.glb`, convex hull):

```ts
import { Layer3D, Physics } from "framefields";


const world = await Physics.world3d({ fps: 30, frames: 150, gravity: 2600 });
world.ground({ y: 1000 });


const box = Layer3D.roundedBox({
  width: 240, height: 160, depth: 32, radius: 12,
  x: 960, y: 200, z: 0,
  material: "glass",
});


const body = world.body(box, {
  restitution: 0.3,
  // Lock rotation to keep cards facing the camera while spinning in the image plane
  lockRotation: { x: true, y: true, z: false },
});


world.simulate();
```

Without a camera layer, the 3D scene’s `z = 0` plane maps 1:1 onto canvas pixels. Points from 2D video tracks or layout line up directly in the 3D simulation.

3D bodies expose `x`, `y`, `z`, `rotateX`, `rotateY`, `rotateZ`, `vx`, `vy`, `vz`, `speed`, and `impact` signals.

## Live worlds & performers

[Section titled “Live worlds & performers”](#live-worlds--performers)

Normally, `world.simulate()` computes the entire timeline up front when the composition is constructed.

When bodies interact with performers tracked via `comp.vision(clip)`, use **`world.live(comp)`** instead. Live mode advances the physics simulation frame-by-frame as each composition frame is requested:

```ts
const comp = new Composition({ width: W, height: H, fps: 30, durationFrames: 180 });
const dancer = comp.vision("assets/dancer.mp4");
const person = dancer.person.place({ width: W, height: H, fit: "cover" });


const world = await Physics.world3d({ fps: 30, frames: 180, gravity: 2400 });
world.ground({ y: H });


// Kinematic sphere colliders following dancer's wrists
world.body({ x: 0, y: 0, z: 0, width: 60, height: 60, depth: 60 }, {
  type: "kinematic",
  shape: "sphere",
  follow: {
    x: (f) => person.point("rightWrist", f).x,
    y: (f) => person.point("rightWrist", f).y,
    z: 0,
  },
});


// Live mode: steps as frames render; preview starts instantly without up-front delay
world.live(comp);
```

### Silhouette deflection

[Section titled “Silhouette deflection”](#silhouette-deflection)

The tracked performer’s silhouette can deflect rigid bodies using `person.sample(frame, x, y)`:

```ts
world.field((s) => {
  const sample = person.sample(s.frame, s.x, s.y);
  if (sample.alpha <= 0.1) return null;
  // Push outwards along the negative normal gradient
  return [-sample.gx * 8000, -sample.gy * 8000];
});
```

## Keyframe baking

[Section titled “Keyframe baking”](#keyframe-baking)

To export plain keyframe animations (inspectable in JSON or usable without the live physics runtime), use `bakeKeyframes` instead of `simulate`:

```ts
world.bakeKeyframes({ tolerance: 0.25 });
```

This applies Ramer–Douglas–Peucker trajectory simplification across each channel, leaving the minimum linear keyframes that stay within `tolerance` pixels of the simulation.

***

## GPU Particles (`Layer.particles`)

[Section titled “GPU Particles (Layer.particles)”](#gpu-particles-layerparticles)

`Layer.particles` simulates tens of thousands of particles in a WebGPU compute shader and draws them in a single instanced render call. Positions inside the particle layer are local layer pixels.

```ts
comp.add(Layer.particles({
  width: W, height: H,
  max: 20000,
  seed: 42,
  emitter: {
    shape: "rect",
    x: 0, y: H * 0.6,
    width: W, height: 200,
    rate: 3000,
  },
  lifetime: [4, 6],
  speed: [20, 80],
  direction: -90, spread: 180,
  gravity: 30,
  drag: 1.2,
  noise: { strength: 240, scale: 160, speed: 0.3 }, // divergence-free curl noise
  attractors: [{ x: W / 2, y: H / 2, strength: 1200, radius: 300 }],
  floor: H - 40, bounce: 0.3,
  size: [10, 2], sizeJitter: 0.4,
  shape: "soft",
  color: ["#ffdd00", "#ff5a1f", "#4a1205"],
  blend: "add",
  prewarm: 3, // starts already full
}));
```

### Dynamic particle drivers

[Section titled “Dynamic particle drivers”](#dynamic-particle-drivers)

Emitter position (`emitter.x`, `emitter.y`), emission `rate`, `gravity`, `drag`, `noise.strength`, attractor positions, and `push` forces can be driven by a static number, a signal, or a per-frame function `(frame: number) => number`.

This allows responsive particle effects:

* **Light painting**: Move a point emitter with a tracked hand `(f) => person.point("rightWrist", f).x`. Births automatically interpolate along the path between frames to avoid gaps.
* **Audio bursts**: Modulate `emitter.rate` with an audio drum signal.
* **Explosions**: Trigger instant bursts at specific frames with `emitter.bursts: [{ frame: 30, count: 5000 }]`.

### Performer interaction

[Section titled “Performer interaction”](#performer-interaction)

Connect particles directly to tracked performers using `push`:

```ts
const dancer = comp.vision("assets/dancer.mp4");
const person = dancer.person.place({ width: W, height: H, fit: "cover" });


comp.add(Layer.particles({
  width: W, height: H,
  push: {
    person,
    strength: 9000,
    sweep: 120000, // extra force imparted by fast-sweeping limbs
  },
  // ...
}));
```

Layer a video cutout of the same clip (`Layer.video(src).withVision({ mode: "matte" })`) above the particles so they appear behind the performer while being swept aside.

### Particle options reference

[Section titled “Particle options reference”](#particle-options-reference)

| Prop             | Type                                 | Default       | Description                                                                  |
| ---------------- | ------------------------------------ | ------------- | ---------------------------------------------------------------------------- |
| `max`            | `number`                             | `20000`       | Maximum particle capacity. Only uses GPU memory, no cost when idle           |
| `emitter.shape`  | \`“rect”                             | “point”       | “line”                                                                       |
| `emitter.rate`   | `PhysicsDriver`                      | `1000`        | Particles emitted per second                                                 |
| `emitter.bursts` | `Array<{ frame, count }>`            | `[]`          | Instant particle bursts at specific frames                                   |
| `lifetime`       | \`number                             | \[min, max]\` | `[2, 4]`                                                                     |
| `speed`          | \`number                             | \[min, max]\` | `[40, 160]`                                                                  |
| `direction`      | `number`                             | `-90`         | Launch direction angle in degrees (0 = right, -90 = up)                      |
| `spread`         | `number`                             | `360`         | Angular spread around direction in degrees                                   |
| `gravity`        | \`PhysicsDriver                      | \[gx, gy]\`   | `0`                                                                          |
| `drag`           | `PhysicsDriver`                      | `0.4`         | Velocity damping per second                                                  |
| `noise`          | `{ strength?, scale?, speed? }`      | —             | Curl noise flow for turbulent smoke and fluid drifts                         |
| `attractors`     | `Array<{ x, y, strength, radius? }>` | `[]`          | Points that pull (positive strength) or repel (negative strength)            |
| `floor`          | `number`                             | —             | Floor line in layer px where particles bounce                                |
| `bounce`         | `number`                             | `0.3`         | Bounciness at floor and bouncing bounds                                      |
| `bounds`         | \`“none”                             | “kill”        | “bounce”                                                                     |
| `shape`          | \`“circle”                           | “soft”        | “square”                                                                     |
| `blend`          | \`“normal”                           | “add”         | “screen”\`                                                                   |
| `color`          | \`string                             | string\[]\`   | `"#ffffff"`                                                                  |
| `colorBySpeed`   | `number`                             | —             | When set, maps color gradient according to speed (px/s to reach final color) |
| `prewarm`        | `number`                             | `0`           | Seconds simulated before frame 0 so the scene starts in mid-flight           |

## Reference example: Physics dancer

[Section titled “Reference example: Physics dancer”](#reference-example-physics-dancer)

The [`examples/28_physics_dancer`](https://github.com/gatewai-dev/framefields/tree/main/examples/28_physics_dancer) reference film demonstrates 3D rigid-body physics, live vision tracking, and GPU compute particles working together in a single shot:

* **3D glass cards:** `Layer3D.roundedBox({ material: "glass" })` cards thrown in on ballistic arcs that bounce off the performer.
* **Kinematic performer tracking:** Kinematic sphere colliders follow arm and head keypoints from `comp.vision(clip).person`.
* **Performer particle push:** 24,000 embers simulated with `Layer.particles` that part around the dancer’s silk dress via `push: { person }`.
* **Live execution:** `world.live(comp)` advances physics as each frame renders, starting the preview immediately.

Watch the render on [YouTube](https://youtu.be/POC7rKnuoE4).

# Live preview

> Serve a localhost WebGPU player that runs the composition itself, with sound, scrubbing, deep links, scripting and notes pinned on the picture.

`startPreview` serves a page that loads your composition’s module and renders it live with WebGPU in the browser — not a video stream. The server only hands over the bundle, your assets, and the soundtrack mixed by the export engine.

src/render.ts (preview mode)

```ts
import { startPreview } from "framefields";


const session = await startPreview(
  { entry: new URL("./film.ts", import.meta.url), export: "buildFilm" },
  { title: "Launch film" },
);
console.log(`Preview at ${session.url}`);
await session.closed; // "idle" | "replaced" | "closed"
process.exit(0);
```

`export` names a function returning the `Composition` (may be async) or a `Composition` itself.

## Options

[Section titled “Options”](#options)

| Option        | Default                            | Notes                                                                                       |
| ------------- | ---------------------------------- | ------------------------------------------------------------------------------------------- |
| `port`        | derived from the project directory | one stable URL per project; a new run replaces the old server and open tabs reload in place |
| `host`        | `127.0.0.1`                        | never leaves the machine                                                                    |
| `open`        | `false`                            | open the system browser; otherwise show `session.url` where you are                         |
| `title`       | —                                  | page header                                                                                 |
| `audio`       | `true`                             | mix and play the soundtrack                                                                 |
| `idleCloseMs` | `5000`                             | stop after the last tab closes; `false` keeps serving until `close()`                       |
| `root`        | project of the entry               | extra folders the page may load assets from                                                 |
| `log`         | stderr                             | where page errors and lifecycle lines go; `false` silences                                  |
| `notesDir`    | `.framefields/preview-notes`       | `false` turns notes off                                                                     |

## Assets must resolve in the browser

[Section titled “Assets must resolve in the browser”](#assets-must-resolve-in-the-browser)

The browser runs `film.ts` itself. Build asset paths from the module URL, not `process.cwd()`:

```ts
import path from "node:path";
import { fileURLToPath } from "node:url";


const HERE = path.dirname(fileURLToPath(import.meta.url));
const FONTS = path.resolve(HERE, "../assets/fonts");
```

`node:path`, `fileURLToPath` and synchronous `fs` reads work through shims; writes are skipped.

## For agents

[Section titled “For agents”](#for-agents)

* Run the preview as a **background** command and read its output: it reports when the page opened, each distinct page error (build failure, a frame that threw, missing font, no WebGPU), and why it stopped.
* After editing, run it again — same URL, and the open tab reloads at the same moment.
* Deep link: `#t=12.5` (seconds) or `#frame=750`.
* Script it: `window.framefieldsPreview` has `seek(seconds)`, `seekFrame(n)`, `play()`, `pause()`, `notes()`, `state()`. `seek` resolves once the frame is drawn — screenshot after it. `<html data-preview="ready">` marks the first drawn frame.

## Notes pinned on the picture

[Section titled “Notes pinned on the picture”](#notes-pinned-on-the-picture)

Paused, a person clicks a spot (or drags an area) and writes what should change. Each note is printed in the preview output and saved:

```plaintext
.framefields/preview-notes/
  notes.json        [{ id, frame, time, x, y, w?, h?, text, done }]
  note-<id>.jpg     the frame with the spot marked
```

To act on notes: read `notes.json`, open each open note’s picture, find the layer from the moment and the spot, edit, rerun the preview. Set `"done": true` yourself or let the person tick them off.

## Playback

[Section titled “Playback”](#playback)

Play runs in real time on the soundtrack’s clock and skips frames a slow device can’t draw; the page shows the frame rate it reaches. Paused, it draws the exact frame. Requires WebGPU (current Chrome, Edge or Safari).

# Render on a server

> Render a framefields project straight from git on a GPU worker (Docker, RunPod pod or serverless). Project checklist, job JSON, outputs and error codes.

The **renderer worker** renders a project from its git repo: it fetches a commit, pulls Git LFS assets, installs dependencies, renders an exported composition on a warm GPU, checks the file with ffprobe and uploads it. It runs as a Docker container on any NVIDIA host, a RunPod pod or a RunPod serverless endpoint. Deploying it: [renderer-service README](https://github.com/gatewai-dev/framefields/tree/main/apps/renderer-service).

The worker has no engine of its own. It renders with the `framefields` version in the project’s lockfile.

## Project checklist

[Section titled “Project checklist”](#project-checklist)

A project that follows [Getting started](/docs/getting-started/) already meets most of this. Check every line before submitting a job.

| Rule                                                                                                         | Why                                                                                     | Fails with                                                        |
| ------------------------------------------------------------------------------------------------------------ | --------------------------------------------------------------------------------------- | ----------------------------------------------------------------- |
| `framefields` is a dependency, version 2.2 or newer                                                          | 2.2 adds progress, cancel, warm workers and safe video encoders                         | `ENGINE_NOT_FOUND`; 2.1 renders but without those                 |
| A lockfile is committed and matches `package.json`: `pnpm-lock.yaml`, `package-lock.json` or `yarn.lock`     | the worker installs with `--frozen-lockfile` / `npm ci`. Bun lockfiles aren’t supported | `LOCKFILE_MISSING`, `INSTALL_FAILED`                              |
| `packageManager` is pinned, e.g. `"pnpm@10.13.1"`                                                            | the worker installs with that exact version                                             | install differences                                               |
| Install-script approvals are in `package.json` (only `skia-canvas`)                                          | the worker runs the same install as you                                                 | `Cannot find module '.../skia.node'`                              |
| The entry module only **builds** the film: `film.ts` exports `buildFilm`, and `render.ts` does the rendering | the worker imports the entry. Rendering or `process.exit()` at import time is rejected  | `EXPORT_SIDE_EFFECT`, `EXPORT_NOT_FOUND`                          |
| Every input file is under `assets/`, stored with Git LFS, and pushed (`git lfs ls-files`)                    | the worker pulls LFS objects for the commit it renders                                  | `LFS_POINTER_UNRESOLVED`, `LFS_OBJECT_MISSING`, `ASSET_NOT_FOUND` |
| Asset paths are built from `import.meta.url`, never `process.cwd()`                                          | the worker’s working directory isn’t your project                                       | `ASSET_NOT_FOUND`                                                 |
| Nothing writes into `node_modules` while rendering                                                           | the worker shares `node_modules` files read-only between projects                       | a permission error (`EACCES`/`EPERM`) with a hint                 |
| Everything is committed and pushed                                                                           | the worker renders the pushed commit, not your working tree                             | stale output                                                      |

## What the export can return

[Section titled “What the export can return”](#what-the-export-can-return)

src/film.ts

```ts
// One video
export async function buildFilm(args?: { locale?: string }): Promise<Composition> { … }


// Several named outputs
export const cuts = () => ({ main: buildFilm(), teaser: buildTeaser() });


// A plan, with settings per output
export const deliverables = () => ({
  outputs: [
    { name: "film", composition: buildFilm() },
    { name: "sheet", composition: buildFilm(), render: { kind: "frameGrid", grid: { step: 30 } } },
  ],
});
```

This is the same `{ entry, export }` convention as [`startPreview`](/docs/guides/preview/). `target.args` is passed to a function export. `target.select` picks names from a record or plan.

## Submit a job

[Section titled “Submit a job”](#submit-a-job)

```bash
curl -s "$WORKER/v2/jobs" -H "authorization: Bearer $RENDERER_API_KEY" -H "content-type: application/json" -d '{
  "version": 2,
  "source": { "repo": "https://github.com/acme/films.git", "ref": "main", "workdir": "videos/launch" },
  "target": { "entry": "src/film.ts", "export": "buildFilm" },
  "render": { "kind": "video", "codec": "h264", "quality": "high" },
  "output": { "type": "s3", "key": "launch/{shortSha}/{name}.{ext}" }
}'
# → {"id":"…","status":"IN_QUEUE"}
```

| Call                        | Returns                                                                       |
| --------------------------- | ----------------------------------------------------------------------------- |
| `GET /v2/jobs/<id>`         | status and result: output URLs or keys, the rendered commit SHA, timings, GPU |
| `GET /v2/jobs/<id>/events`  | live progress (Server-Sent Events)                                            |
| `POST /v2/jobs/<id>/cancel` | stops the render between frames                                               |

On **RunPod serverless**, send the same JSON as `{"input": …}` to `https://api.runpod.ai/v2/<endpoint-id>/run`. Then poll `/status/<id>`, which includes progress.

Useful job fields:

| Field                | Use                                                                                                 |
| -------------------- | --------------------------------------------------------------------------------------------------- |
| `source.ref`         | branch, tag or commit SHA. The result reports the SHA it rendered                                   |
| `source.commit`      | fail if the ref has moved past this commit                                                          |
| `source.workdir`     | the package folder, for a monorepo                                                                  |
| `source.lfs.include` | pull only the assets this render needs                                                              |
| `render.kind`        | `video` (`h264`, `vp9`, `vp8`, `gif`), `audio` (`aac`, `mp3`, `opus`), `frames`, `frameGrid`, `lut` |
| `render.qa.failOn`   | QA codes that fail the job (`QA_FAILED`). The outputs still upload                                  |
| `output.type`        | `s3` (key template), `presigned` (one URL per output), `local`, `inline` (base64)                   |
| `idempotencyKey`     | a retry of the same commit returns the earlier upload                                               |

Key templates can use `{jobId}`, `{sha}`, `{shortSha}`, `{ref}`, `{date}`, `{name}`, `{index}` and `{ext}`.

## Errors

[Section titled “Errors”](#errors)

A failed job returns `{ code, message, phase, retryable, hint }`. When `retryable` is true, resubmit the same job. Otherwise, fix the cause first:

| Code                                                       | Fix                                                                                                             |
| ---------------------------------------------------------- | --------------------------------------------------------------------------------------------------------------- |
| `LOCKFILE_MISSING`, `INSTALL_FAILED`                       | commit a lockfile that matches `package.json`                                                                   |
| `EXPORT_NOT_FOUND`, `EXPORT_INVALID`, `EXPORT_SIDE_EFFECT` | export a builder from the entry; render only in `render.ts`                                                     |
| `LFS_POINTER_UNRESOLVED`, `ASSET_NOT_FOUND`                | push the LFS object, or widen `source.lfs.include`                                                              |
| `REPO_NOT_ALLOWED`, `GIT_AUTH_FAILED`, `REPO_NOT_FOUND`    | the worker’s allowlist or git token doesn’t cover the repo                                                      |
| `REF_NOT_FOUND`, `REF_AMBIGUOUS`                           | push the branch, or pass a SHA                                                                                  |
| `ENGINE_SWITCH` (retryable)                                | the worker loaded another framefields version and restarts. Keep projects on one version per worker to avoid it |
| `GPU_UNAVAILABLE` (retryable)                              | the host’s GPU isn’t usable for WebGPU. Another worker takes the retry                                          |
| `RENDER_TIMEOUT`                                           | raise `timeoutSec`, or the endpoint’s execution timeout on RunPod                                               |

## Hardware

[Section titled “Hardware”](#hardware)

* **Drawing frames.** Frames are drawn with WebGPU on Vulkan. The `FF_GPU` setting: `auto` uses the GPU, or the CPU (lavapipe, much slower) when there is no GPU; `require` refuses to run without a GPU (used by the RunPod templates); `cpu` always draws on the CPU.
* **Vision.** Vision models run on the same GPU through WebGPU. The log line `[vision] Inference on the GPU (WebGPU)` confirms it.
* **Video encoding.** framefields encodes with NVENC when it works and falls back to libx264/libx265/libvpx/SVT-AV1 otherwise. NVENC needs NVIDIA driver 610 or newer. A100 and H100 have no NVENC.
* **Picking a GPU.** RTX/L4/L40S-class cards suit rendering. AI-training GPUs cost more and encode on the CPU.

# Rendering

> Lint, single frames, contact-sheet grids, pixel tests and MP4 export with built-in QA — in the order that wastes the least time.

## The ladder

[Section titled “The ladder”](#the-ladder)

| Step | Call                                                                              | Cost                |
| ---- | --------------------------------------------------------------------------------- | ------------------- |
| 1    | `tsc --noEmit`, `comp.lint()`, `CompositorProgramSchema.safeParse(comp.toSpec())` | no GPU, instant     |
| 2    | `comp.renderFrame({ frame })`                                                     | one frame           |
| 3    | `comp.renderFrameGrid({ frames })`                                                | a handful of frames |
| 4    | Vitest pixel probes                                                               | per test            |
| 5    | `comp.renderVideo({ outputPath, qa: true })`                                      | the whole film      |

## 1. Check without rendering

[Section titled “1. Check without rendering”](#1-check-without-rendering)

```ts
import { CompositorProgramSchema } from "framefields";


const parsed = CompositorProgramSchema.safeParse(comp.toSpec());
if (!parsed.success) throw new Error(JSON.stringify(parsed.error.format()));


for (const d of comp.lint()) console.log(`${d.severity} ${d.code}: ${d.message}${d.layerId ? ` (${d.layerId})` : ""}`);
```

`lint()` finds what renders without error but not as intended: unparseable colors and gradients, low-contrast text, off-canvas layers, truncated animations, PBR props without an environment. Codes are in [QA codes](/docs/reference/qa-codes/).

## 2. One frame

[Section titled “2. One frame”](#2-one-frame)

```ts
import fs from "node:fs/promises";
import { HeadlessMediaRenderer } from "framefields";


const renderer = new HeadlessMediaRenderer();               // reuse across frames
const png = await comp.renderFrame({ frame: 45, renderer }); // PNG Buffer
await fs.writeFile("output/f0045.png", png);
```

Render the key moments — entrance, peak, exit — and look at them.

## 3. Contact sheet

[Section titled “3. Contact sheet”](#3-contact-sheet)

```ts
const grid = await comp.renderFrameGrid({
  frames: [0, 6, 12, 18, 24, 30, 36, 42], // or startFrame / endFrame / stepFrames, or timesMs
  columns: 4,
  cellWidth: 480,                         // 120–960, default 320
  showLabels: true,                       // frame + time badge per cell (default)
  outputPath: "output/intro-grid.png",    // optional: also write it
});
```

A grid shows easing hitches, wrap jumps, wrong z-order and missing cuts in one image. Default cap is 64 frames (`maxFrames`, hard max 120). Not for vision layers — see [Vision](/docs/guides/vision/#timing-the-one-frame-delay).

## 4. Pixel tests

[Section titled “4. Pixel tests”](#4-pixel-tests)

src/film.test.ts

```ts
import { describe, expect, it } from "vitest";
import { HeadlessMediaRenderer } from "framefields";
import { buildFilm } from "./film.js";


describe("hero", () => {
  it("draws the title by frame 30", async () => {
    const comp = await buildFilm();
    const png = await comp.renderFrame({ frame: 30, renderer: new HeadlessMediaRenderer() });
    expect(png.length).toBeGreaterThan(1000);
    // decode with skia-canvas and sample pixels at known coordinates
  });
});
```

## 5. Export

[Section titled “5. Export”](#5-export)

```ts
import { formatQaReport } from "framefields";


const result = await comp.renderVideo({
  outputPath: "output/film.mp4",
  quality: "high",       // "very_low" | "low" | "high" | "very_high" (default)
  codec: "h264",         // default, MP4; "vp9" | "vp8" → WebM
  concurrency: 4,
  qa: { targetLufs: -14 }, // or true
});


console.log(formatQaReport(result.qa!));
if (!result.qa!.passed) process.exitCode = 1;
process.exit();
```

| Option                | Notes                                                                                   |
| --------------------- | --------------------------------------------------------------------------------------- |
| `outputPath`          | written there; `result.filePath` is the final path                                      |
| `quality`             | bitrate preset                                                                          |
| `codec`, `audioCodec` | `h264` → MP4 + AAC; `vp9`/`vp8` → WebM + Opus; `aac`/`mp3`/`opus` alone → audio-only    |
| `concurrency`         | frames rendered in parallel                                                             |
| `ringCapacity`        | GPU→encoder staging depth, 2–8 (or `FRAMEFIELDS_RING_CAPACITY`)                         |
| `qa`                  | measure frames and audio as they encode; returns `result.qa`                            |
| `renderer`            | share a `HeadlessMediaRenderer`                                                         |
| `onProgress`          | `({ phase, frame, totalFrames }) => void`; `phase` is `"render"`, then `"audio"` (2.2+) |
| `signal`              | an `AbortSignal`; stops between frames, rejects with its reason, leaves no file (2.2+)  |

QA thresholds: `minBlackSec` (0.5), `minFrozenSec` (2), `minSilenceSec` (2), `silenceDbfs` (−60), `targetLufs`, `lufsTolerance` (1). `qa.passed` is false on any error or warning. `qa.audio.integratedLufs` is BS.1770 loudness (streaming normalizes to about −14).

Always exit

Native GPU and encoder threads keep Node alive. End every render script with `process.exit()`.

## Inspect a file from elsewhere

[Section titled “Inspect a file from elsewhere”](#inspect-a-file-from-elsewhere)

```bash
ffprobe -v error -show_entries stream=codec_name,width,height,duration,r_frame_rate output/film.mp4
ffmpeg -i output/film.mp4 -vf "blackdetect=d=0.5:pix_th=0.10" -f null -
ffmpeg -i output/film.mp4 -af volumedetect -f null -
```

## Encoders and GPU

[Section titled “Encoders and GPU”](#encoders-and-gpu)

* **Video encoder.** framefields 2.2 tests the hardware encoder (NVENC, VideoToolbox) once per codec and uses it only if it works. Otherwise it encodes with libx264, libx265, libvpx or SVT-AV1. The log says which one it picked (`[video-encoders]`). Set `FRAMEFIELDS_HW_CODECS=off` to always encode in software.
* **Adapter.** `await describeGpu()` returns `{ vendor, architecture, device, description, software }`. `software: true` means frames are drawn on the CPU (lavapipe, SwiftShader), which is much slower.

## Rendering without a GPU

[Section titled “Rendering without a GPU”](#rendering-without-a-gpu)

Without a GPU (such as in headless CI runners, Docker containers, or cloud sandboxes), framefields renders on the CPU via software rasterization. The engine detects this automatically, and `await describeGpu()` reports `software: true`:

| Platform | CPU renderer                   | How to install / enable                               |
| -------- | ------------------------------ | ----------------------------------------------------- |
| Linux    | Mesa lavapipe (Vulkan)         | `apt-get install -y libvulkan1 mesa-vulkan-drivers`   |
| Windows  | WARP (Direct3D 12) or lavapipe | Built into Windows (WARP); or via Vulkan Mesa drivers |
| macOS    | Not needed                     | Every Mac and macOS VM provides a native Metal GPU    |

On Linux with lavapipe, the output is pixel-accurate and matches a GPU render across every feature: layout, typography, SVG paths, Lottie, media, 2D VFX shaders, 3D scenes with shadows/bloom, and on-device vision.

### How slow is it?

[Section titled “How slow is it?”](#how-slow-is-it)

Rendering on CPU is functional but significantly slower than a hardware GPU:

* **Measured benchmark:** In test suite benchmarks (three 720p test films with 3D elements and three vision layers), rendering on CPU via lavapipe took **9.7 minutes** compared to **49 seconds** on an Apple M1 Pro GPU (\~**12× slower**).
* **Desktop GPU comparison:** Compared to modern discrete GPUs (e.g. NVIDIA RTX 3070/4090 or L4), CPU software rendering is typically **10× to 100× slower**, depending on shader complexity, screen resolution, and scene composition.
* **CPU scaling:** Performance scales directly with CPU core count. Mesa lavapipe multithreads fragment rasterization across available CPU cores, and software video encoding (`libx264`) uses thread pools.

### Subsystem fallbacks on CPU

[Section titled “Subsystem fallbacks on CPU”](#subsystem-fallbacks-on-cpu)

When running without a hardware GPU, framefields optimizes each subsystem to avoid unnecessary bottlenecks:

* **WebGPU pipelines (Dawn):** Dawn binds to Mesa’s Vulkan software ICD (`lvp_icd.*.json`). All WGSL shaders, blend passes, and 3D pipelines execute on CPU worker threads.
* **Vision inference:** When `describeGpu()` detects a software adapter (`software: true`), the vision subsystem automatically routes ONNX inference to the native CPU provider (`onnxruntime-node`). This avoids the extreme performance penalty of executing deep neural networks through a software GPU rasterizer.
* **Video encoding:** The renderer automatically falls back from hardware encoders (NVENC, VideoToolbox) to multi-threaded CPU software encoders (`libx264`, `libx265`, `libvpx`, `svtav1`).

### Recommended workflow on CPU

[Section titled “Recommended workflow on CPU”](#recommended-workflow-on-cpu)

Because full video rendering on CPU takes minutes instead of seconds:

* **Use the ladder:** Always run `comp.lint()`, then verify with single frames (`renderFrame`) and contact sheet grids (`renderFrameGrid`). Catch layout and visual defects in seconds before committing to a full export.
* **For production films:** Keep CPU rendering for headless testing, CI sanity checks, and single-frame probes. Offload full-length exports and batch renders to a GPU machine or the [render worker](/docs/guides/render-service/).

## Production rendering

[Section titled “Production rendering”](#production-rendering)

Measured worker memory, vision included: about 0.5 GB on an Apple Silicon Mac, 2–3 GB on Linux with an NVIDIA GPU, where driver memory counts too. To render projects from git on GPU machines, with progress, cancel and uploads to S3, use the [render worker](/docs/guides/render-service/). In your own long-lived process, call `resetRenderCaches()` between jobs: the next job may have different files at the same paths.

# Scenes & transitions

> Split a film into scene functions on a beat grid and join them with Layer.sequence and Transition.

## One function per scene

[Section titled “One function per scene”](#one-function-per-scene)

Each scene returns a full-frame box. Children key from the scene’s own frame 0.

src/scenes/hero.ts

```ts
import { Layer, LayerAnimation } from "framefields";
import { H, PAPER, SANS, W } from "../theme.js";


export function hero() {
  return Layer.box({
    id: "hero",
    position: "absolute", x: 0, y: 0, width: W, height: H,
    background: "#000",
    children: [
      Layer.text("Every frame is a function of its number.", {
        position: "absolute", x: 140, y: 420, width: W - 280,
        fontFamily: SANS, fontSize: 120, fontWeight: 800, fill: PAPER,
      }).animate(LayerAnimation.create().fadeIn(0, 18)),
    ],
  });
}
```

## Join them: `Layer.sequence`

[Section titled “Join them: Layer.sequence”](#join-them-layersequence)

`Layer.sequence` places scenes end to end, overlaps each pair by its transition, and sets every scene’s `startFrame` / `durationFrames` for you.

src/film.ts

```ts
import { Composition, Layer, Transition } from "framefields";
import { beats, FPS, H, W } from "./theme.js";
import { features, hero, outro, proof } from "./scenes/index.js";


const film = Layer.sequence(
  [
    { scene: hero(), duration: beats(8) },
    { scene: features(), duration: beats(12), transition: Transition.push("left", 18) },
    { scene: proof(), duration: beats(8), transition: Transition.wipe("right", 16) },
    { scene: outro(), duration: beats(6), transition: Transition.iris(20) },
  ],
  { width: W, height: H },
);


const comp = new Composition({
  width: W, height: H, fps: FPS,
  durationMs: (film.totalFrames / FPS) * 1000, // the sequence knows its length
});
comp.add(film);
```

`duration` includes the transition. A missing `transition` is a cut.

## Transitions

[Section titled “Transitions”](#transitions)

| Call                                    | Look                              | Defaults                   |
| --------------------------------------- | --------------------------------- | -------------------------- |
| `Transition.cut()`                      | hard cut                          | —                          |
| `Transition.crossfade(frames, ease)`    | new scene fades in over the old   | 15, `sine.inOut`           |
| `Transition.fadeThrough(frames, ease)`  | dip to background and back        | 20, `sine.inOut`           |
| `Transition.push(dir, frames, ease)`    | new pushes old out                | `left`, 18, `power3.inOut` |
| `Transition.slide(dir, frames, ease)`   | new slides over old, which dims   | `left`, 18, `power3.out`   |
| `Transition.zoom(frames, ease)`         | old grows and fades, new settles  | 18, `power2.inOut`         |
| `Transition.whip(dir, frames)`          | fast motion-blurred push          | `left`, 10                 |
| `Transition.wipe(dir, frames, ease)`    | hard edge sweeps across (matte)   | `left`, 18, `power2.inOut` |
| `Transition.iris(frames, { at, ease })` | circle opens from a point (matte) | 20, centre                 |

`dir` is `"left" | "right" | "up" | "down"`. `at` is `[x, y]` as fractions of the frame.

## Carry elements across cuts

[Section titled “Carry elements across cuts”](#carry-elements-across-cuts)

A badge, card or cursor that crosses a scene boundary with continuous position and velocity hides the cut. Put the carrier outside the sequence, on the composition, with its own keyframes spanning both scenes.

# Vision

> On-device detection, segmentation, pose and mattes as reactive signals — pin layers to subjects, cut them out, reframe, blur, and analyze clips.

framefields runs ONNX vision models on the rendered frames and exposes the results as signals that layers and effects consume like any other. All models are Apache-2.0, pinned to an immutable revision, verified by SHA-256, and downloaded lazily on first use.

## Turn it on

[Section titled “Turn it on”](#turn-it-on)

```ts
// Whole composition → a reactive bundle
const vision = comp.withVision({
  enableDetection: true,      // default
  enableSegmentation: true,
  enablePose: true,
  variant: "s",               // "t" | "s" | "m"
  confidence: 0.35,           // default 0.3
  classes: ["person"],        // COCO filter
});


// One layer → render it through a mode
Layer.video("assets/footage/dancer.mp4").withVision({ mode: "matte", enableSegmentation: true });
```

| Option               | Model                      | Gives you                                                           |
| -------------------- | -------------------------- | ------------------------------------------------------------------- |
| `enableDetection`    | RTMDet-Ins                 | COCO-80 boxes, tracked over time → `vision.objects`                 |
| `enableSegmentation` | RTMDet-Ins (same pass)     | soft per-instance masks → `vision.masks`, `vision.segmentation`     |
| `enablePose`         | RTMO                       | 17 COCO keypoints per person → `track.pose`, `vision.poseLandmarks` |
| `enableMatte`        | MediaPipe Selfie Segmenter | fast person alpha → `vision.segmentation.matte`                     |

Layer modes: `passthrough`, `mask`, `matte`, `crop`, `skeleton`, `boxes`, `tracking`. For cutouts, `matteSource: "instance"` (default; any class, merges attached parts like a dress or instrument) or `"selfie"` (people filling the frame only, fastest).

`classes` picks which instance is the subject, not which parts merge into it. COCO has no clothing class, so a flowing dress may come back as a “kite” or “umbrella” and still joins a `classes: ["person"]` cutout (2.2+).

## On-demand clip vision (`comp.vision`)

[Section titled “On-demand clip vision (comp.vision)”](#on-demand-clip-vision-compvision)

For physics, particle deflection, and interactive layout, `comp.vision(source, options)` analyzes the raw clip frames **on demand** before drawing a frame. Unlike render-time `withVision` (which analyzes a layer’s previous rendered frame with a one-frame delay), clip vision decodes the source directly and makes poses and silhouettes available synchronously to the current frame:

```ts
const dancer = comp.vision("assets/footage/dancer.mp4", {
  pose: true,       // 17 COCO keypoints (default true)
  mask: true,       // soft silhouette including dress/held objects (default true)
  maskCell: 8,      // grid cell size in source px (default 8)
  smoothing: 2,     // keypoint temporal averaging in frames (default 2)
});


// Map into composition coordinates
const person = dancer.person.place({ width: 1920, height: 1080, fit: "cover" });


// 1. Read coordinates at any frame
const hand = person.point("rightWrist", 45); // { x, y, visibility }


// 2. Reactive signals for layers
const badge = Layer.box({ width: 80, height: 80, background: "#ff5a1f" });
badge.animate(LayerAnimation.create()
  .signal("x", person.signals("nose").x)
  .signal("y", person.signals("nose").y));


// 3. Silhouette alpha & gradient (for physics deflection & particles)
const sample = person.sample(45, 960, 540); // { alpha, gx, gy }
```

### Caching to disk

[Section titled “Caching to disk”](#caching-to-disk)

Results are cached on disk in `~/.cache/framefields/vision` (or `$FRAMEFIELDS_VISION_CACHE_DIR`) keyed by file size, modification time, and options. A second run, or a final render after a browser preview, loads instantly from cache without re-running model inference.

### Integration with physics and particles

[Section titled “Integration with physics and particles”](#integration-with-physics-and-particles)

* **Live physics worlds:** When used with `world.live(comp)`, the simulation advances frame-by-frame alongside vision preparation, so preview starts immediately without waiting to analyze the whole video first.
* **GPU particles:** Pass `push: { person, strength, sweep }` to `Layer.particles` to have the performer push particles away from their moving silhouette.
* **Initial layout:** Call `await dancer.ready(0)` up front if you need the initial pose at frame 0 to position elements before the animation starts.

## Pin, drive, cut out

[Section titled “Pin, drive, cut out”](#pin-drive-cut-out)

```ts
// Pin a label to the main subject
Layer.text("SUBJECT 01", { fontSize: 40, fill: "#f8fafc" }).pinToObject(vision.objects.primary, {
  anchor: "topCenter", offsetY: -48, smoothFrames: 5, hideWhenLost: true,
});


// Drive an effect from mask coverage
Layer.box({ width: 1920, height: 1080, background: "#000" })
  .withEffect(new Vignette({ strength: vision.segmentation.subject.coverage, radius: 0.9 }));


// Editorial one-liners
comp.addSubjectSandwich({ source: "assets/footage/dancer.mp4", behind: [headline], feather: 4, fit: "cover" }); // text behind subject
comp.addSmartFraming({ source: "assets/footage/action.mp4", target: vision.objects.primary, targetAspect: 9 / 16, damping: 0.15 });
comp.addSubjectOutline(vision.segmentation.subject, { source: "assets/footage/character.mp4", color: "#ff5a1f", width: 6, blur: 12 });
layer.blurRegion(vision.objects.byCategory("person"), { strength: 30 });                               // anonymize
```

Anchors: `center`, `topCenter`, `bottomCenter`, `topLeft`, `topRight`, `bottomLeft`, `bottomRight`, `centerLeft`, `centerRight`. `pinToLandmark(coord, { offsetX, offsetY })` pins to a pose keypoint.

## Signals you get

[Section titled “Signals you get”](#signals-you-get)

| Group                  | Highlights                                                                   |
| ---------------------- | ---------------------------------------------------------------------------- |
| `objects`              | `primary`, `get(trackId)`, `byCategory(cat, rank)`, `count`, `hasCategory`   |
| `objects.*.bounds`     | `x`, `y`, `width`, `height`, `screenX/Y/Width/Height`, `area`, `aspectRatio` |
| `objects.*.kinematics` | `vx`, `vy`, `speed`, `acceleration`, `headingDeg`                            |
| `objects.*.pose`       | 17 keypoints, `hasPose`, `wristSpeed`, `handRaised`, `bodyTiltAngle`         |
| `masks`                | `subject`, `get(trackId)`, `count`; per mask `area`, `coverage`, `solidity`  |
| `segmentation`         | `subject`, `humanSilhouette`, `instanceMasks`, `matte.coverage`              |
| `classes`              | per class `count`, `maxConfidence`, `present`                                |

`vision.summary(frame)` returns a deterministic, serializable snapshot — safe inside a frame hook.

## Analyze before authoring

[Section titled “Analyze before authoring”](#analyze-before-authoring)

```ts
const report = await comp.analyzeVisionSequence("assets/footage/street.mp4", { tasks: ["detect", "pose"], categories: ["person"] });
report.tracks; // [{ trackId, category, frames: [start, end], centerPath, … }]
```

A zod-serializable report of what is in the clip, without ffmpeg — use it to decide where to place things.

## Timing: the one-frame delay

[Section titled “Timing: the one-frame delay”](#timing-the-one-frame-delay)

Vision reads each layer’s **previous** rendered frame. Frame 0 has no results, and masks trail the plate by one frame (invisible at speed).

* `renderVideo` renders in order and handles this.
* For stills, render at least two **consecutive** frames with the same `HeadlessMediaRenderer` and keep the last.
* **Don’t verify vision with `renderFrameGrid`**: jumping between frames cuts out pixels from another frame and shows a ghosted double.

## GPU or CPU

[Section titled “GPU or CPU”](#gpu-or-cpu)

On framefields 2.2+, models run on the GPU through WebGPU (`onnxruntime-web`, installed with framefields), on the same device that draws the frames, so frames never leave the GPU. They fall back to the CPU (`onnxruntime-node`) when the adapter is a software one or WebGPU fails. The log tells you which:

```plaintext
{"backend":"webgpu","sharedDevice":true,"msg":"[vision] Inference on the GPU (WebGPU)"}
```

| Env                           | Effect                                                                                                                                          |
| ----------------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------- |
| `FRAMEFIELDS_VISION_PROVIDER` | `auto` (default), `webgpu`, `cpu`, or `cuda` (Linux x64; needs CUDA 12 and cuDNN 9). A forced backend that fails throws instead of falling back |
| `FRAMEFIELDS_VISION_THREADS`  | CPU threads per model (default on Linux: the CPUs the process may use)                                                                          |

Measured on the GPU, three vision layers, 60 frames at 1280×720: about 17 s on an M1 Pro, about 7 s on an RTX 3070.

## Choosing settings

[Section titled “Choosing settings”](#choosing-settings)

| Task (on the CPU, per \~2K frame) | `t`        | `s`        | `m`        |
| --------------------------------- | ---------- | ---------- | ---------- |
| detect + segment (one pass)       | 150–300 ms | 200–340 ms | 420–700 ms |
| pose                              | \~50 ms    | \~120 ms   | \~280 ms   |
| matte                             | \~20 ms    | \~20 ms    | \~20 ms    |

* Start with `"s"`; use `"m"` for fast motion or busy backgrounds.
* Raise `confidence` to \~0.5 on stylized footage; restrict `classes` to stop the subject switching.
* Garment drops out of a cutout → `variant: "m"` or `keyBackground: true`. Halo on dark backgrounds → lower `featherRadius` or raise `maskThreshold`.

## Models offline

[Section titled “Models offline”](#models-offline)

Models cache in `$FRAMEFIELDS_MODELS_DIR` (default `~/.cache/framefields/models`). For CI or air-gapped renders, preload once or point `FRAMEFIELDS_MODELS_BASE_URL` at a mirror.

```ts
import { VisionRunner } from "@framefields/vision";


const runner = VisionRunner.create({ variant: "s" }); // zero I/O until used
await runner.preload(["detect", "pose"]);             // downloads + verifies SHA-256 (~380 MB for all models)
runner.close();
```

# API index

> Every public entry point of the framefields package on one page — what it is and where it is documented.

Everything imports from `"framefields"`. Subpaths exist for smaller imports: `framefields/effects`, `framefields/audio`, `framefields/signals`, `framefields/renderer`, `framefields/fonts`, `framefields/preview`. Vision runs standalone from `@framefields/vision` (and `@framefields/vision/web` in browsers).

## Composition

[Section titled “Composition”](#composition)

`new Composition({ width, height, fps = 60, durationMs | durationFrames, backgroundColor = "#000000", fonts, antialias3d = true })`

| Member                                                            | Returns                                  | Guide                                                             |
| ----------------------------------------------------------------- | ---------------------------------------- | ----------------------------------------------------------------- |
| `add(layer \| layer[], …)`                                        | `this`                                   | [Core concepts](/docs/core-concepts/)                             |
| `addAudio(audioLayer)`                                            | `this`                                   | [Audio](/docs/guides/audio-and-signals/)                          |
| `addSignal(name, signal)`                                         | `this`                                   | [Signals](/docs/guides/audio-and-signals/#shared-signals-by-name) |
| `apply(effect)`                                                   | `this`                                   | [Effects](/docs/guides/effects/)                                  |
| `section({ source, x, y, width, height, borderRadius, effects })` | node                                     | [Effects](/docs/guides/effects/#where-to-attach)                  |
| `withVision(options)`                                             | vision bundle                            | [Vision](/docs/guides/vision/)                                    |
| `vision(source, options)`                                         | on-demand clip vision                    | [Vision](/docs/guides/vision/#on-demand-clip-vision-compvision)   |
| `onPrepareFrame(cb)`                                              | async per-frame hook                     | [Physics](/docs/guides/physics/#live-worlds--performers)          |
| `addSubjectSandwich` / `addSmartFraming` / `addSubjectOutline`    | `this`                                   | [Vision](/docs/guides/vision/#pin-drive-cut-out)                  |
| `analyzeVisionSequence(src, { tasks, categories })`               | report                                   | [Vision](/docs/guides/vision/#analyze-before-authoring)           |
| `registerFont(…)` / `registerEmojiFont(src)`                      | `this`                                   | [Text](/docs/guides/layout-and-text/#fonts)                       |
| `computeDuration()`                                               | ms                                       | —                                                                 |
| `lint()`                                                          | `{ severity, code, message, layerId }[]` | [QA codes](/docs/reference/qa-codes/)                             |
| `toSpec()`                                                        | program JSON                             | [Rendering](/docs/guides/rendering/#1-check-without-rendering)    |
| `renderFrame({ frame, renderer })`                                | PNG `Buffer`                             | [Rendering](/docs/guides/rendering/)                              |
| `renderFrameGrid(options)`                                        | PNG `Buffer`                             | [Rendering](/docs/guides/rendering/#3-contact-sheet)              |
| `renderVideo(options)`                                            | `{ filePath, cleanup, qa? }`             | [Rendering](/docs/guides/rendering/#5-export)                     |
| `onRequestFrame(ctx => …)`                                        | `this`                                   | per-frame hook                                                    |

## Layer builders

[Section titled “Layer builders”](#layer-builders)

| Builder                                                                                       | Kind                                                      |
| --------------------------------------------------------------------------------------------- | --------------------------------------------------------- |
| `Layer.box(opts)`, `Layer.flex(opts)`                                                         | containers                                                |
| `Layer.text(content \| spans, opts)`                                                          | GPU text                                                  |
| `Layer.shape(type, opts)`                                                                     | `rect` `circle` `ellipse` `polygon` `star` `arrow` `path` |
| `Layer.video` `image` `svg` `lottie` `audio` `caption` `(src, opts)`                          | media                                                     |
| `Layer.chart(options, boxProps)`                                                              | charts                                                    |
| `Layer.particles(opts)`                                                                       | GPU compute particles                                     |
| `Layer.sequence(entries, opts)`                                                               | scenes + transitions; has `totalFrames`                   |
| `Layer.section(source, target?, opts)`, `Layer.trackedRegion`, `Layer.blurTrackedRegion`      | regions                                                   |
| `Layer.camera(opts)`                                                                          | 3D camera                                                 |
| `Layer.environment(preset, opts)`                                                             | PBR world                                                 |
| `Layer.light`, `ambientLight`, `directionalLight`, `pointLight`, `spotLight`                  | lights                                                    |
| `Layer.model(opts)`, `glb` `gltf` `obj` `fbx` `stl` `ply` `vox` `threeDS` `off` `(src, opts)` | models                                                    |
| `Layer.cube` `carousel3d` `plane3d` `prism3d`                                                 | 3D layout (also on `Layer3D`)                             |

## Node methods

[Section titled “Node methods”](#node-methods)

Every builder’s result has:

| Method                                                                                   | Does                                         |
| ---------------------------------------------------------------------------------------- | -------------------------------------------- |
| `.animate(...animations)`                                                                | add keyframes/drivers (merges with existing) |
| `.withEffect(fx)` / `.withEffects([...])` / `.apply(fx)`                                 | effects                                      |
| `.withMatte(layer, mode)` / `.withMask(layer, mode)`                                     | track mattes                                 |
| `.withCrop(crop)`                                                                        | crop                                         |
| `.withVision(opts)`                                                                      | per-layer vision mode                        |
| `.pinToObject(track, opts)` / `.pinToLandmark(coord, opts)`                              | follow vision                                |
| `.blurRegion(track, opts)`                                                               | blur a tracked object                        |
| `.relight(opts)`, `.withDeflicker(opts)`, `.withPbrGlass(opts)`, `.deformWithAudio(cfg)` | specialised passes                           |

## Physics & particles

[Section titled “Physics & particles”](#physics--particles)

| Export               | Members / Notes                                                                                                                       | Guide                                                           |
| -------------------- | ------------------------------------------------------------------------------------------------------------------------------------- | --------------------------------------------------------------- |
| `Physics`            | `world2d(opts)`, `world3d(opts)`                                                                                                      | [Physics](/docs/guides/physics/)                                |
| `PhysicsWorld2D`     | `body`, `ground`, `walls`, `pin`, `hinge`, `weld`, `spring`, `rope`, `force`, `field`, `impulse`, `simulate`, `bakeKeyframes`, `live` | [Physics](/docs/guides/physics/#2d-rigid-bodies)                |
| `PhysicsWorld3D`     | `body`, `ground`, `walls`, `pin`, `hinge`, `weld`, `spring`, `rope`, `force`, `field`, `impulse`, `simulate`, `bakeKeyframes`, `live` | [Physics](/docs/guides/physics/#3d-rigid-bodies)                |
| `PhysicsBody2D / 3D` | `x`, `y`, `z`, `rotation`, `speed`, `vx`, `vy`, `vz`, `impact`, `collisions`, `center`, `localAnchor`                                 | [Physics](/docs/guides/physics/#signals--reactions)             |
| `Layer.particles`    | GPU compute-shader particles (emitters, noise, attractors, performer push)                                                            | [Particles](/docs/guides/physics/#gpu-particles-layerparticles) |

## 3D

[Section titled “3D”](#3d)

| Export                     | Members                                                                                                                 |
| -------------------------- | ----------------------------------------------------------------------------------------------------------------------- |
| `Layer3D`                  | `roundedBox` `sphere` `cylinder` `cube` `carousel` `prism` `plane` `grid` `text` `extrudedText` `model` + model loaders |
| `Light`                    | `ambient(color, intensity)` `directional` `point` `spot` `environment`                                                  |
| `Lighting`                 | `studio` `soft` `sunset` `dramatic` `(rigOptions)` → array of layers                                                    |
| `CameraAnimation.camera()` | `orbit` `dolly` `truck` `pedestal` `pan` `tilt` `roll` `rackFocus` `handheld`                                           |

## Animation

[Section titled “Animation”](#animation)

| Export           | Members                                                                                                                                                                                                                                  |
| ---------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `LayerAnimation` | `create()`, `fromTo`, `keys`, `fadeIn`, `fadeOut`, `slideInX`, `slideInY`, `rotateX/Y/Z`, `translateZ`, `perspective`, `letterSpacing`, `typewriter`, `kineticSweep`, `wiggle`, `spring`, `signal`, `colorSignal`, `followPath`, `merge` |
| `TextAnimator`   | `waveRise` `wordPop` `blurIn` `flip3D` `colorSweep`                                                                                                                                                                                      |
| `Transition`     | `cut` `crossfade` `fadeThrough` `push` `slide` `zoom` `whip` `wipe` `iris`                                                                                                                                                               |

## Signals

[Section titled “Signals”](#signals)

`Signal.builder`, `Signal.sine`, `Signal.cosine`, `Signal.fromArray`, `Signal.programmatic`, `Signal.state`, `Signal.computed`, `Signal.frame`, `Signal.time`, `Signal.progress`; also `sineSignal`, `cosineSignal`, `computed`.

## Text, color, fonts

[Section titled “Text, color, fonts”](#text-color-fonts)

| Export        | Members                                                                                                                       |
| ------------- | ----------------------------------------------------------------------------------------------------------------------------- |
| `FontManager` | `register({ family, source, weight, style })`, `registerEmojiFont`, `measure(text, opts)`, `get`, `has`, `getAll`, `clear`    |
| rich text     | `rich` tagged template, `span`, `bold`, `italic`, `color`, `mark`, `size`, `sub`, `sup`, `parseMarkup`                        |
| `Color`       | `parse` `hex` `rgb` `hsl` `oklch` `mix` `ramp` `steps` `lighten` `darken` `saturate` `rotate` `alpha` `contrast` `readableOn` |
| `Gradient`    | `linear(stops, angle)` `radial(stops, { at, circle, radius })` `conic(stops)`                                                 |

## Audio

[Section titled “Audio”](#audio)

`renderSfx(trigger, ctx)`, `mixSfxInto(bed, sfx[])`, `softLimit(bed, knee?, ceiling?)`, `encodeStereoWav(channels, { sampleRate })`, `mixAudioTracks(...)`.

## Rendering & media

[Section titled “Rendering & media”](#rendering--media)

| Export                                 | Notes                                                                            |
| -------------------------------------- | -------------------------------------------------------------------------------- |
| `HeadlessMediaRenderer`                | share one across `renderFrame` calls                                             |
| `Media.video` `image` `lottie` `audio` | single-clip chains with `apply`, `renderFrame`, `renderFrameGrid`, `renderVideo` |
| `startPreview(entry, options)`         | `{ url, close(), closed }` — [Preview](/docs/guides/preview/)                    |
| `formatQaReport(qa)`                   | human-readable QA                                                                |
| `CompositorProgramSchema`              | zod schema of `toSpec()`                                                         |

## Effects

[Section titled “Effects”](#effects)

54 classes, e.g. `Blur`, `Curves`, `FilmGrain`, `Vignette`, `ColorKey`, `ApplyLUT`, plus `Effect.<camelCase>(props)` factories. See the [Effects reference](/docs/reference/effects/).

# Effects reference

> All 54 effect classes grouped by family, with every prop name. Import from "framefields" or "framefields/effects".

```ts
import { Blur, Curves, FilmGrain } from "framefields";
// same classes: import { … } from "framefields/effects";
// factory form: Effect.blur({ strength: 9 })
```

Attach with `layer.withEffect(fx)`, `layer.withEffects([...])`, `comp.apply(fx)`, `comp.section({ effects })` or `Media.apply(fx)` — see [Effects](/docs/guides/effects/). Every prop is an `EffectProp<T>`: a plain value, a signal, or a function hook. Out-of-range values warn; `FRAMEFIELDS_STRICT_PROPS=1` makes them throw. Types, ranges and defaults for every prop are in the generated [effect catalog](https://github.com/gatewai-dev/framefields/blob/main/plugins/framefields/skills/framefields-effects/references/effects-catalog.md).

Scales

Many strengths are **0–100**: `Vignette.strength` (default 50), `FilmGrain.strength` (15), `Blur.strength` (5). `ColorBalance` shifts are −100…100. Normalized positions and region radii (`centerX`, `centerY`, `radius`) are 0–1.

## Grading

[Section titled “Grading”](#grading)

| Effect              | Props                                                                                                                                                 |
| ------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------- |
| `Curves`            | `curveType`, `master`, `red`, `green`, `blue`, `hueVsHue`, `hueVsSat`, `lumVsSat`, `satVsSat`                                                         |
| `Levels`            | `master`, `red`, `green`, `blue`                                                                                                                      |
| `ColorBalance`      | `shadows`, `midtones`, `highlights`, `preserveLuminosity`                                                                                             |
| `SelectiveColor`    | `method`, `reds`, `yellows`, `greens`, `cyans`, `blues`, `magentas`, `whites`, `neutrals`, `blacks`                                                   |
| `GradientMap`       | `stops`, `smooth`, `dither`, `opacity`                                                                                                                |
| `ShadowsHighlights` | `shadowAmount`, `shadowTonalWidth`, `shadowRadius`, `highlightAmount`, `highlightTonalWidth`, `highlightRadius`, `colorCorrection`, `midtoneContrast` |
| `ApplyLUT`          | `lutUrl`, `intensity`                                                                                                                                 |
| `ExtractLUT`        | `strategy`, `samplePoints`                                                                                                                            |
| `Modulate`          | `hue`, `brightness`, `contrast`, `exposure`, `saturation`, `sepia`                                                                                    |
| `ChannelSplitter`   | `colorSpace`                                                                                                                                          |
| `ChannelMerger`     | `colorSpace`, `defaultChannel4`                                                                                                                       |

## Texture & lens

[Section titled “Texture & lens”](#texture--lens)

| Effect              | Props                                                                                                                                                                   |
| ------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `FilmGrain`         | `strength`, `size`, `monochrome`, `animated`, `speed`, `shadows`, `midtones`, `highlights`                                                                              |
| `Vignette`          | `strength`, `radius`, `softness`, `roundness`, `centerX`, `centerY`                                                                                                     |
| `Blur`              | `blurType`, `strength`, `angle`, `sigmaColor`, `centerX`, `centerY`, `partialBlur`, `radius`, `radiusY`, `shape`, `track`                                               |
| `MotionBlur`        | `shutterAngle`, `samples`, `maxVelocity`, `shutterCurve`, `depthAware`                                                                                                  |
| `UnsharpMask`       | `amount`, `radius`, `threshold`                                                                                                                                         |
| `HighPass`          | `radius`, `contrastBoost`, `monochrome`                                                                                                                                 |
| `HalftoneScreen`    | `mode`, `dotShape`, `frequency`, `angle`, `contrast`, `dotColor`, `paperColor`, `smooth`, `invert`, `cyanAngle`, `magentaAngle`, `yellowAngle`, `blackAngle`, `opacity` |
| `TileOffset`        | `offsetX`, `offsetY`, `wrap`, `edgeMode`                                                                                                                                |
| `TemporalDeflicker` | `blendWeight`, `disocclusionThreshold`, `maxMotionPixels`, `scale`, `windowSize`                                                                                        |

## Distortion & framing

[Section titled “Distortion & framing”](#distortion--framing)

| Effect            | Props                                                                                                                                                                                                                                |
| ----------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ |
| `DisplacementMap` | `strengthX`, `strengthY`, `xChannel`, `yChannel`, `wrapMode`                                                                                                                                                                         |
| `Liquify`         | `operations`                                                                                                                                                                                                                         |
| `MeshWarp`        | `cols`, `rows`, `points`                                                                                                                                                                                                             |
| `CornerPin`       | `points`                                                                                                                                                                                                                             |
| `Rotate`          | `angle`, `fit`, `scale`                                                                                                                                                                                                              |
| `Flip`            | `horizontal`, `vertical`, `diagonal`, `mode`, `symmetry`                                                                                                                                                                             |
| `KenBurns`        | `keyframes`, `motionBlurSize`, `movementStyle`, `aspectRatio`                                                                                                                                                                        |
| `ResizerScaler`   | `aspectRatioPreset`, `resolutionPreset`, `targetWidth`, `targetHeight`, `fitMode`, `zoom`, `offsetX`, `offsetY`, `backgroundMode`, `backgroundColor`, `backgroundColor2`, `blurRadius`, `backgroundBrightness`, `anchorX`, `anchorY` |
| `Crop`            | `cropType`, `leftPercentage`, `topPercentage`, `widthPercentage`, `heightPercentage`, `roundness`, `pathPoints`, `mode`                                                                                                              |

## Temporal

[Section titled “Temporal”](#temporal)

| Effect     | Props                                                                           |
| ---------- | ------------------------------------------------------------------------------- |
| `Trails`   | `decay`, `mode`, `zoom`, `rotation`, `offsetX`, `offsetY`, `hueShift`, `warmup` |
| `Datamosh` | `keyframes`, `strength`, `bleed`, `warmup`                                      |

## 3D & light

[Section titled “3D & light”](#3d--light)

| Effect                 | Props                                                                                                                                                                                                                                                                 |
| ---------------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `Relight3D`            | `lightType`, `intensity`, `lightPosX`, `lightPosY`, `lightPosZ`, `lightRadius`, `spotConeAngle`, `specularRoughness`, `specularStrength`, `metallic`, `ambientIntensity`, `volumetricDensity`, `depthScale`, `depthInvert`, `lightColor`, `ambientColor`, `normalMap` |
| `SSAO`                 | `radius`, `bias`, `intensity`, `kernelSamples`, `blurRadius`                                                                                                                                                                                                          |
| `PBRGlass`             | `ior`, `roughness`, `transmission`, `dispersion`, `fresnelPower`, `tintColor`                                                                                                                                                                                         |
| `DepthOfField`         | `focusDistance`, `focalLength`, `aperture`, `maxBlur`, `quality`                                                                                                                                                                                                      |
| `RefractionCaustics3D` | `ior`, `dispersion`, `refractionScale`, `causticBrightness`, `causticScale`, `fluidRipples`, `rippleSpeed`, `roughness`, `specularIntensity`, `lightPosX`, `lightPosY`, `tintColor`, `tintStrength`, `depthInvert`, `depthScale`                                      |
| `CameraParallax3D`     | `motionPreset`, `panX`, `panY`, `dollyZ`, `fov`, `parallaxAmount`, `dofAperture`, `focusPlane`, `edgeDilation`, `depthInvert`                                                                                                                                         |

## Mattes & keying

[Section titled “Mattes & keying”](#mattes--keying)

| Effect       | Props                                                                                                                          |
| ------------ | ------------------------------------------------------------------------------------------------------------------------------ |
| `ColorKey`   | `keyColor`, `similarity`, `smoothness`, `spillSuppression`, `colorSpace`, `spillSuppressionType`                               |
| `RefineEdge` | `decontaminateAmount`, `radius`, `smooth`, `feather`, `shiftEdge`, `matteChannel`, `outputMode`                                |
| `MaskMath`   | `operation`, `radius`, `threshold`, `clampMin`, `clampMax`, `channelA`, `channelB`, `binarize`, `invertResult`, `outputFormat` |
| `PatchHeal`  | `patches`, `centerX`, `centerY`, `offsetX`, `offsetY`, `radius`, `sourceRadius`, `feather`, `opacity`, `mode`                  |

## Generators & style

[Section titled “Generators & style”](#generators--style)

| Effect           | Props                                                                                                                                                 |
| ---------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------- |
| `NoiseGenerator` | `noiseType`, `outputType`, `width`, `height`, `scale`, `octaves`, `persistence`, `lacunarity`, `speed`, `colorStart`, `colorEnd`, `durationMs`, `fps` |
| `ProceduralVFX`  | `effectType`, `outputType`, `width`, `height`, `density`, `scale`, `speed`, `intensity`, `seed`, `colorStart`, `colorEnd`, `durationMs`, `fps`        |
| `Paint`          | `width`, `height`, `maintainAspect`, `aspectRatio`, `backgroundColor`, `strokes`                                                                      |
| `LayerStyle`     | `dropShadow`, `innerShadow`, `outerGlow`, `innerGlow`, `stroke`, `bevelEmboss`, `colorOverlay`                                                        |

## Audio

[Section titled “Audio”](#audio)

| Effect                 | Props                                                                                                                                                                                  |
| ---------------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `AudioFade`            | `fadeInDuration`, `fadeOutDuration`, `fadeInCurve`, `fadeOutCurve`                                                                                                                     |
| `AudioCompressor`      | `threshold`, `ratio`, `attack`, `release`, `knee`, `makeupGain`                                                                                                                        |
| `AudioParametricEq`    | `type`, `frequency`, `gain`, `q`                                                                                                                                                       |
| `AudioReverb`          | `roomSize`, `damping`, `wet`, `dry`, `width`                                                                                                                                           |
| `AudioDelay`           | `delayTime`, `feedback`, `wet`, `dry`, `pingPong`                                                                                                                                      |
| `AudioNoiseGate`       | `threshold`, `attack`, `hold`, `release`, `range`                                                                                                                                      |
| `StereoPanning`        | `pan`                                                                                                                                                                                  |
| `AudioSignalExtractor` | `extractionMode`, `attackMs`, `releaseMs`, `sensitivity`, `noiseFloorDb`, `dynamicRangeDb`, `autoRange`, `smoothing`, `curve`, `beatThreshold`, `beatDecayMs`, `previewMode`, `nodeId` |

## Vision

[Section titled “Vision”](#vision)

| Effect   | Props                                                                                                                                                                                                                                                                   |
| -------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `Vision` | `enableDetection`, `enableSegmentation`, `enablePose`, `enableMatte`, `classes`, `confidence`, `variant`, `mode`, `matteSource`, `maxMissedFrames`, `modelsDir`, `baseUrl`, `maskThreshold`, `featherRadius`, `keyBackground`, `backgroundKeyThreshold`, `visionBundle` |

## Recipes

[Section titled “Recipes”](#recipes)

```ts
// Warm filmic grade for a whole film
comp.apply(new ColorBalance({ shadows: { cyanRed: 4, yellowBlue: -6 }, highlights: { cyanRed: 2, yellowBlue: -4 } }));
comp.apply(new Vignette({ strength: 30, radius: 0.9, softness: 0.65 }));
comp.apply(new FilmGrain({ strength: 12, size: 1.5, animated: true }));


// Greenscreen key
Layer.video("assets/footage/green.mp4").withEffect(new ColorKey({ keyColor: "#00ff00", similarity: 0.4, smoothness: 0.1, spillSuppression: 0.3 }));


// Print look
layer.withEffect(new HalftoneScreen({ mode: "CMYK", dotShape: "Circle", frequency: 30 }));


// Duotone
layer.withEffect(new GradientMap({ stops: [{ position: 0, color: "#0e0d0c" }, { position: 1, color: "#ff5a1f" }] }));


// Audio clean-up
Layer.audio("assets/audio/vo.wav", { durationFrames }).apply(new AudioNoiseGate({ threshold: -45 })).apply(new AudioCompressor({ threshold: -18, ratio: 3 }));
```

# Lint & QA codes

> Every code comp.lint() and renderVideo({ qa }) can report, its severity, and what to do about it.

`comp.lint()` returns document findings before rendering. `renderVideo({ qa })` returns the same document findings plus measurements of the encoded frames and audio, in `result.qa.issues`. Each issue is `{ severity, code, message, layerId?, startSec?, endSec? }`. `qa.passed` is false if any **error** or **warning** is present.

## Document checks (lint and QA)

[Section titled “Document checks (lint and QA)”](#document-checks-lint-and-qa)

| Code                       | Severity | Means                                                       | Fix                                                |
| -------------------------- | -------- | ----------------------------------------------------------- | -------------------------------------------------- |
| `invalid_gradient`         | error    | a `background`/`fill` gradient doesn’t parse; draws nothing | check the CSS syntax, ≤ 8 stops                    |
| `unsupported_image_format` | error    | `Layer.image` given `.json`/`.lottie`, video or audio       | use `Layer.lottie` / `Layer.video` / `Layer.audio` |
| `unknown_color`            | warning  | a color string that isn’t a color; draws transparent        | use hex, `rgb()`, `hsl()`, `oklch()` or a CSS name |
| `low_contrast_text`        | warning  | text below 3:1 against the solid color behind it            | use the suggested color, or `Color.readableOn(bg)` |
| `offscreen_layer`          | warning  | a static absolute layer entirely outside the canvas         | fix `x`/`y` or remove it                           |
| `animation_truncated`      | warning  | keyframes past the layer’s `durationFrames` never play      | extend the layer or move the keys                  |
| `shadow_unsupported_light` | warning  | `castShadow` on a point or ambient light                    | use a directional or spot light                    |
| `pbr_without_environment`  | info     | `roughness`/`metallic` set without `Layer.environment()`    | add an environment or a `Lighting` rig             |
| `multiple_shadow_lights`   | info     | several `castShadow` lights; only the first casts           | keep one                                           |
| `static_track`             | info     | every keyframe on a track holds the same value              | remove the track or fix the values                 |

## Render measurements (QA only)

[Section titled “Render measurements (QA only)”](#render-measurements-qa-only)

| Code                  | Severity | Means                                                                  |
| --------------------- | -------- | ---------------------------------------------------------------------- |
| `frames_missing`      | error    | fewer frames encoded than the composition has                          |
| `black_frames`        | warning  | ≥ `minBlackSec` (0.5 s) of black; has `startSec`/`endSec`              |
| `audio_clipping`      | warning  | samples at or beyond full scale                                        |
| `loudness_off_target` | warning  | integrated loudness more than `lufsTolerance` (1 LU) from `targetLufs` |
| `frozen_video`        | info     | picture unchanged ≥ `minFrozenSec` (2 s); fine for a held title        |
| `audio_silence`       | info     | ≥ `minSilenceSec` (2 s) below `silenceDbfs` (−60)                      |
| `audio_silent`        | info     | the whole track is silent                                              |

## Use in CI or an agent loop

[Section titled “Use in CI or an agent loop”](#use-in-ci-or-an-agent-loop)

```ts
const result = await comp.renderVideo({ outputPath: "output/film.mp4", qa: { targetLufs: -14 } });
const blocking = result.qa!.issues.filter((i) => i.severity !== "info");
for (const i of blocking) console.error(`${i.code}${i.layerId ? ` [${i.layerId}]` : ""}: ${i.message}`);
process.exit(result.qa!.passed ? 0 : 1);
```

# Troubleshooting

> Symptoms, causes and fixes for install errors, blank frames, missing audio, hanging scripts, vision artifacts and preview problems.

## Install

[Section titled “Install”](#install)

| Symptom                              | Cause                                              | Fix                                                                                                                     |
| ------------------------------------ | -------------------------------------------------- | ----------------------------------------------------------------------------------------------------------------------- |
| `Cannot find module '.../skia.node'` | `skia-canvas` install script was blocked           | approve it ([Getting started](/docs/getting-started/#2-write-packagejson-then-install)), then `npm rebuild skia-canvas` |
| Import fails immediately             | Node < 22, or not an ES module project             | upgrade Node; set `"type": "module"`                                                                                    |
| No adapter / GPU error               | no GPU and no Vulkan driver (VM, plain CI runner)  | on Linux, `apt-get install -y libvulkan1 mesa-vulkan-drivers` to render on the CPU (Windows has WARP built in)          |
| Install fails on the render worker   | lockfile missing or out of date, or a bun lockfile | commit a pnpm, npm or yarn lockfile that matches `package.json`                                                         |

## Picture

[Section titled “Picture”](#picture)

| Symptom                                 | Cause                                             | Fix                                                                    |
| --------------------------------------- | ------------------------------------------------- | ---------------------------------------------------------------------- |
| Whole frame black                       | font not registered, or `NaN` from a signal       | register every family before building text; keep signal outputs finite |
| Text missing or tofu                    | family name doesn’t match a registered font       | match `fontFamily` to the `family` you registered                      |
| Emoji missing                           | no emoji font                                     | `FontManager.registerEmojiFont(...)`                                   |
| Layer invisible                         | path wrong, opacity 0, off canvas, or LFS pointer | check `comp.lint()`; run `git lfs pull`                                |
| Text clipped                            | box narrower than the text                        | set `width` (or `width: "fill"`)                                       |
| Animation stops early                   | keys past `durationFrames`                        | extend the layer (`animation_truncated`)                               |
| Layer jumps to the corner when animated | animated `y` from 0 instead of its layout value   | animate to the placed value                                            |
| Fades look murky grey                   | custom shader with straight alpha                 | output premultiplied alpha, blend with `srcFactor: "one"`              |
| Metal/rough looks flat                  | no environment                                    | add `Layer.environment()` or `Lighting.studio()`                       |
| Effect does nothing                     | strength on a 0–1 scale where it’s 0–100          | check ranges in the [Effects reference](/docs/reference/effects/)      |

## Sound

[Section titled “Sound”](#sound)

| Symptom               | Cause                                  | Fix                                   |
| --------------------- | -------------------------------------- | ------------------------------------- |
| Audio cuts out early  | `Layer.audio` without `durationFrames` | set `durationFrames: totalFrames`     |
| `audio_clipping`      | mix too hot                            | lower volumes; `softLimit` SFX beds   |
| `loudness_off_target` | mix level                              | adjust volume toward the `targetLufs` |

## Process

[Section titled “Process”](#process)

| Symptom                                     | Cause                                                 | Fix                                                                                            |
| ------------------------------------------- | ----------------------------------------------------- | ---------------------------------------------------------------------------------------------- |
| Script never exits after rendering          | native GPU/encoder threads                            | `process.exit()` at the end                                                                    |
| Slow exports                                | too little parallelism or a shallow staging ring      | raise `concurrency`; try `ringCapacity: 4`                                                     |
| Very slow on a Linux server                 | frames drawn on the CPU                               | `describeGpu()` reports `software: true`: fix the GPU’s Vulkan driver (`vulkaninfo --summary`) |
| Encoder fails to open (`h264_rkmpp`, NVENC) | framefields < 2.2 picks an encoder without testing it | upgrade to 2.2, or set `FRAMEFIELDS_HW_CODECS=off`                                             |

## Vision

[Section titled “Vision”](#vision)

| Symptom                        | Cause                                      | Fix                                                                                                |
| ------------------------------ | ------------------------------------------ | -------------------------------------------------------------------------------------------------- |
| Cutout empty on a single still | vision reads the previous frame            | render two consecutive frames with one renderer, keep the last                                     |
| Ghosted double of the subject  | frames rendered out of order (grid, jumps) | verify with the MP4 or consecutive frames                                                          |
| Wrong object cut out           | subject choice                             | `classes: ["person"]`                                                                              |
| Garment drops out              | fast motion                                | `variant: "m"` or `keyBackground: true`                                                            |
| Halo on dark backgrounds       | feathering                                 | lower `featherRadius` or raise `maskThreshold`                                                     |
| Vision slow                    | inference on the CPU                       | look for `[vision] Inference on the GPU` in the log; needs framefields 2.2+ and a hardware adapter |
| Fails offline / in CI          | models not cached                          | preload into `$FRAMEFIELDS_MODELS_DIR` or set `FRAMEFIELDS_MODELS_BASE_URL`                        |

## Preview

[Section titled “Preview”](#preview)

| Symptom                          | Cause                              | Fix                                                                             |
| -------------------------------- | ---------------------------------- | ------------------------------------------------------------------------------- |
| Font or media 404 in the page    | path built from `process.cwd()`    | build paths from `import.meta.url`; pass `root` for folders outside the project |
| “No WebGPU”                      | browser without WebGPU             | current Chrome, Edge or Safari                                                  |
| Preview stopped                  | last tab closed (5 s), or replaced | run it again; same URL                                                          |
| Cutouts look wrong while playing | device skipped frames              | check paused, or in the export                                                  |

Render worker error codes: [Render on a server](/docs/guides/render-service/#errors).

Still stuck? Open an issue at [github.com/gatewai-dev/framefields](https://github.com/gatewai-dev/framefields/issues) or ask on [Discord](https://discord.gg/cbqMGGme5).