Generative AI (ai)

framefields includes built-in generative AI capabilities via the ai module. One API key unlocks every supported generative model with unified credit billing and direct WebGPU composition integration.

import { ai, Composition, Layer } from "framefields";

// Subscribe to a queue-based generation
const { data } = await ai.subscribe("seedream/5.0-pro", {
  input: {
    prompt: "An atmospheric 35mm film still of a cinematic neon street",
    size: "2K",
  },
});

console.log("Image URL:", data.images[0].url);

Setup & API Keys

You don’t need separate provider accounts or keys.

1. Sign in at framefields.dev/dashboard.
2. Top up credits on the Billing page with Dodo Payments.
3. Create an API key on the API Keys page.
4. Add the key to your project .env:

FRAMEFIELDS_API_KEY=ff_live_...

A project made with framefields init already has a .env.example (commit it) and ignores .env (never commit that). Its render.ts loads .env when the file exists, so renders and scripts pick up the key without extra setup. Without a key everything else still works, including in CI: only ai.* calls need it.

Or configure it in code:

import { ai } from "framefields";

ai.config({
  apiKey: process.env.FRAMEFIELDS_API_KEY,
});

Model Catalog

Explore available models in the Model Catalog, including text-to-image, image-to-video, voice synthesis, and upscaling models. Each model page lists its input and output fields.

Kind	Model IDs
Video (text, image, audio and video to video, with sound)	seedance/2.5, seedance/2.0, seedance/2.0-fast, seedance/2.0-mini
Images (generation, editing, batches, layer decomposition)	seedream/5.0-pro, seedream/5.0-flash, seedream/5.0-lite, seedream/4.5
Speech, music and sound effects (with word timestamps)	seed-audio/1.0

You can also fetch the catalog programmatically for agentic workflows:

curl https://api.framefields.dev/v1/models

API Methods

ai.subscribe (Recommended)

Submits a request to the async processing queue, waits for completion, and returns the final output. Ideal for video and high-resolution rendering models.

const { data } = await ai.subscribe("seedance/2.5", {
  input: {
    prompt: "A neon geometric sculpture rotating in a void, slow orbit",
    resolution: "1080p",
    ratio: "16:9",
    duration: 5,
  },
  pollInterval: 2000, // poll status every 2 s
});

console.log(data.video.url);

ai.run

Executes over a single HTTP connection and returns the output. Fine for images and audio; video models only run through the queue (ai.subscribe or ai.queue).

const { data } = await ai.run("seed-audio/1.0", {
  input: {
    text_prompt: "Warm analog synth bed, then a calm narrator says: Welcome to framefields.",
    audio_config: { format: "mp3", enable_subtitle: true },
  },
});

console.log(data.audio.url, data.subtitle); // subtitle has sentence and word timestamps

ai.stream

Streams generation events (tokens or partial image steps) over Server-Sent Events (SSE), for models that stream.

const stream = await ai.stream("flux/dev", {
  input: { prompt: "Cyberpunk cityscape" },
});

for await (const event of stream) {
  console.log("Progress:", event);
}

ai.storage.upload

Uploads local buffers or files to framefields storage through signed upload URLs and returns a CDN URL any model accepts as input (e.g. image-to-video or style transfer). Blobs inside input upload automatically.

import fs from "node:fs";

const file = fs.readFileSync("input.png");
const fileUrl = await ai.storage.upload(new Blob([file], { type: "image/png" }));

const { data } = await ai.subscribe("seedance/2.5", {
  input: {
    prompt: "Slow cinematic camera push forward",
    content: [{ type: "image_url", image_url: { url: fileUrl }, role: "first_frame" }],
  },
});

Using Generated Assets in Compositions

Download or stream generated assets directly into your framefields layer hierarchy:

import { ai, Composition, Layer } from "framefields";
import fs from "node:fs/promises";

// 1. Generate an image asset
const { data } = await ai.subscribe("seedream/5.0-pro", {
  input: { prompt: "Abstract glowing glass caustics, dark mode", size: "2K" },
});

// 2. Fetch image buffer
const res = await fetch(data.images[0].url);
const imageBuffer = Buffer.from(await res.arrayBuffer());
await fs.writeFile("assets/generated-bg.png", imageBuffer);

// 3. Compose in WebGPU layer
export default Composition.create({
  width: 1920,
  height: 1080,
  fps: 60,
  durationInFrames: 300,
  children: [
    Layer.image({
      source: "assets/generated-bg.png",
      width: 1920,
      height: 1080,
    }),
  ],
});

Credits & Error Handling

Credits are charged in micro-USD ($0.000001 resolution) based on model usage plus margin.

When credits are depleted, requests will return HTTP 402 Payment Required:

try {
  await ai.subscribe("seedream/5.0-pro", { input: { prompt: "..." } });
} catch (err: any) {
  if (err.status === 402) {
    console.error("Credits depleted. Top up at https://framefields.dev/dashboard/billing");
  }
}

HTTP Status	Error Type	Solution
401	invalid_api_key	Verify FRAMEFIELDS_API_KEY in .env
402	insufficient_credits	Add credits in Billing
402	key_limit_reached	Increase key monthly limit in API Keys
429	rate_limited	Respect retry-after backoff (60 submits/min, 600 status/min)