# Generative AI (ai)

> Generate images, videos, sound and assets directly within framefields compositions using the AI gateway.

`framefields` includes built-in generative AI capabilities via the `ai` module. One API key unlocks every supported generative model with unified credit billing and direct WebGPU composition integration.

```ts
import { ai, Composition, Layer } from "framefields";

// Subscribe to a queue-based generation
const { data } = await ai.subscribe("seedream/5.0-pro", {
  input: {
    prompt: "An atmospheric 35mm film still of a cinematic neon street",
    size: "2K",
  },
});

console.log("Image URL:", data.images[0].url);
```

## Setup & API Keys

You don't need separate provider accounts or keys.

1. Sign in at [framefields.dev/dashboard](https://framefields.dev/dashboard).
2. Top up credits on the [Billing page](https://framefields.dev/dashboard/billing) with Dodo Payments.
3. Create an API key on the [API Keys page](https://framefields.dev/dashboard/keys).
4. Add the key to your project `.env`:

```bash
FRAMEFIELDS_API_KEY=ff_live_...
```

A project made with `framefields init` already has a `.env.example` (commit it) and ignores `.env` (never commit that). Its `render.ts` loads `.env` when the file exists, so renders and scripts pick up the key without extra setup. Without a key everything else still works, including in CI: only `ai.*` calls need it.

Or configure it in code:

```ts
import { ai } from "framefields";

ai.config({
  apiKey: process.env.FRAMEFIELDS_API_KEY,
});
```

## Model Catalog

Explore available models in the [Model Catalog](https://framefields.dev/models), including text-to-image, image-to-video, voice synthesis, and upscaling models. Each model page lists its input and output fields.

| Kind | Model IDs |
| --- | --- |
| Video (text, image, audio and video to video, with sound) | `seedance/2.5`, `seedance/2.0`, `seedance/2.0-fast`, `seedance/2.0-mini` |
| Images (generation, editing, batches, layer decomposition) | `seedream/5.0-pro`, `seedream/5.0-flash`, `seedream/5.0-lite`, `seedream/4.5` |
| Speech, music and sound effects (with word timestamps) | `seed-audio/1.0` |

You can also fetch the catalog programmatically for agentic workflows:

```bash
curl https://api.framefields.dev/v1/models
```

## API Methods

### `ai.subscribe` (Recommended)

Submits a request to the async processing queue, waits for completion, and returns the final output. Ideal for video and high-resolution rendering models.

```ts
const { data } = await ai.subscribe("seedance/2.5", {
  input: {
    prompt: "A neon geometric sculpture rotating in a void, slow orbit",
    resolution: "1080p",
    ratio: "16:9",
    duration: 5,
  },
  pollInterval: 2000, // poll status every 2 s
});

console.log(data.video.url);
```

### `ai.run`

Executes over a single HTTP connection and returns the output. Fine for images and audio; video models only run through the queue (`ai.subscribe` or `ai.queue`).

```ts
const { data } = await ai.run("seed-audio/1.0", {
  input: {
    text_prompt: "Warm analog synth bed, then a calm narrator says: Welcome to framefields.",
    audio_config: { format: "mp3", enable_subtitle: true },
  },
});

console.log(data.audio.url, data.subtitle); // subtitle has sentence and word timestamps
```

### `ai.stream`

Streams generation events (tokens or partial image steps) over Server-Sent Events (SSE), for models that stream.

```ts
const stream = await ai.stream("flux/dev", {
  input: { prompt: "Cyberpunk cityscape" },
});

for await (const event of stream) {
  console.log("Progress:", event);
}
```

### `ai.storage.upload`

Uploads local buffers or files to framefields storage through signed upload URLs and returns a CDN URL any model accepts as input (e.g. image-to-video or style transfer). `Blob`s inside `input` upload automatically.

```ts
import fs from "node:fs";

const file = fs.readFileSync("input.png");
const fileUrl = await ai.storage.upload(new Blob([file], { type: "image/png" }));

const { data } = await ai.subscribe("seedance/2.5", {
  input: {
    prompt: "Slow cinematic camera push forward",
    content: [{ type: "image_url", image_url: { url: fileUrl }, role: "first_frame" }],
  },
});
```

## Using Generated Assets in Compositions

Download or stream generated assets directly into your `framefields` layer hierarchy:

```ts
import { ai, Composition, Layer } from "framefields";
import fs from "node:fs/promises";

// 1. Generate an image asset
const { data } = await ai.subscribe("seedream/5.0-pro", {
  input: { prompt: "Abstract glowing glass caustics, dark mode", size: "2K" },
});

// 2. Fetch image buffer
const res = await fetch(data.images[0].url);
const imageBuffer = Buffer.from(await res.arrayBuffer());
await fs.writeFile("assets/generated-bg.png", imageBuffer);

// 3. Compose in WebGPU layer
export default Composition.create({
  width: 1920,
  height: 1080,
  fps: 60,
  durationInFrames: 300,
  children: [
    Layer.image({
      source: "assets/generated-bg.png",
      width: 1920,
      height: 1080,
    }),
  ],
});
```

## Credits & Error Handling

Credits are charged in micro-USD ($0.000001 resolution) based on model usage plus margin.

When credits are depleted, requests will return HTTP `402 Payment Required`:

```ts
try {
  await ai.subscribe("seedream/5.0-pro", { input: { prompt: "..." } });
} catch (err: any) {
  if (err.status === 402) {
    console.error("Credits depleted. Top up at https://framefields.dev/dashboard/billing");
  }
}
```

| HTTP Status | Error Type | Solution |
| --- | --- | --- |
| `401` | `invalid_api_key` | Verify `FRAMEFIELDS_API_KEY` in `.env` |
| `402` | `insufficient_credits` | Add credits in [Billing](https://framefields.dev/dashboard/billing) |
| `402` | `key_limit_reached` | Increase key monthly limit in [API Keys](https://framefields.dev/dashboard/keys) |
| `429` | `rate_limited` | Respect retry-after backoff (60 submits/min, 600 status/min) |