AI Tools · Const

Caching

Cache AI provider responses temporarily and replay them for repeated requests. The recommended method uses language model middleware, with simulateReadableStream for cached streaming output.

How it works

Caching stores model responses temporarily and reuses them for matching inputs. The recommended current approach is language model middleware: cache complete generate results directly, while storing stream parts and replaying them with simulateReadableStream.

Syntax

wrapLanguageModel({ model, middleware })

Parameters

Name	Type	Description
`middleware`	`LanguageModelMiddleware`	Language model middleware configuration containing wrapGenerate and/or wrapStream handlers.
`model`	`LanguageModel`	The model supplied to the wrapped language-model helper.

Returns

A wrapped language model whose generation and streaming operations can serve cached responses.

Key points

Use language model middleware as the primary integration point for response caching.
Replay cached streaming output with simulateReadableStream.
A non-streaming generation result can be stored and returned directly on a cache hit.
Replaying stream parts preserves the incremental delivery behavior expected by streaming consumers.
As an alternative, use the generation lifecycle callback onEnd to save the completed response.

Common mistakes

If using structured output, do not cache the raw response before schema validation; cache only a response that has passed validation.
Streaming requires caching stream parts rather than treating the stream like a complete generate result.

Example: Cache AI SDK model responses

import { generateText, wrapLanguageModel, simulateReadableStream } from 'ai';
import { openai } from '@ai-sdk/openai';

const cache = new Map<string, any>();

const middleware = {
  wrapGenerate: async ({ doGenerate, params }: any) => {
    const key = JSON.stringify(params);
    const cached = cache.get(key);
    if (cached) return cached;
    const result = await doGenerate();
    cache.set(key, result);
    return result;
  },
  wrapStream: async ({ doStream, params }: any) => {
    const key = JSON.stringify(params);
    const cached = cache.get(key);
    if (cached) {
      return { stream: simulateReadableStream({ chunks: cached }) };
    }
    const { stream, ...rest } = await doStream();
    const chunks: any[] = [];
    const response = stream.pipeThrough(new TransformStream({
      transform(chunk, controller) { chunks.push(chunk); controller.enqueue(chunk); },
    }));
    cache.set(key, chunks);
    return { stream: response, ...rest };
  },
};

const model = wrapLanguageModel({ model: openai('gpt-4o-mini'), middleware });
await generateText({ model, prompt: 'Say hello.' });

Wraps generation and streaming calls with an in-memory cache; cached stream chunks are replayed through simulateReadableStream.

When to use it

Reduce repeated generation

Repeated prompts or deterministic requests should avoid a new provider call.

Replay streamed output

Streaming UI responses need to retain incremental delivery on cache hits.

Cache with onEnd

Completed generation results should be stored through an SDK lifecycle hook.

Language Model Middleware
wrapGenerate
wrapStream
simulateReadableStream
onEnd

Source: Caching. Checked 2026-10-15.