AI features

The kit ships an assistant: a chat that streams its answers, on the paid plans, through the AI SDK and the Vercel AI Gateway. Use it as it is, or as the pattern for your own AI features.

Each answer is one AI message of the plan’s monthly allowance: Pro includes 500, then charges 2¢ each. Set yours in PLANS; see Charge for usage. Conversations are not saved; leaving the page ends one.

Turn it on

1. Create a key in the AI Gateway and put it in .env as AI_GATEWAY_API_KEY.
2. Restart bun dev. Assistant is in the sidebar.
3. It needs a plan with the aiAssistant feature. Subscribe, or add the feature to the free plan while you build: see Charge for a feature.

The model is deepseek/deepseek-v4.1-flash unless you set AI_MODEL to another provider/model the gateway lists. What the assistant is told is in apps/api/src/ai/ai.service.ts.

Build your own

Follow apps/api/src/ai:

@WorkspaceScoped()
@RequireFeature('aiAssistant')
@Controller('workspaces/:workspaceId/ai')
export class AiController {
  @Post('chat')
  async chat(
    @CurrentWorkspace() workspace: WorkspaceContext,
    @CurrentUser() user: CurrentUser,
    @Body({ schema: chatInputSchema }) input: ChatInput,
    @Res() response: Response,
  ): Promise<void> {
    const stream = await this.ai.chat(workspace, user, input);
    await pipeUIMessageStreamToResponse({ response, stream });
  }
}

const result = streamText({
  model: this.model,
  instructions: `You are the assistant inside ${this.app.name}.`,
  messages: await convertToModelMessages(messages as UIMessage[]),
  maxOutputTokens: 2000,
});
return toUIMessageStream({
  stream: result.stream,
  onError: () => 'The assistant could not answer.',
});

The rules:

• Inject the model with @Inject(AI_MODEL) model: AiModel. It is null when no key is set, so answer 503 then. Never create a model or read a key anywhere else.
• Write the instructions on the server. Never accept a system message or instructions of any kind from the client; the chat contract only allows user and assistant.
• Bound what one request can cost: a contract that limits the input (CHAT_LIMITS allows 50 messages of 8000 characters) and maxOutputTokens.
• Count each call with UsageService.record() before calling the model, as the assistant does, so a workspace past its allowance is refused before it costs you anything.
• A tool you give the model must only read data of workspace.id.
• AI_MODEL is available in every module; there is nothing to import.

For one answer instead of a stream, generateText throws when the model fails. Catch it, log the reason, and answer with a sentence the person can act on:

try {
  const { text } = await generateText({
    model: this.model,
    instructions,
    prompt,
    maxOutputTokens: 400,
  });
  return text;
} catch (error) {
  this.logger.error(`Summary failed: ${(error as Error).message}`);
  throw new ServiceUnavailableException('The summary could not be written. Try again in a moment.');
}

Put what the person sent in prompt, never in instructions, and tell the model not to follow instructions inside it.

On the web, Chat in apps/web/src/features/assistant/chat.tsx uses useChat from @ai-sdk/react. If you put your own proxy in front of the app, make sure it passes streamed answers on as they are written; the Caddy in the kit does.

Answers survive a closed tab

Each answer is kept in Redis as it arrives, for an hour, under the person, the workspace and the conversation’s id. The model keeps writing when the browser stops reading: close the tab or lose the connection, and the answer is finished all the same. GET .../ai/chat/:id/stream gives the latest answer from its start, then the rest as it arrives, or 204 when there is none, on whichever instance of the API it reaches. It is where useChat({ resume: true }) looks.

The web app keeps conversations in the browser, without an answer still arriving. When the page comes back with a question unanswered, it picks that answer up, whole. New chat starts afresh, and History lists the last 50, grouped by day. The mobile app keeps them on the phone the same way, with its own History and New chat; an answer cut off there is kept as far as it got. For an endpoint of your own, wrap its stream with AnswersService.keep() and add a route like resume in apps/api/src/ai/ai.controller.ts.

Test it

Tests never call a model. Pass MockLanguageModelV4 from ai/test:

const model = new MockLanguageModelV4({
  doStream: async () => ({
    stream: simulateReadableStream({
      chunks: [
        { type: 'text-start', id: 'text-1' },
        { type: 'text-delta', id: 'text-1', delta: 'Hello, Ada!' },
        { type: 'text-end', id: 'text-1' },
        // ... and a 'finish' chunk: see apps/api/test/ai.e2e.test.ts
      ],
    }),
  }),
});
const t = await createTestApp({ teams: false, model });

model.doStreamCalls holds what the model was given, to check that instructions came from the server. Without model, the test runs as if no key were set.

For generateText, mock doGenerate instead, and read model.doGenerateCalls:

const model = new MockLanguageModelV4({
  doGenerate: async () => ({
    content: [{ type: 'text', text: 'A short summary.' }],
    finishReason: { unified: 'stop', raw: undefined },
    usage: {
      inputTokens: { total: 10, noCache: 10, cacheRead: undefined, cacheWrite: undefined },
      outputTokens: { total: 5, text: 5, reasoning: undefined },
    },
    warnings: [],
  }),
});

To test the 402 of a plan without the feature, pin the plans as on Charge for a feature: createTestApp({ teams, model, plans: { free: { features: [] } } }).

Errors you may see: Troubleshooting.