The backend-for-frontend pattern for AI apps: thin client, fat API, one job¶

Most AI apps in 2026 still ship as "browser → OpenAI" with an API key in the frontend bundle. This is broken in every dimension that matters: security, cost, observability, provider lock-in, and rate limiting. The fix is a Backend-for-Frontend (BFF) — a thin server between the browser and the LLM provider that owns auth, cost, streaming, and provider abstraction. The browser does UI. The BFF does everything else.

This is the pattern every serious AI product has converged on by 2026. Here's what it looks like, why each piece exists, and how to build it without over-engineering.

The "browser to OpenAI" anti-pattern¶

A typical AI app from a 2024-era tutorial:

// frontend/src/api.ts
const apiKey = import.meta.env.VITE_OPENAI_KEY  // 🤡

export async function chat(messages) {
  const response = await fetch("https://api.openai.com/v1/chat/completions", {
    method: "POST",
    headers: {
      "Authorization": `Bearer ${apiKey}`,
      "Content-Type": "application/json",
    },
    body: JSON.stringify({ model: "gpt-4o", messages, stream: true }),
  })
  // ...parse SSE, update UI
}

Five things wrong, in order of how fast they'll bite you:

API key is in the browser bundle — anyone can grep your JS and use your key
CORS will break you — OpenAI doesn't allow browser-origin requests, so this doesn't even work in production
No per-user rate limit — one user can burn $200/day
No observability — you have no idea what your users are asking
Provider lock-in — switching from OpenAI to Anthropic means rewriting every frontend call

The fix: add a server in the middle. That server is the BFF.

The BFF architecture¶

┌──────────────┐       ┌──────────────┐       ┌──────────────────┐
│   Browser    │  SSE  │     BFF      │ HTTP  │  LLM Provider    │
│   (UI only)  │ ────▶ │  (own auth,  │ ────▶ │  - OpenAI        │
│              │ ◀──── │   streaming) │ ◀──── │  - Anthropic     │
└──────────────┘       │              │       │  - Self-hosted   │
                       └──────┬───────┘       └──────────────────┘
                              │
                              ▼
                       ┌──────────────┐
                       │ Observability│
                       │ + rate limit │
                       │ + cost track │
                       └──────────────┘

Three rules:

The browser never sees an LLM provider. The browser only knows about your BFF.
The BFF never does business logic about which model to use. It asks a config service or reads env. Provider abstraction lives in a thin adapter, not in routes.
Every request has a user identity at the BFF. No anonymous traffic reaches the provider.

That's it. Everything else is implementation.

The BFF's five jobs¶

A BFF for AI apps has exactly five responsibilities. Resist the urge to add a sixth.

1. Auth¶

The browser authenticates to your BFF using your existing auth system (cookies + session, or short-lived JWT from your auth provider). The BFF holds the LLM provider credentials as server-side secrets.

// BFF: Hono 4.12.x — auth middleware
import { Hono } from "hono"
import { getCookie } from "hono/cookie"

const app = new Hono()

app.use("/v1/*", async (c, next) => {
  const sessionId = getCookie(c, "session")
  const user = await sessions.get(sessionId)
  if (!user) return c.json({ error: "unauthorized" }, 401)

  // Attach user to context — every downstream handler can access it
  c.set("user", user)
  await next()
})

The browser sends Cookie: session=... automatically. No API key in the bundle, no CORS issues, no exposed secrets.

2. Streaming proxy¶

The BFF receives the LLM provider's SSE stream, authenticates it, rate-limits it, logs it, and forwards it to the browser as SSE. This is the bread and butter.

import { trace, SpanStatusCode } from "@opentelemetry/api"

const tracer = trace.getTracer("bff")

app.post("/v1/chat", async (c) => {
  const user = c.get("user")
  const { messages, model } = await c.req.json()

  // Rate limit at the BFF — one user, one rate limit bucket
  const allowed = await rateLimiter.check(user.id, { rpm: 60, tpm: 100_000 })
  if (!allowed) return c.json({ error: "rate_limited" }, 429)

  // Start an OpenTelemetry span — every chat has a trace
  const span = tracer.startSpan("chat", {
    attributes: {
      "user.id": user.id,
      "gen_ai.request.model": model,
    },
  })

  return streamSSE(c, async (stream) => {
    const upstream = await provider.stream({
      user: user.id,
      model,
      messages,
      signal: c.req.raw.signal,  // forward client disconnect to upstream
    })

    let inputTokens = 0
    let outputTokens = 0

    // try/catch/finally lives INSIDE the stream callback — otherwise it only
    // wraps stream *creation*, mid-stream errors are lost, and the span never ends.
    try {
      for await (const chunk of upstream) {
        // Chunk uses OpenAI's `completion_tokens`; OTel semantic conventions
        // call the same field `output_tokens` — map explicitly.
        inputTokens += chunk.usage?.input_tokens ?? 0
        outputTokens += chunk.usage?.completion_tokens ?? 0
        await stream.writeSSE({ data: JSON.stringify(chunk) })
      }

      // Record cost attribution
      span.setAttribute("gen_ai.usage.input_tokens", inputTokens)
      span.setAttribute("gen_ai.usage.output_tokens", outputTokens)
      span.setAttribute("gen_ai.usage.cost_usd", cost(model, inputTokens, outputTokens))
      span.setStatus({ code: SpanStatusCode.OK })
    } catch (err) {
      span.recordException(err)
      span.setStatus({ code: SpanStatusCode.ERROR })
      throw err
    } finally {
      span.end()
    }

    // Persist async (don't block the response)
    void usage.record(user.id, { model, inputTokens, outputTokens })
  })
})

Three things to notice:

c.req.raw.signal is forwarded upstream. If the user closes the tab, the BFF cancels the request to the provider, and you stop paying for tokens you'll never see.
Cost is computed in the BFF and attached to the span. You get full FinOps visibility per user, per feature.
The BFF doesn't know which model "is right" for the request — it accepts model as a parameter from the (trusted) frontend, validates it against an allowlist, and forwards. Choosing models is a product decision; the BFF is plumbing.

3. Provider abstraction¶

The BFF owns the swap from OpenAI to Anthropic to self-hosted. The frontend never imports an SDK from a provider; it only talks to your BFF.

// providers/index.ts
export interface LLMProvider {
  stream(params: StreamParams): AsyncIterable<Chunk>
  countTokens(params: CountParams): Promise<number>
}

export function getProvider(name: string): LLMProvider {
  switch (name) {
    case "openai":    return new OpenAIProvider()
    case "anthropic": return new AnthropicProvider()
    case "vllm":      return new VLLMProvider()
    default: throw new Error(`Unknown provider: ${name}`)
  }
}

Each provider normalizes to a common Chunk interface:

// providers/types.ts
export interface StreamParams {
  model: string
  messages: ChatMessage[]
  signal?: AbortSignal
  user?: string  // optional — used by providers that accept a per-request user identifier (e.g. OpenAI `user` field for abuse detection)
}

export interface ChatMessage {
  role: "system" | "user" | "assistant"
  content: string
}

export interface Chunk {
  delta: string
  finish_reason?: "stop" | "length" | "tool_use" | "content_filter"
  usage?: {
    input_tokens: number
    completion_tokens: number
  }
  tool_call?: {
    name: string
    arguments: string  // JSON string
  }
}

The BFF streams chunks to the browser in the BFF's format, not the provider's. When you switch providers, the frontend doesn't change.

A real OpenAI adapter:

// providers/openai.ts
export class OpenAIProvider implements LLMProvider {
  async *stream({ model, messages, signal }: StreamParams): AsyncIterable<Chunk> {
    const res = await fetch("https://api.openai.com/v1/chat/completions", {
      method: "POST",
      headers: {
        "Authorization": `Bearer ${env.OPENAI_KEY}`,
        "Content-Type": "application/json",
      },
      body: JSON.stringify({ model, messages, stream: true, stream_options: { include_usage: true } }),
      signal,
    })
    if (!res.ok) throw new ProviderError(res.status, await res.text())

    const reader = res.body!.getReader()
    const decoder = new TextDecoder()
    let buffer = ""

    while (true) {
      const { done, value } = await reader.read()
      if (done) break
      buffer += decoder.decode(value, { stream: true })
      const events = buffer.split("\n\n")
      buffer = events.pop() || ""

      for (const evt of events) {
        const line = evt.split("\n").find(l => l.startsWith("data: "))
        if (!line) continue
        const data = line.slice(6)
        if (data === "[DONE]") continue
        const json = JSON.parse(data)
        const choice = json.choices?.[0]
        if (choice?.delta?.content) yield { delta: choice.delta.content }
        if (json.usage) yield { delta: "", usage: json.usage }
      }
    }
  }
}

The Anthropic adapter is a different class with different protocol handling. The BFF routes are the same.

4. Cost + rate limit¶

The BFF is the only place that knows which user is making which request. That's where rate limits and cost controls must live.

// lib/rate-limit.ts — token bucket per user
class TokenBucket {
  private tokens: number
  private lastRefill: number

  constructor(
    private capacity: number,
    private refillPerSecond: number,
  ) {
    this.tokens = capacity
    this.lastRefill = Date.now()
  }

  tryConsume(amount: number): boolean {
    const now = Date.now()
    const elapsed = (now - this.lastRefill) / 1000
    this.tokens = Math.min(this.capacity, this.tokens + elapsed * this.refillPerSecond)
    this.lastRefill = now

    if (this.tokens >= amount) {
      this.tokens -= amount
      return true
    }
    return false
  }
}

const buckets = new Map<string, TokenBucket>()

export function checkRate(userId: string, cost: number): boolean {
  let bucket = buckets.get(userId)
  if (!bucket) {
    // 100K tokens/min per user is a reasonable default for chat
    bucket = new TokenBucket(100_000, 100_000 / 60)
    buckets.set(userId, bucket)
  }
  return bucket.tryConsume(cost)
}

You can also use a Redis-backed bucket if you have multiple BFF instances:

// Distributed rate limit
import { redis } from "./redis"

export async function checkRateDistributed(userId: string, cost: number): Promise<boolean> {
  const key = `ratelimit:${userId}`
  const result = await redis.eval(LUA_SCRIPT, 1, key, cost, 100_000, "60")
  return result === 1
}

Cost budgets go a step further. Set per-user, per-day, per-feature budgets in OPA or your gateway, and reject before reaching the provider:

# OPA policy
package bff.ratelimit

deny[msg] {
    input.user.daily_spend_usd > 50
    msg := sprintf("Daily budget exceeded: $%.2f", [input.user.daily_spend_usd])
}

deny[msg] {
    input.user.monthly_spend_usd > 500
    msg := sprintf("Monthly budget exceeded: $%.2f", [input.user.monthly_spend_usd])
}

5. Observability¶

Every chat request gets a trace that ties together the user's request, the BFF's processing, and the provider's response. This is the only way to debug "why is the latency bad" or "why is the bill up 40%."

// BFF: every chat emits an OTel span
import { trace, SpanStatusCode } from "@opentelemetry/api"

const tracer = trace.getTracer("bff")

app.post("/v1/chat", async (c) => {
  const user = c.get("user")
  const { model } = await c.req.json()
  const span = tracer.startSpan("chat", {
    attributes: {
      "user.id": user.id,
      "user.tier": user.tier,
      "gen_ai.request.model": model,
    },
  })

  // try/catch/finally lives INSIDE the stream callback — otherwise it only
  // wraps stream *creation*, mid-stream errors are lost, and the span never ends.
  return streamSSE(c, async (stream) => {
    try {
      for await (const chunk of provider.stream({...})) {
        span.addEvent("token", { "gen_ai.usage.completion_tokens": chunk.usage?.completion_tokens ?? 0 })
        await stream.writeSSE({ data: JSON.stringify(chunk) })
      }
      span.setStatus({ code: SpanStatusCode.OK })
    } catch (err) {
      span.recordException(err)
      span.setStatus({ code: SpanStatusCode.ERROR })
      throw err
    } finally {
      span.end()
    }
  })
})

The trace uses the OpenTelemetry GenAI semantic conventions¹ — gen_ai.request.model, gen_ai.usage.*, and per-token events. The conventions are stable as of 2026, so attributes you set here are portable across backends.

The trace has: - User ID and tier (so you can segment cost by plan) - Model and token counts - Latency breakdown (BFF processing vs provider response time) - Errors with stack traces

Send it to Langfuse, Honeycomb, Datadog, or any OTLP backend. When the CFO asks why the bill jumped, you answer with a query.

The full request lifecycle¶

A user clicks "send" in the chat UI. Here's what happens in 8 steps:

Browser posts to /v1/chat on the BFF with the user's messages
BFF auth middleware validates the session cookie, attaches the user to context
BFF rate limiter checks the user's token bucket
BFF starts an OTel span with user.id, model, request size
BFF calls provider.stream() with the user's signal (for cancellation)
Provider returns an SSE stream of chunks
BFF normalizes each chunk to the BFF's common format, attaches usage + cost
BFF writes each chunk to the browser as SSE and to Langfuse as span events

When the user closes the tab, the browser's AbortSignal fires, the BFF's c.req.raw.signal propagates to the provider's fetch, and the upstream request is cancelled. You stop paying for tokens you'll never see.

What the BFF is NOT¶

Resist scope creep. The BFF is not:

A business-logic service. The BFF doesn't decide which model to use, doesn't do RAG, doesn't call tools. That's an "agent service" sitting behind the BFF.
A gateway. BFF is per-frontend (web BFF, mobile BFF, internal-tools BFF). A gateway is shared across services and routes between them.
An API aggregator. The BFF calls ONE provider per request. If you're calling 5 services to assemble a response, that's an orchestrator, not a BFF.
A model router. The BFF takes model: string from a trusted caller. Smart routing (cost-based, quality-based, latency-based) is a separate concern.

If your BFF is doing any of these things, split them into a separate service and let the BFF call them.

Pre-build checklist¶

Before you build a BFF:

The BFF in different stacks¶

Hono 4.12.x (Node/Bun/Deno):

import { Hono } from "hono"
import { streamSSE } from "hono/streaming"

const app = new Hono()
app.post("/v1/chat", (c) => streamSSE(c, async (stream) => {
  for await (const chunk of provider.stream({...})) {
    await stream.writeSSE({ data: JSON.stringify(chunk) })
  }
}))

FastAPI 0.137.x (Python):

from fastapi import FastAPI
from fastapi.responses import StreamingResponse

app = FastAPI()

@app.post("/v1/chat")
async def chat(req: ChatRequest, user: User = Depends(auth)):
    async def stream():
        async for chunk in provider.stream(req.messages):
            yield f"data: {chunk.model_dump_json()}\n\n"

    return StreamingResponse(stream(), media_type="text/event-stream")

Next.js 16.x Route Handler (server-side only):

// app/api/chat/route.ts
import { auth } from "@/lib/auth"

export async function POST(req: Request) {
  const user = await auth()
  if (!user) return new Response("Unauthorized", { status: 401 })

  const encoder = new TextEncoder()
  const stream = new ReadableStream({
    async start(controller) {
      for await (const chunk of provider.stream({...})) {
        controller.enqueue(encoder.encode(`data: ${JSON.stringify(chunk)}\n\n`))
      }
      controller.close()
    },
  })

  return new Response(stream, {
    headers: { "Content-Type": "text/event-stream" },
  })
}

All three expose the same shape: an authenticated, streaming, observable endpoint that owns its provider credentials.

Summary¶

Browser never sees an LLM provider. Browser → BFF → provider. Period.
BFF owns five things: auth, streaming, provider abstraction, rate limit + cost, observability.
Provider abstraction is a thin adapter, not a framework. Three classes, one interface.
Cost is computed at the BFF and attached to every span. No more bill surprises.
Cancellation is wired end-to-end — close the tab, stop paying.
Don't conflate BFF with gateway, agent service, or model router. They have different jobs.

The 2024-era "frontend calls OpenAI directly" pattern is dead. Every AI product that took off in 2025-2026 has a BFF. The BFF is the boring, correct, unsexy architecture that wins.

Questions or discussion? Connect on LinkedIn, X or reach out via email.

OpenTelemetry, Semantic Conventions for Generative AI — gen_ai.request.model, gen_ai.usage.input_tokens, gen_ai.usage.output_tokens, and per-event token counts. Stable as of 2026; supported by Langfuse, Honeycomb, Datadog APM, and the OTel Collector. ↩
100K tokens/min is a starter bucket for chat workloads (avg 200-token completions × ~30 requests/min). Real limits depend on your model price, plan tier, and provider quotas — measure, don't guess. ↩

Discussion

Have thoughts on this post? Share them below — questions, corrections, or your own experience are all welcome.