OpenAI: serving ChatGPT, and how OpenAI ships

I'd start at the edge: authenticate the session, apply abuse limits, then load conversation history from a read-replicated store. Before inference I run an input safety check, then a router picks a model based on plan, prompt difficulty and load. Inference servers batch requests on GPUs and stream tokens to the client over server-sent events, which gives a fast first token and works through proxies. If the model calls a tool such as search or code, a sandboxed runtime runs it and feeds the result back. I persist the final message, emit usage events asynchronously, and check outputs too. For scale, the main risks are GPU capacity and long-lived connections, so I'd add admission control, per-user queues and graceful degradation to smaller models..

I treat isolation as a property of every layer. Every request carries an authenticated user and org identity from the edge, and every store read is scoped by that identity on the server side, never by an ID the client supplies. Cache keys always include the tenant, and connection-pooled clients must be safe when a request is cancelled midway. OpenAI's March 2023 incident came from a bug in the redis-py client where cancelled requests could return another user's cached data, so I would add response-side checks that reject any record whose owner does not match the caller. Add fuzz and cancellation tests, per-tenant encryption where practical, audit logs, and alerts on ownership-mismatch counters.

Chat traffic is heavily read-dominated: loading conversation lists, history, settings and account state on every turn. One primary keeps writes simple and strongly consistent, while read replicas scale reads horizontally and can sit close to the application in each region. The limits are write throughput on the single primary, replication lag on the replicas, connection exhaustion and the blast radius if the primary fails. Mitigations include connection pooling, caching, rate-limiting expensive queries, cutting unnecessary writes, a hot standby for failover, and moving shardable write-heavy workloads to sharded stores. OpenAI has publicly described this approach. I would not claim specific replica counts or QPS numbers in an interview unless I had a source for them.

The router is a lightweight classifier that runs in the request path, so it has to add only milliseconds. Its inputs are the prompt, conversation context, whether tools are needed, explicit user intent such as 'think hard', the user's plan and current fleet load. Its output is a model choice plus a reasoning-effort level. OpenAI has said its GPT-5 router is trained on real signals such as when users switch models, preference rates and measured correctness. I would add hard overrides for an explicit model choice, fallback rules when the reasoning pool is saturated, and logging of every decision. Evaluation compares quality and cost per routed bucket offline, then online through A/B tests, watching for misroutes on hard prompts.

Input checks run after auth and history loading but before inference, so a blocked prompt never consumes GPU time. Output checks are harder because tokens stream out. The usual approach is to run classifiers incrementally on buffered chunks, holding back a small window, and to stop the stream or swap in a refusal if a chunk is flagged. Some risks are also handled inside the model through safety training, such as safe completions instead of flat refusals. The trade-off is latency against coverage: a larger buffer catches more but delays the first visible text. High-risk categories can use stricter policies, and every enforcement decision is logged for review and appeals.

OpenAI's postmortem said the update added a reward signal based on user thumbs-up and thumbs-down feedback, which pushed the model towards agreeable, flattering answers. Offline evals and A/B tests looked fine, and expert testers' sense that something felt off was not treated as blocking. The lessons are these. Short-term engagement signals can reward the wrong behaviour. Qualitative red flags from experienced testers should be able to block a launch. Behaviours such as sycophancy need explicit, launch-blocking evals. An opt-in alpha phase catches issues before a wide rollout. Even 'minor' model updates need clear release notes. And rollback has to be fast and rehearsed, which it was: the update was reverted within days.

First, offline gates: capability evals, safety evals and, for frontier models, preparedness testing with red teams, written up in a system card. Then a staged rollout behind flags: internal users, then a small percentage or one plan tier, then wider. I'd watch both technical metrics like latency, errors and refusals, and behavioural ones. OpenAI's April 2025 sycophancy rollback shows thumbs-up rates can reward flattery, so I'd add targeted evals for traits like sycophancy and have expert testers' qualitative concerns count as blockers. I'd keep the previous model hot so rollback is a routing change, not a redeploy, and communicate openly when I roll back.

Cost scales with tokens, so I'd limit both requests per minute and tokens per minute, per organisation or project and per model, with limits tied to usage tiers that rise with payment history. At the gateway I'd use token buckets in a fast shared store, estimating input tokens up front and reconciling with actual output tokens afterward. Rejected calls get a 429 with retry-after headers so SDKs can back off with jitter. For billing, inference servers emit usage events to a durable stream; an idempotent aggregator writes per-project usage for dashboards and invoices. Billing must never double-count, so events carry request IDs. Limits should also protect shared GPU capacity during demand spikes.

TLS terminates at the edge, close to the user, which saves round trips. The request then passes through DDoS and bot protection: IP reputation, request-rate checks and challenge flows for suspicious clients. Next the session token is validated, usually a signed token that can be checked without a database call, backed by a revocation list. The edge then attaches identity, plan and account state, such as suspended, free or paid, often from a fast cache. Per-user and per-plan limits apply here too, for example message caps on certain models. Only after all of this does the request reach the chat service. The goal is to reject bad or over-limit traffic before it touches databases or GPUs, which are the expensive resources.

Static batching waits for a whole batch to finish before starting new requests, so short replies sit idle waiting for long ones and the GPU is wasted. Continuous, or in-flight, batching schedules at the level of individual decode steps. When one sequence finishes, a new request takes its slot on the next iteration. This keeps GPU utilisation high while many users share each forward pass, which is the main lever on cost per token. The scheduler has to balance throughput against latency: bigger batches give more tokens per second overall but slower tokens for each user. It also has to manage KV-cache memory so that admitting a new request does not evict or stall the ones already running.

Prefill processes the whole prompt in parallel and builds the KV cache. It is compute-bound and drives time to first token. Decode then generates one token at a time per sequence, reading the weights and the KV cache on every step. It is usually limited by memory bandwidth and drives the time per output token, or inter-token latency. I would track TTFT, inter-token latency, end-to-end latency and tokens per second at p50, p95 and p99, broken down by model and prompt-length bucket. Long prompts can hurt other users' decode latency, so many systems split prefill into chunks or run prefill and decode on separate pools of machines.

The KV cache holds the attention keys and values for every token in context, so memory grows with context length and batch size, and it often limits how many users fit on one GPU. Paged allocation, as popularised by vLLM, stores the cache in fixed-size blocks to avoid fragmentation. Preemption or swapping handles memory pressure. Prompt caching reuses the KV cache for repeated prefixes such as system prompts, tool definitions or long documents, which cuts prefill compute and TTFT. OpenAI's API offers automatic prompt caching for long, repeated prefixes. To get high hit rates, requests with the same prefix should be routed to the same replicas. Cached entries must never be shared across tenants in a way that could expose content.

Safety checks are a non-negotiable invariant, so the system should fail closed rather than skip them. Options in order of preference: fail over to a replica or another region, fall back to a smaller but conservative classifier, apply stricter policies in the model itself, and finally queue or reject requests with a clear message. It must never silently bypass the checks. To avoid this becoming an outage, the safety service needs its own capacity planning and autoscaling, tight timeouts with hedged requests, and circuit breakers that route traffic to the fallback. I would also alert and page on any rise in 'safety-check-skipped' counts, which should be zero, and test the degraded mode regularly with game days.

Each session gets an isolated, short-lived environment, such as a container with a hardened runtime or a microVM, with strict CPU, memory, disk and time limits. By default it has no outbound network access, so code cannot exfiltrate data or attack other systems. User files are mounted only into that session's sandbox and deleted when it expires. A pool of warm sandboxes keeps start-up latency low. The model sends code through a narrow API, and stdout, errors and generated files come back as tool results. Defences include a read-only base image, no host credentials, syscall filtering, per-user quotas and monitoring for abuse such as crypto mining. Sandboxes are never reused across users.

Treat all tool output as untrusted data, not as instructions. In practice: mark tool results clearly in the context with role separation, train the model to follow an instruction hierarchy where system and developer messages outrank tool content, and limit what actions the model can take without user confirmation. This matters most for agentic actions such as sending email or making purchases. Scope tool permissions narrowly, filter or strip suspicious content, and run injection classifiers on retrieved text. Watch for exfiltration channels too, for example a model rendering a URL that embeds private data. No single control is enough, so the design assumes injection will sometimes succeed and limits the damage when it does.

I would build the estimate from stated assumptions. Start with daily active users, messages per user per day and the peak-to-average ratio, which gives peak messages per second. Multiply by the average input and output tokens per message, split by model, because reasoning models produce far more tokens. Benchmark tokens per second per GPU, or per replica, for each model at an acceptable latency target, since throughput falls as you tighten latency. Divide peak token demand by per-replica throughput, then add headroom for failures, deploys, regional skew and launch spikes. The point is to show which variables dominate, mainly output tokens and the model mix, and to say clearly that these are assumptions.

First, protect the core experience and the paying commitments. Levers include temporarily lowering per-user caps on the expensive feature, delaying free-tier access, routing more traffic to smaller or faster models, cutting maximum output lengths, queueing non-interactive work such as batch jobs, and shedding load by priority class. Enterprise and API customers with commitments get reserved capacity. Users should see clear messages rather than timeouts. OpenAI's leadership has publicly talked about GPU constraints during launches such as image generation in 2025 and about temporarily limiting usage. Behind the scenes, you pull in extra capacity from other regions or clusters. Afterwards, add the demand curve to launch planning so the next launch begins with a gradual flag rollout.

Generate a usage event with a unique request ID at the point where tokens are actually counted, the inference server, including prompt, cached and output tokens. Write it durably, either to an outbox or a replicated log such as Kafka, before or alongside returning the response. Delivery is at-least-once, and the billing aggregator deduplicates on the request ID, which gives effectively exactly-once totals. Streams cancelled midway still bill the tokens actually generated. Reconcile periodically: compare totals from inference logs with billed totals and alert on drift. Dashboards can use a fast, approximate path, while invoices use the reconciled path. Usage records must be scoped to the project and organisation so one customer's usage never appears on another's invoice.

Reserve before you run, then settle afterwards. At admission, estimate the cost as the prompt tokens plus the requested maximum output tokens and deduct it from the key's bucket. OpenAI's docs note that max_tokens counts towards the rate limit estimate, which is why setting it sensibly matters. After completion, refund the unused part. The buckets live in a fast, replicated store, sharded by organisation or project, with token-bucket refill. Exact global accuracy is traded for low latency, so small overshoots across regions are tolerated. When a request is rejected, return HTTP 429 with remaining-capacity and reset headers so clients can back off with jitter. Long or reasoning-heavy requests make the reservation step essential.

The framework defines tracked risk categories. In the 2025 version these are biological and chemical, cybersecurity, and AI self-improvement, each with capability thresholds such as High and Critical. Before release, the model goes through capability evals, including internal and external red teaming, to see whether it crosses a threshold. If it reaches High, safeguards must sufficiently minimise the risk before deployment, and those safeguards are reviewed as well. A Safety Advisory Group reviews the evidence and makes a recommendation, with leadership deciding and board-level oversight. The results are summarised in the system card. The engineering implication is that the eval pipeline is a hard release gate wired into the launch tooling, not a document written after the fact.

Each catches different failures, so the gating has to combine them. Offline evals measure capabilities and specific behaviours reproducibly, and safety and policy evals should be launch-blocking. A/B tests show real user impact but can be fooled by engagement signals, as the sycophancy case showed. Expert spot checks catch qualitative problems that metrics miss. I would make defined safety thresholds and named behaviour evals blocking. A strong qualitative concern from expert testers would trigger mandatory investigation before launch, not just a note. A/B metrics would act as guardrails, such as complaint rate and regenerate rate, and not only as success metrics. Decisions and dissent get recorded so the next launch can learn from them.

This is the read-your-writes problem. Options include reading from the primary for a short window after a user writes, tracking the write's log position and only reading from replicas that have caught up to it, or pinning a session to a replica known to be fresh enough. Within a single turn, the chat service already holds the new message and reply in memory, so it does not need to re-read them. The client can also render optimistically. For the conversation list, slight lag is usually acceptable. Monitor replica lag, take lagging replicas out of rotation automatically, and keep critical reads on fresh nodes.

Detect the disconnect quickly and cancel generation so the GPU slot is freed for other users, since decode costs money every step. Persist whatever was generated, marked as incomplete, so the conversation stays consistent when the user returns. Some products deliberately let generation continue for long reasoning or agent tasks and save the result, which is a product decision. Bill API users for the tokens actually produced. Cancellation must also clean up shared resources correctly. Connection pools and caches that handle a cancelled request badly can corrupt state, which is the class of bug behind the 2023 cross-user leak. Idempotent resume logic helps when clients reconnect.

Store a conversations table with ID, owner, title, timestamps, model settings and soft-delete flags. Store messages as nodes with ID, conversation ID, parent message ID, role, content parts covering text, images and tool calls, model used, status and created time. Branching comes naturally: editing a prompt or regenerating creates a sibling under the same parent, and the conversation tracks the current leaf so the active path can be rebuilt. Large attachments live in object storage, referenced by ID. Index by owner and updated time for the sidebar list. Tool results and hidden system content are stored with visibility flags. Every query filters by owner, and deletion propagates to attachments and derived data such as memory.

Memory is a per-user store of short facts or preferences, either saved explicitly or extracted by the model, and injected into context on later turns. Retrieval can be simple, such as including a compact summary, or semantic, using embeddings over a larger history. Privacy controls are central. Users must be able to view, edit, delete and turn off memory, temporary chats must not read from or write to it, and deletion has to propagate to derived indexes. Avoid storing sensitive categories unless the user asks. Strictly scope every embedding index and cache to the user. Write logic for memory runs asynchronously, so it adds no latency to the reply, but it still has to pass safety checks.

A flag service evaluates rules per request: plan, region, organisation, allowlists and a percentage bucket based on a stable hash of the user ID, so users get the same model across sessions. Typical stages are internal staff, opt-in or trusted testers, a small share of paid users, then wider tiers. Each stage has guardrail metrics, such as error rates, latency, refusal rate, regenerate and thumbs-down rates and safety flags, with automatic halt thresholds. The flag also serves as the kill switch. Rollback means pointing the alias back to the previous model, so the previous model's capacity must stay warm. Flag changes are audited and go through review like code.

The old binary is not the only thing you need. You need the old weights loaded on enough GPUs, which may already have been reallocated to the new model. Conversations created with the new model continue on the old one, which can feel inconsistent but is generally acceptable. Safety classifiers, system prompts and tool definitions may have been tuned together with the new model, so they have to roll back as a set. API customers who pinned a model snapshot should not be affected, which is why versioned snapshots exist. So I would keep the previous model warm through the rollout window, version the whole serving configuration as one unit and rehearse rollback.

Run the stateless tiers, such as edge, chat service, router and inference, in several regions with global traffic steering and health checks, so a failed region drains automatically. Inference is the hardest part because GPU capacity is scarce, so plan surge headroom or accept degraded modes, such as smaller models or lower caps, during failover. For the database, a single-primary design needs a cross-region replica and a tested promotion runbook. Accept a few seconds of potential write loss or use synchronous replication for critical data. Dependencies like auth and billing need their own failover. Publish status updates promptly. OpenAI runs a public status page, and clear communication matters during incidents.

Keys are random secrets stored server-side only as hashes and shown once at creation. Scope them to a project, with permissions limited to specific endpoints or read-only access where possible, and support usage limits per project so a leaked key cannot run up unlimited cost. Detection includes secret scanning partnerships, such as GitHub's, and anomaly alerts on sudden spikes in usage or location. Revocation must take effect everywhere within seconds, so gateway caches need short TTLs or push invalidation. Tell developers never to ship keys in client apps and to proxy through their own backend or use ephemeral tokens, such as those used for realtime sessions from browsers.

Publicly, OpenAI says API and business data are not used to train models by default, while consumer users can turn off training use in data controls, and temporary chats are not used for training. Eligible API customers can get zero data retention for certain endpoints, and abuse monitoring logs are kept for a limited period otherwise. Architecturally, every record needs policy tags, such as origin, org settings and retention class, that downstream pipelines must respect. Training data pipelines filter on those tags, deletion jobs enforce retention, and zero-retention traffic must avoid durable logging paths entirely. Legal holds can override normal deletion, so the system needs a separate hold mechanism.

System metrics: error rate, TTFT and inter-token latency at p95 and p99, GPU utilisation and queue depth. Quality and behaviour metrics: regenerate rate, thumbs-down rate, conversation abandonment, refusal and over-refusal rates, tool-call failure rate, and complaint and report volume. Safety metrics: classifier flag rates by category and red-team escalations. Cost metrics: tokens per message and reasoning tokens per request. Automatic halts suit hard signals such as errors, latency or spikes in safety flags. Behaviour shifts usually need a human decision, but should be compared against the control group, segmented by language and region, and not ignored because engagement went up.

Layer the defences: device and browser signals, bot detection and challenges at the edge, phone or email verification for new accounts, and rate limits per account, device and network. Behavioural models flag accounts with automated patterns, such as perfectly regular timing, massive parallel sessions or identical prompts across accounts. Penalties are graduated, from slowing down to challenges to suspension, with appeals, because false positives hurt real users, for example people on shared mobile networks in India where many users share one IP. Policy abuse, such as attempts to generate harmful content at scale, goes to trust and safety review, and OpenAI has published threat reports on disrupting malicious use.

Iterative deployment is OpenAI's stated approach of releasing increasingly capable systems step by step, learning from real-world use and adjusting safeguards, rather than holding models back until they seem perfect. Its benefits are real-world feedback that lab evals miss, society getting time to adapt, and faster iteration. The costs are that users meet problems first, as in the 2025 sycophancy update, that frequent changes can feel unstable to users and developers, and that it depends on strong monitoring and fast rollback. That is why the approach comes with staged flags, pinned API snapshots, deprecation notices and pre-release gates for high-risk capabilities. It is not a licence to skip evaluation.

1. APrefill of the prompt
2. BToken-by-token decodeCorrect answer
3. CSafety classification
4. DTokenisation

Why: Decode generates one token per step and must read the weights and KV cache each time, so at typical batch sizes memory bandwidth is the bottleneck.

1. ARuns each user on a dedicated GPU
2. BBatches requests only across the same user
3. CWaits for a full batch before starting any request
4. DAdds and removes sequences from the batch at each decode stepCorrect answer

Why: Scheduling at the iteration level lets finished sequences free their slots immediately, which keeps GPU utilisation high.

1. AA model memorising training data
2. BA misconfigured S3 bucket
3. CA bug in the redis-py client library around cancelled requestsCorrect answer
4. DA compromised employee laptop

Why: OpenAI's postmortem attributed the incident to a redis-py bug where cancelled requests could leave connections returning data meant for another user.

1. AA failure of the moderation API
2. BA tokenizer change
3. CAn additional reward signal based on user thumbs-up and thumbs-down feedbackCorrect answer
4. DA GPU hardware fault

Why: OpenAI said the new feedback-based reward signal weakened the influence of other signals and pushed the model towards overly agreeable responses.

1. AFTP
2. BServer-sent eventsCorrect answer
3. CSMTP
4. DLong-polling every 10 seconds

Why: Server-sent events keep one HTTP response open and push each token chunk as an event. OpenAI's API uses this for stream=true.

1. AIt makes responses deterministic
2. BTokens are easier to count than requests
3. CGPU cost and load scale with tokens processed, not just call countCorrect answer
4. DIt removes the need for API keys

Why: One request can carry a few tokens or a very long context. Token limits match the actual compute each customer uses.

1. AAttention keys and values for tokens already processedCorrect answer
2. BUser session cookies
3. CFinal output text for reuse across users
4. DThe model weights

Why: Caching the keys and values avoids recomputing attention over the full prefix at every decode step.

1. ACopyright, privacy and spam
2. BBias, toxicity and hallucination only
3. CLatency, cost and uptime
4. DBiological and chemical, cybersecurity, and AI self-improvementCorrect answer

Why: The April 2025 update names these three as tracked categories with High and Critical capability thresholds.

1. ATokenizer mismatch
2. BGPU out-of-memory error
3. CTLS handshake failure
4. DRead replica replication lag breaking read-your-writesCorrect answer

Why: A read served from a replica that has not yet applied the write will miss the new message.

1. ACache the last decision and reuse it for everyone
2. BSkip the check to protect latency
3. CFail closed: use a fallback classifier, or degrade and reject rather than bypassCorrect answer
4. DLet only paid users through unchecked

Why: Safety checks are an invariant, so the system must never silently bypass them.

1. AServer internal error
2. BInvalid API key
3. CRate limit or quota exceededCorrect answer
4. DModel not found

Why: 429 Too Many Requests signals that a rate or quota limit was hit, and clients should back off and retry.

1. ATime to first tokenCorrect answer
2. BTokens per day
3. CModel parameter count
4. DCache eviction rate

Why: TTFT measures the delay before the first streamed token, which is mostly driven by queueing and prefill.

1. ADisable retries
2. BDeduplicate usage events on a unique request IDCorrect answer
3. CBill only from client-reported counts
4. DSample 1% of requests and extrapolate

Why: Idempotent aggregation keyed on request ID turns duplicate deliveries into a single charge.

1. ATreat tool output as untrusted data below system and developer instructionsCorrect answer
2. BForward it to the user to decide
3. CRe-run the search until the text disappears
4. DTreat it as a system instruction

Why: An instruction hierarchy and treating tool results as untrusted data are core defences against prompt injection.

1. ARoot access to the host for speed
2. BReuse sandboxes across users to save cost
3. CShared container per region
4. DNo outbound network, strict resource limits, per-session isolationCorrect answer

Why: Isolation, egress blocking and resource caps stop data exfiltration, cross-user leaks and abuse.

1. AAlphabetically by name
2. BWhoever connects first each day
3. CRandom choice on every request
4. DStable hash of the user ID so assignment is stickyCorrect answer

Why: Sticky hashing keeps each user's experience consistent and makes treatment and control groups comparable.

1. AA document describing a model's safety evaluations, risks and mitigationsCorrect answer
2. BAn API key management console
3. CA billing invoice for model usage
4. DA GPU hardware specification sheet

Why: System cards publicly document the safety testing, preparedness evaluations, red teaming and mitigations done before release.

1. Atext/event-streamCorrect answer
2. Bmultipart/form-data
3. Capplication/grpc
4. Dapplication/json

Why: SSE responses use text/event-stream with data: lines delivered over a long-lived HTTP response.

1. ACounting tokens for billing
2. BRouting requests between regions
3. CGenerating images
4. DClassifying text and images for potentially harmful contentCorrect answer

Why: It returns category flags and scores so developers can screen inputs and outputs against policy.

1. AOnly prompt length
2. BSignals such as users switching models, preference rates and measured correctnessCorrect answer
3. CRandom assignment
4. DThe user's IP address

Why: OpenAI described training the router continuously on real signals including model switches, preferences and correctness.

1. ASkips safety checks to save time
2. BCaches answers across users
3. CGuesses the user's next prompt
4. DUses a small draft model to propose tokens that the large model verifies in parallelCorrect answer

Why: Verifying several drafted tokens in one forward pass of the large model cuts decode latency without changing the output distribution.

1. AIt compresses model weights
2. BIt bypasses rate limits
3. CIt stores the final answers
4. DIt reuses computation for repeated prompt prefixes, reducing prefill cost and latencyCorrect answer

Why: Reusing the KV state for an identical prefix avoids recomputing prefill for long, shared system prompts or documents.

1. ATo replace read replicas
2. BTo encrypt data at rest
3. CTo limit and reuse database connections, since each Postgres connection is costlyCorrect answer
4. DTo shard writes automatically

Why: Postgres uses a process per connection, so pooling prevents connection storms from exhausting the server.

1. AKey by request timestamp
2. BUse a single global key per endpoint
3. CKey by conversation title
4. DInclude the authenticated user or org ID in the key and check ownership on readCorrect answer

Why: Tenant-scoped keys plus an ownership check on read stop one user's data from being served to another.

1. AToken bucketCorrect answer
2. BConsistent hashing
3. CTwo-phase commit
4. DRound robin

Why: Token buckets allow controlled bursts while enforcing an average rate, and they map naturally to token budgets.

1. AThe tokens-per-minute estimate for that requestCorrect answer
2. BImage limits only
3. CNothing, because only actual tokens count
4. DRequests-per-minute only

Why: The requested maximum output is factored into the token estimate at admission, so setting it very high can trigger TPM limits early.

1. ARunning many unrelated models on one GPU
2. BEncrypting tensors in transit
3. CBatching users by region
4. DSplitting each layer's computation across several GPUs so large models fit and run fasterCorrect answer

Why: Sharding weight matrices across GPUs spreads memory and compute for models too large for one device.

1. AIt reviews capability and safeguard reports and recommends deployment decisions to leadershipCorrect answer
2. BIt sets API prices
3. CIt operates the GPU fleet
4. DIt writes the model code

Why: The framework has SAG review the evidence and advise leadership, with board-level oversight of the process.

1. ATo avoid running evals
2. BTo raise prices
3. CSo developers get stable behaviour and can migrate on their own scheduleCorrect answer
4. DBecause aliases cannot stream

Why: Snapshots protect production integrations from behaviour changes when the alias is updated to a newer model.

1. AAdd more read replicas
2. BCut unnecessary writes and move shardable write-heavy workloads to sharded storesCorrect answer
3. CDisable WAL
4. DRoute writes to replicas

Why: Replicas only scale reads, so write pressure needs fewer writes or a different, horizontally sharded store.

1. ACatching qualitative behaviour issues with willing testers before broad exposureCorrect answer
2. BAvoiding all evaluation
3. CReducing GPU cost permanently
4. DMarketing buzz

Why: After the sycophancy rollback, OpenAI said it would add opt-in alpha testing so problems surface early from users who expect rough edges.

1. ACancel generation to free GPU capacity and persist the partial reply as incompleteCorrect answer
2. BDelete the whole conversation
3. CRetry the request on another region
4. DKeep generating to the maximum length

Why: Cancelling frees decode slots for other users, and saving the partial reply keeps the conversation consistent.

1. AAggregate token counts for billing
2. BPrompt and completion contentCorrect answer
3. CThe status page
4. DPublic model documentation

Why: Zero data retention is about not storing request and response content, while minimal usage metadata is still needed for billing.