Companies blueprint · AI / Foundation models
Anthropic: from model training to the Claude API
Traces how a Claude model moves from data and pretraining through RLHF and Constitutional AI, Responsible Scaling Policy safety gates and staged release, to the Claude API serving requests with rate limits, prompt caching and usage monitoring. It must never ship a model whose measured capabilities exceed the safeguards in place, and it must never let one tenant's traffic or cached context affect another's.
- Steps
- 9
- Subjective questions
- 33
- Objective questions
- 33
The problem and the outcome
Problem: A frontier model is costly to train, can be misused, and must serve large numbers of developers reliably. The company needs one pipeline that ties capability gains to safety checks and still delivers fast, predictable API access.
Outcome: Each model release is gated by documented evaluations and ships with a public model card and usage policy. Developers get a stable Messages API with tiered rate limits, prompt caching for repeated context, and usage visibility, while misuse signals feed back into research.
Architecture
How it works, step by step
- Assemble training data from publicly described sources such as public web data, licensed data, data from users who opted in and synthetic data, then filter and deduplicate it.
- Pretrain a base model on large accelerator clusters (Anthropic has publicly described using cloud TPUs, AWS Trainium and GPUs), with research teams planning scale and tracking capability trends.
- Post-train with human feedback (RLHF) and AI feedback guided by a written constitution (Constitutional AI), shaping helpfulness, honesty and harmlessness, including Claude's character.
- Run safety and capability evaluations required by the Responsible Scaling Policy, including dangerous-capability tests in areas like CBRN and autonomy, with internal red teams and publicly described pre-deployment testing by external partners such as government AI safety institutes.
- Decide the AI Safety Level: if capabilities cross a threshold, the matching deployment and security safeguards must be in place before release, otherwise the model is mitigated or held back.
- Release in stages, from limited testers to general availability on the Claude API, Claude apps and cloud partners such as Amazon Bedrock and Google Cloud Vertex AI, publishing a model or system card and applying the usage policy.
- Serve developer requests through the Messages API, authenticating API keys and enforcing tiered rate limits on requests and tokens per minute, returning 429 with retry guidance when exceeded.
- Use prompt caching where the caller marks stable prefixes with cache_control, so repeated system prompts, tools or documents are read from cache, lowering latency and cost; then run inference and stream tokens.
- Record usage for the console and billing, and run monitoring and classifiers for policy violations; enforcement actions and misuse patterns feed back into safeguards and the next training cycle.
Tech stack
- Large-scale distributed training on cloud accelerators (TPU, Trainium, GPU)
- RLHF and Constitutional AI post-training
- Responsible Scaling Policy evaluations and ASL safeguards
- Claude Messages API with streaming, tool use and Batch API
- Prompt caching with cache_control breakpoints
- Tiered rate limiting (RPM, input and output tokens per minute)
- Console usage dashboards and admin API
- Safety classifiers and trust-and-safety tooling
Key design decisions
- Tie release to capability thresholds, not calendar dates: the RSP commits to safeguards matching the measured AI Safety Level before deployment, as when Claude Opus 4 was publicly released under ASL-3 protections.
- Use Constitutional AI so much of harmlessness feedback comes from a model applying written principles, which scales better than human labels alone and makes the values more inspectable.
- Make prompt caching explicit and opt-in via cache_control rather than implicit, so developers control what is cached and can reason about cost, with cache scoped to their organisation.
- Rate limit on tokens as well as requests, by usage tier, because token volume rather than request count drives GPU load.
- Publish model cards, the RSP and usage policies publicly, so customers and researchers can see evaluation results and limits for each release.
Common mistakes
- Treating evaluations as a one-time checkbox: capabilities can grow after fine-tuning or with better scaffolding, so evals need re-running and elicitation effort.
- Over-tuning for harmlessness until the model refuses benign requests; over-refusal must be measured alongside harmful compliance.
- Putting changing content such as timestamps before the cache breakpoint, so the prefix never matches and every request pays a cache write.
- Clients retrying 429s immediately without backoff, which amplifies load; honour retry-after and use exponential backoff with jitter.
- Assuming cloud-partner endpoints share identical limits, features or model availability with the first-party API; check each platform's documentation.
What to measure
- Dangerous-capability eval results versus RSP thresholds for each candidate model
- Harmful-compliance and over-refusal rates on internal and public benchmarks
- Time to first token and output tokens per second per model
- Prompt cache hit share (cache read input tokens versus total input tokens)
- Rate of 429 and 529 overloaded responses by tier and region
Most-asked interview questions
Subjective questions 33
1How would you design a release gate that stops a more capable model from shipping before its safeguards are ready?Asked almost every time
I would anchor it in a written policy like Anthropic's Responsible Scaling Policy. Define capability thresholds in risky domains, such as biology or autonomous replication, and map each to required deployment and security safeguards. Every candidate model runs a fixed evaluation suite with serious elicitation effort, plus red teaming and external testing. Results go to a decision owner who is separate from the product team, and the default if a threshold is crossed without safeguards is to hold or mitigate, not ship. I would log the decision, publish a model card, and re-run evals after significant post-training changes. The key design idea is that the gate is about measured capability, so a deadline cannot override it.
2Walk me through how a Claude model gets from raw data to serving a request on the Claude API.Asked almost every time
I'd split it into four planes. The data plane collects public web data, licensed data, opted-in user data and synthetic data, then filters and deduplicates it. The training plane pretrains a base model on large accelerator clusters and post-trains it with RLHF and Constitutional AI. The governance plane runs Responsible Scaling Policy evaluations, assigns an AI Safety Level and blocks release until the matching safeguards exist. The serving plane releases the model in stages, then handles Messages API traffic: key authentication, tiered rate limits, prompt caching, inference and streaming, usage metering and misuse monitoring. Enforcement findings feed back into safeguards and the next training cycle. Two invariants cut across every plane: capability never outruns safeguards, and tenants stay isolated.
3Rate limits apply to requests, input tokens and output tokens per minute. How do you enforce an output-token limit before the output exists?Asked almost every time
Anthropic documents its limits as token buckets, which refill continuously up to a maximum rather than resetting at fixed intervals. For output tokens, admission uses an estimate based on the request's max_tokens, then reconciles against actual output when the response completes. Input tokens can be counted at admission, and for most current models cached reads don't count toward the input-token limit. The buckets live in a low-latency distributed store keyed by organisation and model. To keep latency down I'd allow small, bounded over-admission rather than a strongly consistent global counter. A rejection returns 429 with a retry-after header and rate-limit headers describing remaining capacity.
4How do you guarantee that prompt caching never lets one tenant read or influence another tenant's cached context?Asked almost every time
Make tenant identity part of the cache key, not a check performed after lookup. The key combines the isolation scope, which Anthropic documents as at least the organisation, with the model and a hash of the exact prefix up to the breakpoint. Two tenants with byte-identical prompts therefore get different entries. There is no cross-tenant cache to probe, so no timing side channel. Cache storage is in-memory or short-lived, with TTL eviction. Entries are never written to training data or logs as content. I'd add tests that send identical prefixes from different organisations and assert there are no cache reads, plus audits on the key-construction code, since a bug there would be a cross-tenant data leak.
5Explain Constitutional AI and why a lab would use it alongside human feedback.Very often asked
Constitutional AI, described in Anthropic's 2022 paper, trains a model using a written set of principles. In a supervised phase, the model critiques and revises its own answers against those principles. In a reinforcement phase, a model compares pairs of responses according to the constitution, producing AI preference labels used to train a reward model, which is called RL from AI feedback. Human feedback is still valuable for helpfulness and nuance, but labelling large amounts of harmful content is slow, costly and hard on labellers. A constitution makes the target values explicit and editable, and it scales. The trade-off is that quality depends on the principles and on the judging model, so you still need human review and evaluations.
6How would you design rate limiting and prompt caching for a large LLM API?Very often asked
I would rate limit per organisation and tier on three axes: requests per minute, input tokens per minute and output tokens per minute, because GPU cost follows tokens. A token bucket at the gateway gives smooth enforcement, and over-limit calls get a 429 with a retry-after header, while overload returns a distinct error so clients can tell the difference. For caching, the client marks a stable prefix such as the system prompt, tools or a long document. The server hashes that exact prefix, scoped to the organisation, and stores the computed attention state with a short time-to-live. Matching requests skip recomputing it, cutting latency and cost. Usage responses report cache reads and writes separately, so developers can verify savings.
7How would you design the filtering and deduplication stage for a web-scale pretraining corpus?Very often asked
I'd run it as a batch pipeline over sharded object storage, with every stage writing versioned outputs so a run can be reproduced. Stage one handles provenance and policy: record source and licence, and honour crawler rules such as robots.txt and the exclusion of password-protected pages. Stage two is quality and safety filtering with heuristics and classifiers, plus PII handling. Stage three is deduplication: exact hashing for identical documents, then MinHash with locality-sensitive hashing for near-duplicates, run at document and paragraph level. Stage four decontaminates by removing overlap with held-out evaluation sets. I'd keep per-stage drop statistics and spot-check samples so that one filter bug can't silently remove whole domains or languages.
8How do you stop evaluation benchmarks from leaking into training data, and why does it matter for a safety gate?Very often asked
If evaluation items leak into training, scores go up through memorisation rather than real capability. For capability benchmarks that only flatters the model. For dangerous-capability evals it is worse, because it corrupts the measurement the release gate depends on. I'd register every eval set with canary strings and n-gram fingerprints, then scan the training corpus against them before training starts. Sensitive evals stay in access-controlled stores that are never crawled or exported. I'd also keep private, regularly refreshed held-out sets and compare performance on public and private variants. A large gap between them is a contamination signal and should trigger investigation before the result counts toward an ASL decision.
9A multi-week pretraining run on thousands of accelerators keeps losing nodes. How do you design for failure?Very often asked
At this scale, hardware failure is routine, so the design has to assume it. I'd take frequent asynchronous sharded checkpoints of weights, optimizer state, data-loader position and RNG state to durable storage, so a restart resumes deterministically. Health checks and straggler detection find bad nodes, and a scheduler swaps in hot spares and restarts from the last checkpoint automatically. I'd watch for loss spikes and NaNs, with a playbook to roll back to an earlier checkpoint, skip the offending data and adjust the learning rate. Checkpoint frequency is a trade-off: checkpointing too often costs throughput and storage bandwidth, while checkpointing too rarely wastes compute on every failure.
10How does the RLHF stage work, and how do you guard against reward hacking?Very often asked
Humans compare pairs of model responses. A reward or preference model learns to predict those preferences, and the policy is then optimised with RL against it, usually with a KL penalty that keeps it close to the supervised model. Reward hacking happens when the policy exploits weaknesses in the reward model: excessive length, sycophancy, confident-sounding but wrong answers, or gaming of test checks in coding tasks. Mitigations include a KL constraint, ensembles and periodic retraining of reward models on fresh comparisons, held-out human evaluation, and targeted evals for sycophancy and honesty. I'd also audit samples where reward is high but humans disagree, since those show exactly where the proxy diverges.
11Design the harness for dangerous-capability evaluations such as CBRN uplift and autonomy.Very often asked
I'd build an isolated, access-controlled evaluation environment, because both the questions and the transcripts are sensitive. Tasks cover knowledge probes, multi-step agentic tasks in sandboxes with tools, and expert-graded uplift studies that compare people with and without model access. Each eval is versioned, has a pre-registered threshold tied to an RSP capability threshold, and runs on the final candidate and on any snapshot meant for release. Results go into a capability report that records the elicitation used, such as scaffolding, tool access and best-of-n sampling. Sandboxes for autonomy tasks need network egress controls and monitoring. Nothing produced in them should be able to touch production systems.
12What kinds of safeguards does the ASL-3 standard involve?Very often asked
Anthropic's ASL-3 standard has two parts, which it publicly activated with Claude Opus 4 in 2025. Deployment safeguards aim to stop misuse for CBRN weapons. They include real-time constitutional classifiers on inputs and outputs, ongoing jailbreak monitoring, a bug bounty for universal jailbreaks, and rapid response to patch new attacks. Security safeguards aim to prevent theft of model weights. They include stronger access controls, two-party authorisation for sensitive access, and egress bandwidth controls that make large exfiltration hard to do and easy to detect. Both parts have to be judged adequate before deployment. A model that needs ASL-3 does not ship under ASL-2 protections just because the release date is near.
13How would you protect frontier model weights from theft, both by insiders and by external attackers?Very often asked
I'd treat the weights as a crown-jewel asset. Store them encrypted, with keys held in hardware-backed key management. Restrict access to a small, audited set of identities using least privilege, and require two-party approval for any sensitive operation, such as copying weights or accessing them in plaintext. Serving nodes load weights into hardened environments with no general interactive access. Egress bandwidth limits and monitoring on the networks that hold weights make a bulk copy slow and noisy. I'd add full audit logs, anomaly detection, supply-chain controls on serving software, and regular red-team exercises. Insider risk gets the same rigour as external risk, because one compromised credential shouldn't be enough.
14How would you design the staged release from limited testers to general availability, including rollback?Very often asked
Define stages as ring deployments: internal use, trusted testers under agreements, limited preview, then GA across the API, apps and cloud partners. Each ring has exit criteria: no unresolved severe safety findings, classifier and abuse metrics within bounds, error rates and latency within SLO, and enough capacity for the next ring. Model routing is controlled by configuration, so an organisation's access can be widened or pulled back without redeploying. Rollback means removing access to the new model ID or tightening safeguards. You don't silently swap weights behind a pinned model ID, because customers depend on stable behaviour. Release artefacts such as the system card and usage policy updates are finalised before GA, not after.
15How would you design API key authentication for the Messages API, including handling leaked keys?Very often asked
The client sends the key in the x-api-key header, alongside an anthropic-version header. At the edge, I'd hash the presented key and look it up in a replicated key store that maps it to organisation, workspace, tier and status, cached briefly so revocation propagates within seconds. Keys are shown once at creation, stored only as hashes, scoped to workspaces, and can be revoked from the console. Anthropic participates in secret-scanning programmes so that keys leaked to public repositories can be detected and deactivated. Every request is attributed to a key and organisation for rate limiting, billing and abuse investigation. Failed authentication returns 401 quickly, without touching inference capacity.
16What is the difference between a 429 and a 529 from the Claude API, and how should clients and the server behave?Very often asked
A 429 means this organisation exceeded its own rate limit. It is the caller's quota, and the fix is to slow down, respect retry-after, or move to a higher tier. A 529 overloaded_error means the service as a whole is temporarily overloaded, independent of the caller's quota. Clients should retry both with exponential backoff and jitter, cap total retries, and avoid synchronised retry storms. The official SDKs retry certain errors by default. On the server, load shedding should happen early and cheaply at admission, before tokens are processed, and priority or committed capacity can be protected while best-effort traffic is shed first.
17How is the cache key structured for prompt caching, and what invalidates a cache entry?Very often asked
The cacheable prefix is built in a fixed order: tools, then system, then messages, up to a block marked with cache_control. You can set up to four breakpoints. A hit requires the prefix to be identical up to that breakpoint, so any change earlier in the order invalidates everything after it. Editing a tool definition invalidates the system and message cache. Editing the system prompt invalidates the messages. Changes to some settings, such as images or tool_choice, can also break hits. Entries expire by TTL: five minutes by default, refreshed on each hit, with an optional one-hour TTL. The design advice is to put stable content first and volatile content last.
18How would you design token streaming, and what happens if the connection fails mid-stream?Very often asked
The Messages API streams over server-sent events with a typed sequence: message_start, content_block_start, repeated content_block_delta, content_block_stop, message_delta carrying the stop reason and usage, then message_stop, plus ping events. Typed blocks let text and tool-use JSON stream on the same channel. Errors can also arrive as an event mid-stream, after a 200 has been sent, so clients must handle in-stream error events. If the connection drops, the request does not resume. The client retries, and the prefix cache makes the retry cheaper. Server-side, generation should be cancelled when the client disconnects, to free capacity, and usage should be billed for the tokens actually processed.
19How would you design usage metering so billing and the console are accurate and never double-count?Very often asked
Each request has a unique request ID, returned to the caller in a response header. At completion, the serving layer emits one usage event with organisation, workspace, key, model, input tokens, cache-creation tokens, cache-read tokens and output tokens. Events go to a durable log with at-least-once delivery, and consumers deduplicate on request ID, which gives effectively-once aggregation. Real-time counters for the console can be approximate. Invoices come from the reconciled ledger. I'd keep pricing in versioned rate tables applied at aggregation time, so a price change never rewrites history. Daily reconciliation compares serving logs, rate-limit counters and the ledger to catch dropped or duplicated events.
20How do you monitor for policy violations at API scale while respecting user privacy?Very often asked
Use layers. Automated classifiers score traffic for high-severity categories in real time or near real time, and constitutional classifiers block certain CBRN content inline. Account-level signals, such as volume patterns and repeated flagged requests, feed trust-and-safety review, with enforcement that ranges from warnings to suspension. For understanding aggregate patterns, Anthropic has described Clio, which summarises and clusters conversations so analysts see aggregated trends rather than individual chats. Access to raw content is restricted, logged and justified case by case. Retention follows published policies and contractual terms such as zero-data-retention arrangements. I'd track classifier false positives carefully, because over-blocking legitimate developers is also a failure.
21How would research teams plan the scale of a pretraining run and track capability trends?Often asked
They fit scaling laws on smaller runs, which relate loss to parameters, data and compute, and use them to choose a compute-efficient model size and token budget for the target budget. During the run they track training loss and a suite of downstream evals at checkpoints, and compare the trajectory with the prediction. For safety, this trend tracking feeds the RSP: if checkpoints are on course to approach a capability threshold, the safeguards work has to start early rather than at release time. The key point is that the gate is only useful if capability is forecast ahead of time. Discovering a threshold crossing at the end leaves no time to build safeguards.
22Anthropic has described training and serving on Google TPUs, AWS Trainium and NVIDIA GPUs. What are the engineering trade-offs of that?Often asked
The upside is capacity and resilience. You aren't limited by one vendor's supply, you can place workloads where capacity and cost-efficiency are best, and you can serve close to cloud partners like Bedrock and Vertex AI. The cost is engineering complexity. Kernels, compilers, parallelism strategies and numerics differ by chip, so you need a portable model definition, per-backend optimisation, and strong numerical equivalence tests so the same weights behave the same on every platform. Performance tuning is duplicated, and debugging spans multiple toolchains. To manage this, I'd keep a single source of truth for model code, add hardware-specific backends behind it, and run cross-platform eval regression suites before any deployment.
23Why must dangerous-capability evaluations make a serious effort to elicit the model's best performance?Often asked
The gate asks whether the model could provide dangerous capability, not whether it did so under a lazy prompt. If you under-elicit, you understate capability and might deploy with too little protection. Real attackers will use good prompting, agent scaffolds, tools, fine-tuning where it's available and many attempts. So evaluators should use strong prompts, chain-of-thought, tool use, multiple samples and, where it makes sense, helpful-only variants without refusal training. They should also add a safety margin for capability gains likely to come from better scaffolding after release. The conservative principle is that uncertainty about whether a threshold is crossed should push you toward assuming it is crossed.
24The same model is offered on the Claude API, Amazon Bedrock and Google Cloud Vertex AI. How do you keep behaviour and safeguards consistent?Often asked
Treat the model as an immutable versioned artefact: the same weights, tokenizer and default configuration, identified by a dated snapshot ID. Every surface gets a conformance test suite that checks outputs, tool-use formats, stop behaviour and safety behaviour against reference results. Usage policies and acceptable-use requirements apply on every surface, though the partner platforms run their own authentication, quotas and billing. Feature availability, such as particular beta features, can lag on partner platforms. So I'd publish a feature matrix rather than assume parity. I'd also monitor cross-surface metrics, so a numerical or configuration drift on one platform shows up as an anomaly instead of waiting for a customer report.
25Where would cached prompt state live on the serving side, and what trade-offs does the TTL involve?Often asked
What gets reused is the attention key-value state computed for the prefix, so the model skips recomputing it. That state is large, so it lives in accelerator memory or a nearby fast tier, and routing needs to send requests to replicas that hold, or can quickly fetch, the prefix state. TTL is a cost and capacity trade-off. A longer TTL raises hit rates for bursty workloads but holds scarce memory longer. That is why the one-hour option has a higher write price than the five-minute default, and why the five-minute TTL refreshes on each hit. Eviction must be safe: a miss only costs recomputation, never correctness, so the cache can be dropped under pressure.
26Give a back-of-the-envelope capacity estimate approach for an LLM API endpoint.Often asked
I'd state assumptions explicitly rather than quote real figures. Start with peak requests per second, average input and output tokens per request, and expected cache hit rate. Prefill cost scales with uncached input tokens, and decode cost scales with output tokens. Decode is usually the bottleneck because it runs token by token. Then measure a single replica's sustainable throughput at the target latency, for example time to first token and tokens per second per stream, under realistic batching. Divide peak load by per-replica throughput and add headroom for failures, deployments and burst. Caching mainly reduces prefill, so a high hit rate raises effective capacity for long-context workloads more than for short chatty ones.
27How do enforcement findings and misuse patterns feed back into the next model, and what data governance applies?Often asked
Misuse findings mostly feed the safeguards first. New jailbreak patterns become classifier training data and red-team cases, and recurring harmful request types become eval items and post-training targets for the next model. Governance is strict about what can be used. By default, commercial API inputs and outputs are not used to train models. Consumer conversations are used only if the user opted in. Flagged content may be reviewed for trust and safety under published policy. In practice, I'd pass abstracted patterns and synthetic reproductions into training rather than raw customer data, tag every dataset with its provenance and allowed uses, and have the pipeline refuse data lacking a permitted-use tag.
28How should model versioning and deprecation work so customers aren't surprised by behaviour changes?Often asked
Publish immutable, dated snapshot IDs and, optionally, aliases that move to newer snapshots. Production customers should pin snapshots, because the snapshot's behaviour is fixed. Aliases suit experimentation. Changes to a model's behaviour ship as a new snapshot rather than as silent in-place updates, apart from safety-critical safeguard changes in the classifier layer. Deprecation follows a published schedule with advance notice, deprecation status in docs and console, and a retirement date after which requests fail clearly. Anthropic has also published commitments about preserving retired model weights. Internally, I'd track usage by model ID so the team can contact remaining users before retirement.
29Inline safety classifiers add latency and can conflict with streaming. How would you balance that trade-off?Often asked
Score the input before generation starts, since that is cheap relative to decoding and lets you block early. For outputs, run a streaming classifier over chunks as they are generated. If it fires, stop the stream and return a refusal instead of waiting for the full response. The classifier has to be fast and calibrated, so you tune thresholds per harm category. The aim is to block high-severity content such as CBRN uplift with high recall, while keeping false positives on legitimate science questions low. Apply the strictest inline checks only where the risk justifies them, rely on asynchronous monitoring for lower-severity policies, and measure added latency as a first-class SLO.
30How would you evaluate whether post-training produced the intended character, such as honesty and appropriate refusals?Sometimes asked
I'd turn the intended traits into concrete, testable behaviours and build an eval suite around them. Honesty evals check calibration, admitting uncertainty, and not inventing facts or citations. Sycophancy evals check whether the model changes a correct answer under user pressure. Refusal evals measure both sides: harmful requests should be declined, and benign but borderline requests should not be over-refused. Character consistency gets checked across long multi-turn conversations and personas. I'd combine automated graders with human review, track every metric per checkpoint, and treat a regression on either harmfulness or over-refusal as a release blocker. Optimising one side alone is the classic failure.
31How would you integrate pre-deployment testing by external partners such as government AI safety institutes without weakening security?Sometimes asked
Give them access through a controlled channel rather than handing over weights. For example, an API endpoint to a specific pre-release snapshot, with logging, rate limits and scoped credentials that expire after the testing window. Agree in advance on the domains they will test, the timeline and how findings are reported, so their results arrive in time to inform the ASL decision rather than after launch. Sensitive test content and transcripts stay protected on both sides. Findings go into the same tracking system as internal red-team issues, with owners and resolutions. The value is independent scrutiny. The risk is leakage of an unreleased model, so isolation and auditing matter.
32What belongs in a model or system card, and why publish one?Sometimes asked
A system card describes what the model is and how it was tested. That covers training data at a high level, intended uses, capability benchmark results, safety evaluations including dangerous-capability results and the AI Safety Level decision, red-teaming and external testing, alignment and welfare-related findings where relevant, and known limitations. Publishing it creates accountability. Outsiders can scrutinise the evidence behind a deployment decision, developers learn where the model is weak, and the lab commits to a record it can be held to. The engineering implication is that eval pipelines must produce reproducible, reportable results. If a number can't be traced back to a versioned eval run, it shouldn't appear in the card.
33When would you recommend the Message Batches API over real-time Messages calls?Sometimes asked
Use batches for large workloads that aren't latency-sensitive, such as classification, evaluation runs, document processing or data generation. The Batches API processes requests asynchronously, typically completing within 24 hours, and is priced at a discount compared with standard calls. From the provider's side, batches are easier to schedule into idle capacity. That lets you smooth load and protect real-time traffic. Batches can also be combined with prompt caching for shared prefixes. The trade-offs are no streaming, no interactive tool loops, and result retrieval that must handle partial failures per request. Real-time calls remain the right choice for chat, agents and anything user-facing.
Objective questions 33
1What is the default lifetime of a prompt cache entry on the Claude API?Asked almost every time
Why: The default ephemeral cache lasts five minutes and is refreshed on each hit. A longer one-hour TTL is available as an option at a higher write price.
2Evaluations show a model crosses a capability threshold, but the required safeguards are not ready. What does the RSP require?Asked almost every time
Why: The RSP's core commitment is that a model is not deployed without the safeguards matching its capability level.
3Under Anthropic's Responsible Scaling Policy, what determines whether a model can be deployed?Very often asked
Why: The RSP ties deployment to capability thresholds; a model that reaches a given AI Safety Level must have the corresponding safeguards before release.
4In Constitutional AI's reinforcement phase, where do most harmlessness preference labels come from?Very often asked
Why: RL from AI feedback uses a model guided by the constitution to compare responses, producing preference data that trains the reward model.
5In what order are request components combined to form the cacheable prefix?Very often asked
Why: The prefix is built as tools, then system, then messages. A change earlier in that order invalidates the cache for everything after it.
6Which response usage field tells you that tokens were served from the prompt cache?Very often asked
Why: cache_read_input_tokens counts tokens read from cache. cache_creation_input_tokens counts tokens written to cache on a miss.
7Which algorithm does Anthropic document for its API rate limits?Very often asked
Why: Anthropic's documentation says capacity replenishes continuously up to a maximum, which is the token bucket model, rather than resetting at fixed intervals.
8After receiving a 429, what should a well-behaved client do first?Very often asked
Why: The 429 includes a retry-after header, and backoff with jitter avoids synchronised retry storms. Using other tenants' keys would violate policy.
9What happens in the supervised phase of Constitutional AI?Very often asked
Why: In the supervised phase the model critiques and revises its own outputs against the constitution, and is then fine-tuned on the revised responses. The RL phase uses AI feedback.
10Which capability areas are central to the Responsible Scaling Policy's capability thresholds?Very often asked
Why: The RSP defines capability thresholds around CBRN weapons uplift and autonomous AI R&D, and those thresholds trigger stronger required safeguards.
11Which statement about prompt cache isolation is correct?Very often asked
Why: Anthropic documents that prompt caches are isolated between organisations, so identical prefixes from different tenants never produce cross-tenant hits.
12What is required for a cache hit?Very often asked
Why: Caching uses exact prefix matching, so even small changes before the breakpoint cause a miss.
13What transport does the Messages API use for streaming responses?Very often asked
Why: Streaming uses server-sent events with typed events such as message_start, content_block_delta and message_stop.
14By default, are inputs and outputs from commercial Claude API customers used to train Anthropic's models?Very often asked
Why: Anthropic's commercial terms state that it does not train on customer API data by default. Consumer data is used only when the user opts in.
15A developer's prompt caching shows almost no cache reads. What is the most likely cause?Often asked
Why: Caching matches an exact prefix, so any varying content before the cache_control breakpoint breaks the match and forces a fresh cache write each time.
16What does an HTTP 529 overloaded_error from the Claude API indicate?Often asked
Why: 529 signals service-wide overload. 429 is the per-organisation rate-limit response.
17How many cache_control breakpoints can a single request define?Often asked
Why: The API allows up to four cache breakpoints per request, which lets you cache tools, system and conversation segments separately.
18With which model did Anthropic first activate ASL-3 protections?Often asked
Why: In May 2025 Anthropic activated ASL-3 deployment and security standards alongside the launch of Claude Opus 4.
19A developer edits one tool description while keeping the system prompt identical. What happens to caching?Often asked
Why: Tools sit at the start of the prefix, so changing them alters every later part of the prefix and invalidates the system and message cache.
20Which headers authenticate and version a direct Claude API request?Often asked
Why: Direct API calls send the key in x-api-key and require the anthropic-version header.
21What best describes the Message Batches API?Often asked
Why: Batches are processed asynchronously, usually within 24 hours, at a discount compared with standard calls, which suits latency-tolerant bulk work.
22Why deduplicate pretraining data?Often asked
Why: Duplicates encourage memorisation, waste compute and can leak evaluation items. Deduplication improves data efficiency and makes measurement more trustworthy.
23Which is an example of an ASL-3 security safeguard for model weights?Often asked
Why: Anthropic described egress bandwidth controls and multi-party access controls among its ASL-3 weight-protection measures.
24What are constitutional classifiers used for in Claude's deployment safeguards?Often asked
Why: Constitutional classifiers are trained from a constitution of allowed and disallowed content and screen inputs and outputs. They are a key ASL-3 deployment safeguard.
25In RLHF, what is reward hacking?Often asked
Why: The policy optimises a proxy reward, so it can learn behaviours such as verbosity or sycophancy that the reward model overrates.
26How is a cache read priced relative to the base input token price, per Anthropic's public pricing?Often asked
Why: Cache reads are billed at a fraction of base input, published as 0.1x. Five-minute cache writes cost 1.25x base input.
27A request marks a 300-token system prompt with cache_control and never sees cache writes. Why?Often asked
Why: Each model has a minimum cacheable prompt length, for example 1,024 tokens for many models. Shorter prefixes are processed normally without caching.
28Usage events are delivered at-least-once. How do you stop customers from being double-billed?Often asked
Why: Idempotent consumers keyed on request ID turn at-least-once delivery into effectively-once aggregation without losing events.
29How do organisations usually get higher standard Claude API rate limits?Sometimes asked
Why: Rate limits increase automatically as organisations move up usage tiers, and custom limits are available through Anthropic's sales team.
30For most current Claude models, how do cache read tokens interact with the input-tokens-per-minute limit?Sometimes asked
Why: Anthropic documents that cached input tokens read from cache do not count toward ITPM for most models, so caching also increases effective throughput.
31What is the main safety benefit of staged release before general availability?Sometimes asked
Why: Staged rings limit the blast radius and give time to detect misuse and failures before broad deployment.
32Which organisations has Anthropic publicly described as doing pre-deployment testing of its models?Sometimes asked
Why: Anthropic has published that the US and UK government AI safety and security institutes tested models before deployment.
33What is Clio, as described by Anthropic?Sometimes asked
Why: Clio summarises and clusters conversations so analysts can study usage and misuse trends in aggregate without reading individual chats.
Track your progress through every blueprint in your dashboard.
Start learning