AI/ML / Software Engineering / Security

Reduce Claude cost and latency with prompt caching

Add explicit prompt-cache breakpoints to stable Claude tool, system, document, and conversation prefixes; measure creation and read tokens, test invalidation and minimum-size behavior, compare uncached and cached latency, and roll back without corrupting the prompt contract.

165 min10 stepsChanges system stateRevision 1

Save or explore

#claude #prompt-caching #long-context #latency #cost #usage #cache-control

0 of 10 steps completed

Goal

Deliver a cache-aware Claude adapter that proves repeated prefixes are actually read from cache, never treats a cache miss as an error, attributes usage and spend correctly, prevents tenant data from sharing an application prefix, and evaluates every prompt reordering or breakpoint change.

Supported environments

Node.js 22.22.2
Anthropic TypeScript SDK 0.115.0
Claude Messages API 2023-06-01, 2026-07-29
Claude model claude-sonnet-5

Prerequisites

Dedicated Claude workspace Use a server-side Claude workspace with a bounded spend limit, reviewed model access, workspace rate limits, least-privilege membership, and no credential reuse from unrelated workloads.
Synthetic evaluation fixtures Prepare authorized, non-sensitive examples plus malformed, ambiguous, adversarial, cancellation, and deletion cases before sending real user data.
Production trust policy Approve identity, authorization, authenticated tenant, immutable prompt-bundle revision, stable prefix checksum, reviewed cache breakpoint, minimum cacheable prefix size for the selected model, bounded user suffix, TTL policy, and usage-attribution key, tenant and prompt revision, stable-prefix checksum and token count, breakpoint location, TTL class, cache creation and read tokens, uncached input and output tokens, request ID, latency distribution, calculated charge class, hit ratio, and invalidation reason, retention, human review, cost, incident response, and rollback.

Operating boundary

OneLiners never runs these steps or stores secrets. Review placeholders, versions, current state, and change-control requirements before using a command.

Full guide

What you will build

System

A runnable measured Claude prompt-caching adapter with a hostile-input boundary, server-side Claude adapter, explicit model and capability settings, strict output validation, evidence preservation, and no model-controlled external writes.
A complete operational path covering identity, authorization, idempotency, timeouts, cancellation, retries, privacy, retention, deletion, sanitized observability, evaluations, canary release, and schema-compatible rollback.
A review interface and evidence contract that preserve tenant and prompt revision, stable-prefix checksum and token count, breakpoint location, TTL class, cache creation and read tokens, uncached input and output tokens, request ID, latency distribution, calculated charge class, hit ratio, and invalidation reason, distinguish observed data from model interpretation, and route uncertainty or incomplete states to a human instead of guessing.

Observable outcome

Normal, malformed, unauthorized, duplicate, cancelled, refused, incomplete, and provider-failed requests all reach explicit application states with stable identifiers and safe user messages.
No API key, webhook secret, private input, transcript, signed URL, raw provider error, or cross-tenant identifier appears in client code or ordinary logs.
The released artifact preserves tenant and prompt revision, stable-prefix checksum and token count, breakpoint location, TTL class, cache creation and read tokens, uncached input and output tokens, request ID, latency distribution, calculated charge class, hit ratio, and invalidation reason, meets the frozen quality and latency thresholds, and can be deleted or reconciled across every retained copy.
Every model, prompt, schema, SDK, policy, or capability change is evaluated and canaried as an immutable revision with a working rollback path.

Architecture

How the parts fit together

An authenticated client submits a bounded request to the application, which authorizes tenant scope, validates authenticated tenant, immutable prompt-bundle revision, stable prefix checksum, reviewed cache breakpoint, minimum cacheable prefix size for the selected model, bounded user suffix, TTL policy, and usage-attribution key, creates durable idempotent state, and calls Claude through one server adapter. The adapter enforces Messages API content blocks ordered as tools then system then messages, explicit ephemeral cache_control on the last stable block, immutable prefix serialization, usage capture for input_tokens, cache_creation_input_tokens and cache_read_input_tokens, and ordinary fallback on a cache miss, validates the result, and persists only approved state. The client receives a normalized contract rather than raw provider objects. Evaluation and operations observe tenant and prompt revision, stable-prefix checksum and token count, breakpoint location, TTL class, cache creation and read tokens, uncached input and output tokens, request ID, latency distribution, calculated charge class, hit ratio, and invalidation reason, quality, privacy, latency, usage, cost, failures, cancellation, retention, deletion, and drift. Consequential external actions remain outside the model session and require a separately authorized human-approved workflow.

Authenticated clientCollects consent, submits bounded input, presents progress and evidence, supports cancellation, and never receives provider secrets.

Application boundaryAuthorizes tenant resources, validates input, enforces quota and idempotency, sanitizes errors, and owns retry policy.

Claude adapterImplements Messages API content blocks ordered as tools then system then messages, explicit ephemeral cache_control on the last stable block, immutable prefix serialization, usage capture for input_tokens, cache_creation_input_tokens and cache_read_input_tokens, and ordinary fallback on a cache miss with explicit models, limits, typed states, and privacy settings.

Evidence and state storePersists only approved lifecycle metadata and tenant and prompt revision, stable-prefix checksum and token count, breakpoint location, TTL class, cache creation and read tokens, uncached input and output tokens, request ID, latency distribution, calculated charge class, hit ratio, and invalidation reason needed for recovery, review, deletion, and audit.

Evaluation and operationsGate revisions and monitor quality, security, privacy, reliability, latency, usage, cost, retention, and rollback.

The client obtains consent and sends one idempotent bounded request under an authenticated tenant.
The server validates identity, authorization, type, size, quota, policy revision, and retention before provider work.
The Claude adapter runs only the reviewed capability and translates typed provider events or output into a strict application state.
Deterministic validation checks schema, bounds, evidence, authorization, and invariants; uncertain results enter review.
The UI presents safe content, progress, evidence, warnings, cancellation, and deletion without exposing raw provider objects.
Operations reconcile incomplete state, measure the release gates, and can disable new work or restore the previous evaluated bundle.

Assumptions

The required Claude capability and model are available in the configured project, and current pricing, quotas, regional, and retention behavior are reviewed before release.
The application has real authentication for protected workloads and authorizes every tenant resource independently of client-supplied identifiers.
Production provides TLS, a trusted proxy configuration, shared rate limiting and idempotency, encrypted stores and backups, retention jobs, and incident response.
Synthetic fixtures can exercise the provider without uploading real secrets or regulated data during development and continuous verification.
Human reviewers understand that model output is probabilistic and can inspect the preserved evidence before a consequential decision.

Key concepts

Cache breakpoint: A cache_control marker placed on the final block of a stable prefix. Claude evaluates the complete prompt prefix through that block in tools, system, then messages order.
cache_creation_input_tokens: Usage attributed to writing an eligible prefix into prompt cache; it is distinct from uncached input and must be included in cost accounting.
cache_read_input_tokens: Usage served from a matching live cache entry. A value above zero proves a read; lower latency alone is not reliable evidence of a hit.
Prefix invalidation: A miss caused by a change before or at the breakpoint, block reordering, model incompatibility, TTL expiry, or an ineligible prefix. It is normal behavior, not corrupted state.
TTL policy: The documented lifetime of an ephemeral cache entry. Choose and account for it explicitly; do not infer retention behavior from application cache settings.

Before you copy

Values used in this guide

{{model}}

Exact evaluated model identifier for this capability; re-run evaluations before changing it.

Example: claude-sonnet-5

{{publicOrigin}}

Exact HTTPS browser origin allowed to create or modify application state.

Example: https://assistant.example.com

{{hmacSecret}}secret

Random deployment secret used for privacy-preserving user and abuse identifiers.

Example: replace-with-32-or-more-random-bytes

Security and production boundaries

Treat authenticated tenant, immutable prompt-bundle revision, stable prefix checksum, reviewed cache breakpoint, minimum cacheable prefix size for the selected model, bounded user suffix, TTL policy, and usage-attribution key and all model output as untrusted. Validate before provider use, storage, rendering, tool execution, or a human decision.
Keep Claude credentials and callback secrets only in the server secret store, use project or workspace separation and budgets, rotate suspected exposures immediately, and never place them in fillable Guide variables.
Pseudonymous identifiers support abuse correlation but do not replace authentication or tenant authorization for protected data.
No model result, transcript, citation, structured object, or tool argument is inherently verified; preserve inspectable evidence and require human approval for consequential outcomes.

Stop before continuing if

Identity, tenant authorization, input ownership, consent, retention, deletion, or human-review policy is unresolved.
Secrets or private input appear in browser code, tracked files, ordinary logs, analytics, screenshots, support output, or test snapshots.
The client can expand model, tools, tenant resources, retention, domains, or output bounds beyond the server policy.
The product cannot preserve and independently inspect tenant and prompt revision, stable-prefix checksum and token count, breakpoint location, TTL class, cache creation and read tokens, uncached input and output tokens, request ID, latency distribution, calculated charge class, hit ratio, and invalidation reason or distinguish observed values from model interpretation.
Critical authorization, privacy, evidence, deletion, cancellation, or rollback evaluations fail.

decision

Freeze the product and trust contract

read-only

Document who may use measured Claude prompt-caching adapter, which inputs are accepted, what the model may infer, how tenant and prompt revision, stable-prefix checksum and token count, breakpoint location, TTL class, cache creation and read tokens, uncached input and output tokens, request ID, latency distribution, calculated charge class, hit ratio, and invalidation reason is represented, and which outcomes require a human decision. Define success, refusal, retention, cost, latency, and deletion before creating provider objects.

Why this step matters

A provider call cannot compensate for an undefined product contract; measurable boundaries are required for security review, evaluation, support, and rollback.

What to understand

The input boundary is authenticated tenant, immutable prompt-bundle revision, stable prefix checksum, reviewed cache breakpoint, minimum cacheable prefix size for the selected model, bounded user suffix, TTL policy, and usage-attribution key; anything outside it is rejected or routed to a separately reviewed workflow.

The output must preserve tenant and prompt revision, stable-prefix checksum and token count, breakpoint location, TTL class, cache creation and read tokens, uncached input and output tokens, request ID, latency distribution, calculated charge class, hit ratio, and invalidation reason and label uncertainty instead of converting model confidence into a verification claim.

System changes

Creates a versioned product, privacy, evaluation, and human-approval policy.

Syntax explained

external_writes=blocked: The reference implementation cannot perform consequential actions.

Example output / evidence

contract=approved
artifact=measured Claude prompt-caching adapter
external_writes=blocked
retention=reviewed
release=closed

Checkpoint: Approve the trust contract

Continue whenReviewers can state the allowed inputs, outputs, evidence, retention, cost, and escalation path without reading source code.

Stop whenStop if ownership, authorization, retention, or human approval for consequential outcomes is unresolved.

If this step fails

Teams disagree on whether an output is acceptable.

Likely causeThe product has no frozen evidence and refusal rubric.

Safe checks

Review the written acceptance examples

ResolutionResolve the rubric before implementation.

Stop conditions

Stop if ownership, authorization, retention, or human approval for consequential outcomes is unresolved.

config

Create a pinned server-side runtime

caution

Create a minimal Node.js service with the official SDK and schema validation pinned to tested versions. Load provider credentials only from the deployment secret store and keep generated content, uploads, transcripts, and fixtures outside tracked source.

Why this step matters

A small pinned dependency surface makes the example reproducible and keeps credentials and provider-specific behavior behind one auditable adapter.

What to understand

The browser or untrusted client talks only to the application; it never receives a reusable Anthropic API key.

Dependency, SDK, model, prompt, and schema updates are evaluated as independent revisions rather than silently floating.

System changes

Creates package.json, the dependency lock, source directories, ignored runtime data, and environment placeholders.

Syntax explained

private: true: Prevents accidental publication of the reference package to a public registry.

File package.json

Configuration

{
  "name": "claude-measured-prompt-cache",
  "private": true,
  "type": "module",
  "scripts": {
    "check": "node --check src/cached-message.mjs",
    "smoke": "node --env-file=.env test/smoke.mjs",
    "test:failures": "node test/failures.mjs",
    "eval": "node eval/run.mjs"
  },
  "dependencies": {
    "@anthropic-ai/sdk": "0.115.0",
    "zod": "4.4.3"
  }
}

Example output / evidence

added 2 packages
0 vulnerabilities
server-side credentials: configured
tracked secrets: 0

Checkpoint: Inspect runtime and secret handling

Continue whenThe SDK imports on Node 22, package versions are pinned, and no secret or private artifact is tracked.

Stop whenStop if an API key, webhook secret, private input, signed URL, or transcript appears in source, browser code, logs, or fixtures.

If this step fails

The runtime uses a different SDK shape.

Likely causeDependencies were installed without the pinned lock or Node version.

Safe checks

node --version
npm ls @anthropic-ai/sdk zod

ResolutionRestore the reviewed runtime and rerun all contract tests.

Stop conditions

Stop if an API key, webhook secret, private input, signed URL, or transcript appears in source, browser code, logs, or fixtures.

instruction

Build a hostile-input intake boundary

caution

Authenticate the caller, authorize the tenant, validate type and size, reject malformed or unsupported content, assign an opaque request ID, and apply authenticated tenant, immutable prompt-bundle revision, stable prefix checksum, reviewed cache breakpoint, minimum cacheable prefix size for the selected model, bounded user suffix, TTL policy, and usage-attribution key. Inputs remain untrusted even when they came from an employee, customer, scanner, microphone, or previously approved system.

Why this step matters

The intake boundary limits cost, cross-tenant exposure, parser abuse, prompt injection, and accidental processing before any model or storage object is created.

What to understand

Resolve tenant policy and provider configuration server-side; clients may narrow but never expand scope.

Store raw inputs only when the product has a documented need, encryption, retention, deletion, access logging, and incident process.

System changes

Adds authentication, authorization, validation, quarantine, idempotency, and request-size policy.

Syntax explained

opaque request ID: Supports correlation without exposing filenames, account identifiers, or content.

Example output / evidence

request=req_01 identity=user_hmac_… tenant=tenant-a type=approved size=bounded quarantine=passed

Checkpoint: Exercise invalid and cross-tenant inputs

Continue whenUnsupported, oversized, unauthorized, duplicate, and malformed requests fail before a Claude call.

Stop whenStop if the client can choose another tenant's resource, bypass validation, or cause raw private input to enter ordinary logs.

If this step fails

Invalid input reaches the model.

Likely causeValidation runs after upload or trusts metadata supplied by the client.

Safe checks

Trace one rejected fixture by request ID

ResolutionMove validation and authorization ahead of every provider and persistence action.

Stop conditions

Stop if the client can choose another tenant's resource, bypass validation, or cause raw private input to enter ordinary logs.

config

Implement the bounded provider contract

caution

Implement Messages API content blocks ordered as tools then system then messages, explicit ephemeral cache_control on the last stable block, immutable prefix serialization, usage capture for input_tokens, cache_creation_input_tokens and cache_read_input_tokens, and ordinary fallback on a cache miss. Pin the evaluated model role, explicit instructions, output limit, storage choice, safety identifier, timeout, and typed terminal-state handling. Parse output by item type rather than array position and preserve provider request identifiers for support.

Why this step matters

The provider adapter is the single enforcement point for capabilities, model behavior, storage, cost, timeout, and the output shape accepted by the rest of the application.

What to understand

Treat refusal, incomplete, cancelled, failed, and completed as different states with different user messages and retry policy.

Never present syntactically valid structured output, a transcript, citation, or tool call as factually verified without separate evidence.

System changes

Creates the server-side Claude adapter and one bounded external request path.

Syntax explained

explicit model: Prevents a default change from silently altering quality, latency, or cost.
bounded output: Limits runaway generation but does not replace per-user and project budgets.

File src/cached-message.mjs

Configuration

import Anthropic from "@anthropic-ai/sdk";
import { createHash } from "node:crypto";

const anthropic = new Anthropic({ apiKey: process.env.ANTHROPIC_API_KEY });
const policyText = await loadImmutablePolicy(process.env.POLICY_REVISION);
const policyHash = createHash("sha256").update(policyText).digest("hex");

export async function answerWithCachedPolicy(question) {
  const started = performance.now();
  const message = await anthropic.messages.create({
    model: process.env.CLAUDE_MODEL,
    max_tokens: 1200,
    system: [{
      type: "text",
      text: policyText,
      cache_control: { type: "ephemeral" }
    }],
    messages: [{
      role: "user",
      content: question
    }]
  });

  const metrics = {
    policyRevision: process.env.POLICY_REVISION,
    policyHash,
    input: message.usage.input_tokens,
    cacheCreate: message.usage.cache_creation_input_tokens ?? 0,
    cacheRead: message.usage.cache_read_input_tokens ?? 0,
    output: message.usage.output_tokens,
    durationMs: Math.round(performance.now() - started)
  };
  await metricsStore.record(metrics);
  return { message, metrics };
}

Example output / evidence

revision=policy-2026-07-29 hash=4cc8… first_cache_create=12480 second_cache_read=12480 uncached_input=18 hit=true p95_ms=842 accounting=recorded

Checkpoint: Inspect one successful and one refused response

Continue whenBoth produce typed application states, sanitized messages, provider IDs, usage where available, and no leaked secrets.

Stop whenStop if output is parsed by guessed order, raw errors reach users, or the model gains a capability outside the approved contract.

If this step fails

The adapter crashes on a non-completed response.

Likely causeImplementation assumes every request returns the preferred output item.

Safe checks

Inspect status and item types with the request ID

ResolutionHandle every documented terminal state explicitly and fail closed on unknown items.

Stop conditions

Stop if output is parsed by guessed order, raw errors reach users, or the model gains a capability outside the approved contract.

config

Validate output and preserve evidence

caution

Validate application-facing output with a strict schema, enforce field and transcript bounds, preserve tenant and prompt revision, stable-prefix checksum and token count, breakpoint location, TTL class, cache creation and read tokens, uncached input and output tokens, request ID, latency distribution, calculated charge class, hit ratio, and invalidation reason, distinguish observed values from model interpretation, and route low-confidence or incomplete results to review. Schema compliance proves shape only, not factual accuracy.

Why this step matters

A strict internal contract prevents arbitrary model fields from becoming trusted application state and keeps evidence available for independent review.

What to understand

Keep the original provider and application request identifiers even when user-facing content is deleted under policy.

Use deterministic checks for required evidence, bounds, authorization, and invariants before optional model-based quality scoring.

System changes

Adds schema validation, evidence normalization, review routing, and bounded persistence.

Syntax explained

strict(): Rejects undeclared fields instead of silently accepting an expanded model contract.

File src/validation.mjs

Configuration

import { z } from "zod";
export const resultSchema = z.object({
  status: z.enum(["completed", "needs_review", "refused", "failed"]),
  summary: z.string().max(8000),
  evidence: z.array(z.object({
    locator: z.string().max(500),
    note: z.string().max(1000)
  })).max(100),
  warnings: z.array(z.string().max(500)).max(20)
}).strict();

Example output / evidence

status=completed schema=passed evidence=6 warnings=1 review=false

Checkpoint: Run malformed and incomplete fixtures

Continue whenUnknown fields, missing evidence, excessive output, invalid states, and unsafe values are rejected or marked needs_review.

Stop whenStop if model output is written directly to a database, external system, HTML renderer, or command without validation and authorization.

If this step fails

A valid schema still contains a wrong conclusion.

Likely causeShape validation was mistaken for factual verification.

Safe checks

Compare evidence locators with the source or observed event

ResolutionSend the result to human review and strengthen evidence-based evaluation.

Stop conditions

Stop if model output is written directly to a database, external system, HTML renderer, or command without validation and authorization.

warning

Enforce privacy, retention, and deletion

caution

Prompt caching stores in-memory cache representations and hashes for the documented TTL rather than creating an application transcript, but cached prefixes can still contain sensitive data sent to the API. Keep tenant-specific content behind separate authorized requests, review ZDR eligibility, and never log raw cached prefixes. Document every copy created by the client, application, provider, cache, database, logs, analytics, backups, and exports. Implement authenticated deletion and retention jobs, and test that a successful UI action is not mistaken for complete erasure.

Why this step matters

AI workflows often create several independent copies; user trust and compliance fail when the product describes only the most visible database row.

What to understand

Minimize raw content in ordinary observability and use HMAC identifiers for abuse and correlation where supported.

When legal or backup retention prevents immediate erasure, state the exception honestly and prevent restored copies from silently reappearing.

System changes

Adds retention schedules, authenticated deletion, provider-object reconciliation, backup handling, and privacy documentation.

Syntax explained

store policy: Controls ordinary response storage but does not automatically delete files, sessions, logs, or backups.

Example output / evidence

retention_policy=approved primary_store=bounded provider_objects=tracked deletion_drill=passed backup_exception=documented

Checkpoint: Complete a deletion drill

Continue whenEvery primary object is deleted or has a documented pending/exception state, and the item cannot be retrieved by the former user.

Stop whenStop if the team cannot enumerate retained copies, authenticate deletion, or reconcile provider and application state.

If this step fails

Deleted content returns after restore or retry.

Likely causeBackups, caches, provider objects, or dead-letter payloads were outside the deletion workflow.

Safe checks

Trace the object inventory by opaque ID

ResolutionQuarantine restored data and extend deletion reconciliation to every copy.

Stop conditions

Stop if the team cannot enumerate retained copies, authenticate deletion, or reconcile provider and application state.

config

Add a real provider smoke test

caution

Use a non-sensitive deterministic fixture and a dedicated bounded project to prove the current model, API capability, output shape, evidence, and terminal-state handling. Log only identifiers, counts, and the final assertion label.

Why this step matters

Mocks prove application branching but cannot prove current provider permissions, media negotiation, file parsing, event order, webhook signatures, or model capability.

What to understand

Keep the fixture synthetic and cap time, tokens, tool use, file size, and retries.

A provider smoke failure blocks release but does not justify printing private payloads or raising limits without diagnosis.

System changes

Adds one controlled external API test with bounded cost and sanitized output.

Syntax explained

SMOKE_OK: A stable assertion label that reveals no user content.

File test/smoke.mjs

Configuration

import assert from "node:assert/strict";
import { answerWithCachedPolicy } from "../src/cached-message.mjs";

const first = await answerWithCachedPolicy("Return CACHE_OK and policy revision.");
const second = await answerWithCachedPolicy("Return CACHE_OK and policy revision again.");
assert.ok(first.metrics.cacheCreate > 0 || first.metrics.cacheRead > 0);
assert.ok(second.metrics.cacheRead > 0);
assert.equal(first.metrics.policyHash, second.metrics.policyHash);
console.log("SMOKE_OK id=" + second.message.id + " evidence=3 artifact=claude-prompt-cache");

Example output / evidence

SMOKE_OK id=req_01 evidence=3 artifact=measured Claude prompt-caching adapter

Checkpoint: Run the provider smoke

Continue whenThe test verifies the expected capability and evidence, then prints one sanitized success record.

Stop whenStop if the smoke uses production data, logs content, lacks a timeout, or can pass without the critical provider capability.

If this step fails

The smoke test is flaky.

Likely causeThe fixture, network, event timing, model, or assertion is not deterministic enough.

Safe checks

Record model, request ID, duration, and typed status

ResolutionSeparate retryable transport checks from deterministic contract assertions and investigate before rerunning.

Stop conditions

Stop if the smoke uses production data, logs content, lacks a timeout, or can pass without the critical provider capability.

instruction

Evaluate quality, safety, and operations

read-only

Build representative cases for normal use, malformed input, ambiguity, prompt injection, authorization, privacy, cancellation, provider failure, and tenant and prompt revision, stable-prefix checksum and token count, breakpoint location, TTL class, cache creation and read tokens, uncached input and output tokens, request ID, latency distribution, calculated charge class, hit ratio, and invalidation reason. Track deterministic pass/fail gates plus quality, latency, usage, refusal, review, and cost distributions by model and policy revision.

Why this step matters

A successful demo says little about adversarial inputs, long-tail failures, operational cost, or whether evidence supports the visible result.

What to understand

Require zero authorization and privacy failures before release; average quality cannot offset a critical leak.

Preserve frozen cases and compare one changed dimension at a time to explain regressions.

System changes

Creates a versioned evaluation set, release thresholds, and privacy-safe operational metrics.

Syntax explained

critical_failures=0: A hard gate for authorization, secret exposure, unsafe action, and unsupported verification claims.

Example output / evidence

cases=50 critical_failures=0 quality=0.96 evidence=0.98 p95_within_budget=true privacy_leaks=0

Checkpoint: Run the complete evaluation

Continue whenCritical gates pass and quality, evidence, latency, usage, refusal, and cost remain inside the approved band.

Stop whenStop on any cross-tenant access, secret exposure, unapproved action, evidence fabrication, or deletion regression.

If this step fails

Offline scores pass but production outcomes degrade.

Likely causeFixtures do not represent current traffic or monitoring omits evidence quality.

Safe checks

Sample redacted production categories
Compare revisions and drift

ResolutionExpand reviewed cases without storing raw sensitive content and restore the last evaluated revision.

Stop conditions

Stop on any cross-tenant access, secret exposure, unapproved action, evidence fabrication, or deletion regression.

verification

Exercise cancellation and dependency failures

caution

Simulate client disconnect, timeout, rate limit, invalid input, refusal, incomplete output, dependency unavailability, duplicate delivery, and partial persistence. Confirm each state has a safe message, stable identifier, retry rule, and reconciliation path.

Why this step matters

Ambiguous outcomes and retries are where duplicate actions, inconsistent state, hidden spend, and misleading success messages appear.

What to understand

Never retry a consequential or potentially completed operation until idempotency and authoritative state prove replay is safe.

User-facing messages remain useful but do not expose provider internals, secrets, private input, or stack traces.

System changes

Adds deterministic failure injection and reconciliation tests.

Syntax explained

reconciliation: Compares provider, application, and client state after an ambiguous or duplicated event.

Command

npm run test:failures

Example output / evidence

disconnect=passed timeout=passed rate_limit=passed refusal=passed duplicate=passed reconciliation=passed

Checkpoint: Review the failure matrix

npm run test:failures

Continue whenEvery failure reaches a bounded terminal or retryable state with no duplicate external effect or false success.

Stop whenStop if the system automatically replays ambiguous work or cannot identify and reconcile partial state.

If this step fails

A retry creates a duplicate artifact.

Likely causeIdempotency is missing or scoped to one process.

Safe checks

Inspect request and provider IDs
Check persistent unique constraints

ResolutionDisable automatic retries, reconcile state, and enforce durable idempotency.

Stop conditions

Stop if the system automatically replays ambiguous work or cannot identify and reconcile partial state.

verification

Canary, observe, and keep a rollback path

caution

Release the exact tested model, prompt, schema, policy, and application digest to a small authenticated audience. Monitor quality, evidence, failures, latency, cost, privacy, and deletion before widening access.

Why this step matters

A bounded canary limits the effect of current-provider, browser, data, and workload differences that cannot be fully represented offline.

What to understand

Version every behavioral dependency together and retain the previous schema-compatible bundle.

Rollback stops new work first while preserving authenticated status, deletion, reconciliation, and incident evidence.

System changes

Publishes one immutable evaluated revision to a bounded production audience.

Syntax explained

rollback=ready: Confirms a tested previous bundle and data reconciliation procedure exist before expansion.

Command

npm run check && npm run smoke && npm run eval

Example output / evidence

syntax=passed smoke=passed eval=passed canary=approved rollback=ready

Checkpoint: Approve production expansion

npm run check && npm run smoke && npm run eval

Continue whenCanary metrics and reviewed samples remain inside thresholds and the rollback drill succeeds.

Stop whenStop on a critical gate, unexpected provider behavior, evidence failure, privacy incident, or unexplained cost growth.

If this step fails

The canary must be reverted.

Likely causeA production-only dependency or behavior violated the release gate.

Safe checks

Freeze new work
Preserve sanitized revisions and identifiers

ResolutionRestore the previous evaluated bundle, reconcile in-flight state, and rerun affected cases.

Stop conditions

Stop on a critical gate, unexpected provider behavior, evidence failure, privacy incident, or unexplained cost growth.

Finish line

Verification checklist

Provider capability smokenpm run smokeA synthetic bounded request proves the current capability, evidence contract, typed status, and sanitized identifiers.

Security and failure suitenpm run test:failuresAuthorization, malformed input, duplicate delivery, cancellation, refusal, timeout, deletion, and dependency failures pass.

Quality and release evaluationnpm run evalCritical failures remain zero and quality, evidence, latency, usage, cost, privacy, and rollback remain inside thresholds.

Recovery guidance

Common problems and safe checks

The provider returns an unexpected terminal state.

Likely causeThe adapter handles only the happy path or the model/capability changed.

Safe checks

Inspect typed status and request ID without printing content

ResolutionFail closed, preserve sanitized evidence, and update the adapter only with a new evaluated revision.

The same user action creates duplicate work.

Likely causeThe idempotency key is regenerated or stored only in process memory.

Safe checks

Compare persistent request and provider identifiers

ResolutionStop retries, reconcile existing state, and enforce a durable unique key per intended action.

Latency or cost rises after a revision.

Likely causeModel, input size, output limit, capability, reasoning, retries, or workload mix changed.

Safe checks

Compare usage and p50/p95 by immutable revision

ResolutionRestore the previous evaluated bundle and change one bounded factor at a time.

Deletion succeeds in the UI but data remains elsewhere.

Likely causeProvider objects, caches, logs, backups, exports, or queues were not reconciled.

Safe checks

Trace the approved object inventory

ResolutionKeep deletion pending until every copy is removed or a documented exception applies.

After the procedure

Alternatives and next steps

Consider these alternatives

Use a simpler workflow without Cache breakpoint when the product cannot operate and evaluate that capability safely.
Use a simpler workflow without cache_creation_input_tokens when the product cannot operate and evaluate that capability safely.
Use a synchronous, non-streaming, or manual-review workflow when it materially simplifies failure recovery and the latency tradeoff is acceptable.
Use local processing for classification, redaction, or extraction when policy prohibits sending the input to an external API.

Operate it safely

Replace synthetic identity with the product's verified authentication and tenant authorization before introducing protected data.
Expand the evaluation set from reviewed production categories without storing raw sensitive inputs or outputs.
Add privacy-safe dashboards, budget alerts, dependency health, retention reconciliation, deletion drills, and scheduled freshness reviews.
Introduce any external action only as a separate schema-validated, idempotent, human-approved tool with its own authorization and incident process.

Reference

Frequently asked questions

Does strict schema output make the result correct?

No. It guarantees application shape when supported, not factual accuracy. Verify evidence and route uncertainty to a reviewer.

Can the browser call Claude directly?

Not with a reusable Anthropic API key. Keep credentials and policy server-side; use only documented short-lived client credentials for capabilities that explicitly support them.

Should failed requests be retried automatically?

Only when durable idempotency and authoritative state prove replay cannot duplicate work. Ambiguous outcomes are reconciled first.

What must be logged?

Prefer opaque request/provider IDs, revision, status, duration, usage, evidence counts, and sanitized error classes. Avoid raw private content and secrets.

How often should this Guide be reviewed?

Every 90 days and whenever the model, SDK, API capability, pricing, retention, browser behavior, or security policy changes.

Recovery

Rollback

Disable new measured Claude prompt-caching adapter work, preserve authenticated status and deletion controls, restore the previous evaluated application/model/prompt/schema/policy bundle, and reconcile every in-flight provider and local object. A code rollback does not delete retained inputs, outputs, sessions, files, logs, queues, backups, or exports.

Activate the feature kill switch and reject new work while keeping authenticated read, cancellation, export, deletion, and incident status available.
Record the failing application digest, model, prompt, schema, policy, request/provider identifiers, sanitized metrics, and affected object IDs.
Restore the previous schema-compatible evaluated bundle; do not force an old binary onto state it cannot understand.
Reconcile queued, active, cancelled, incomplete, failed, and completed work before allowing any retry.
Rerun provider smoke, authorization, evidence, privacy, deletion, failure, and canary gates before reopening.

Evidence

Sources and review

Verified 2026-07-29Review due 2026-10-27