OneLinersCommand workbench
Guides
AI/ML / Software Engineering / Security

Build a Claude tool-use support agent with approval gates

Build a support agent that uses strict Claude tool schemas, validates every tool request again in application code, executes independent read-only calls safely, pauses writes for human approval, returns correctly ordered tool results, and recovers from refusals, timeouts, and partial failures.

210 min10 stepsChanges system stateRevision 1
Save or explore
Save to collectionCreate a collection in the sidebar first.
0 of 10 steps completed
Goal

Deliver a runnable agent loop where Claude proposes typed tool calls but never owns authorization or execution, parallel read-only work is bounded, every consequential action has a server-side approval record, and tool errors become explicit results rather than prompts for unsafe blind retries.

Supported environments
  • Node.js 22.22.2
  • Anthropic TypeScript SDK 0.115.0
  • Claude Messages API 2023-06-01, 2026-07-29
  • Claude model claude-sonnet-5
Prerequisites
  • Dedicated Claude workspace Use a server-side Claude workspace with a bounded spend limit, reviewed model access, workspace rate limits, least-privilege membership, and no credential reuse from unrelated workloads.
  • Synthetic evaluation fixtures Prepare authorized, non-sensitive examples plus malformed, ambiguous, adversarial, cancellation, and deletion cases before sending real user data.
  • Production trust policy Approve identity, authorization, authenticated support operator, authorized customer and ticket scope, bounded message history, allowlisted tool names, strict JSON input schemas, per-tool timeouts, read/write effect classification, approval record, and maximum loop depth, conversation and turn IDs, tool_use ID, tool name and schema revision, HMAC customer scope, validated arguments, effect class, approval decision and actor, execution status, sanitized result digest, stop reason, usage, and final response review state, retention, human review, cost, incident response, and rollback.
Operating boundary

OneLiners never runs these steps or stores secrets. Review placeholders, versions, current state, and change-control requirements before using a command.

Full guide

What you will build

System
  • A runnable approval-gated Claude support tool agent with a hostile-input boundary, server-side Claude adapter, explicit model and capability settings, strict output validation, evidence preservation, and no model-controlled external writes.
  • A complete operational path covering identity, authorization, idempotency, timeouts, cancellation, retries, privacy, retention, deletion, sanitized observability, evaluations, canary release, and schema-compatible rollback.
  • A review interface and evidence contract that preserve conversation and turn IDs, tool_use ID, tool name and schema revision, HMAC customer scope, validated arguments, effect class, approval decision and actor, execution status, sanitized result digest, stop reason, usage, and final response review state, distinguish observed data from model interpretation, and route uncertainty or incomplete states to a human instead of guessing.
Observable outcome
  • Normal, malformed, unauthorized, duplicate, cancelled, refused, incomplete, and provider-failed requests all reach explicit application states with stable identifiers and safe user messages.
  • No API key, webhook secret, private input, transcript, signed URL, raw provider error, or cross-tenant identifier appears in client code or ordinary logs.
  • The released artifact preserves conversation and turn IDs, tool_use ID, tool name and schema revision, HMAC customer scope, validated arguments, effect class, approval decision and actor, execution status, sanitized result digest, stop reason, usage, and final response review state, meets the frozen quality and latency thresholds, and can be deleted or reconciled across every retained copy.
  • Every model, prompt, schema, SDK, policy, or capability change is evaluated and canaried as an immutable revision with a working rollback path.

Architecture

How the parts fit together

An authenticated client submits a bounded request to the application, which authorizes tenant scope, validates authenticated support operator, authorized customer and ticket scope, bounded message history, allowlisted tool names, strict JSON input schemas, per-tool timeouts, read/write effect classification, approval record, and maximum loop depth, creates durable idempotent state, and calls Claude through one server adapter. The adapter enforces Messages tool definitions with strict input validation, explicit tool_choice policy, handling of one or many tool_use blocks, client-side execution, one correctly ordered user message containing all matching tool_result blocks, stop_reason control, and a hard iteration limit, validates the result, and persists only approved state. The client receives a normalized contract rather than raw provider objects. Evaluation and operations observe conversation and turn IDs, tool_use ID, tool name and schema revision, HMAC customer scope, validated arguments, effect class, approval decision and actor, execution status, sanitized result digest, stop reason, usage, and final response review state, quality, privacy, latency, usage, cost, failures, cancellation, retention, deletion, and drift. Consequential external actions remain outside the model session and require a separately authorized human-approved workflow.

Authenticated clientCollects consent, submits bounded input, presents progress and evidence, supports cancellation, and never receives provider secrets.
Application boundaryAuthorizes tenant resources, validates input, enforces quota and idempotency, sanitizes errors, and owns retry policy.
Claude adapterImplements Messages tool definitions with strict input validation, explicit tool_choice policy, handling of one or many tool_use blocks, client-side execution, one correctly ordered user message containing all matching tool_result blocks, stop_reason control, and a hard iteration limit with explicit models, limits, typed states, and privacy settings.
Evidence and state storePersists only approved lifecycle metadata and conversation and turn IDs, tool_use ID, tool name and schema revision, HMAC customer scope, validated arguments, effect class, approval decision and actor, execution status, sanitized result digest, stop reason, usage, and final response review state needed for recovery, review, deletion, and audit.
Evaluation and operationsGate revisions and monitor quality, security, privacy, reliability, latency, usage, cost, retention, and rollback.
  1. The client obtains consent and sends one idempotent bounded request under an authenticated tenant.
  2. The server validates identity, authorization, type, size, quota, policy revision, and retention before provider work.
  3. The Claude adapter runs only the reviewed capability and translates typed provider events or output into a strict application state.
  4. Deterministic validation checks schema, bounds, evidence, authorization, and invariants; uncertain results enter review.
  5. The UI presents safe content, progress, evidence, warnings, cancellation, and deletion without exposing raw provider objects.
  6. Operations reconcile incomplete state, measure the release gates, and can disable new work or restore the previous evaluated bundle.

Assumptions

  • The required Claude capability and model are available in the configured project, and current pricing, quotas, regional, and retention behavior are reviewed before release.
  • The application has real authentication for protected workloads and authorizes every tenant resource independently of client-supplied identifiers.
  • Production provides TLS, a trusted proxy configuration, shared rate limiting and idempotency, encrypted stores and backups, retention jobs, and incident response.
  • Synthetic fixtures can exercise the provider without uploading real secrets or regulated data during development and continuous verification.
  • Human reviewers understand that model output is probabilistic and can inspect the preserved evidence before a consequential decision.

Key concepts

tool_use
A Claude content block containing a tool name, unique ID, and JSON input. It is a request for the application to evaluate, not proof that the tool ran.
Strict tool use
Schema-constrained generation for supported tool definitions. The application still revalidates input, authenticates the actor, authorizes the resource, and enforces business invariants.
tool_result
A user-role content block matched to a tool call through tool_use_id. Parallel results are returned together and before any additional text in that user message.
Effect classification
Server-owned metadata identifying read-only, network, write, privileged, destructive, or consequential behavior; the model cannot downgrade this classification.
Approval record
A short-lived, authenticated, scope-bound decision tied to exact tool arguments and an immutable policy revision; changing arguments invalidates approval.

Before you copy

Values used in this guide

{{model}}

Exact evaluated model identifier for this capability; re-run evaluations before changing it.

Example: claude-sonnet-5
{{publicOrigin}}

Exact HTTPS browser origin allowed to create or modify application state.

Example: https://assistant.example.com
{{hmacSecret}}secret

Random deployment secret used for privacy-preserving user and abuse identifiers.

Example: replace-with-32-or-more-random-bytes

Security and production boundaries

  • Treat authenticated support operator, authorized customer and ticket scope, bounded message history, allowlisted tool names, strict JSON input schemas, per-tool timeouts, read/write effect classification, approval record, and maximum loop depth and all model output as untrusted. Validate before provider use, storage, rendering, tool execution, or a human decision.
  • Keep Claude credentials and callback secrets only in the server secret store, use project or workspace separation and budgets, rotate suspected exposures immediately, and never place them in fillable Guide variables.
  • Pseudonymous identifiers support abuse correlation but do not replace authentication or tenant authorization for protected data.
  • No model result, transcript, citation, structured object, or tool argument is inherently verified; preserve inspectable evidence and require human approval for consequential outcomes.

Stop before continuing if

  • Identity, tenant authorization, input ownership, consent, retention, deletion, or human-review policy is unresolved.
  • Secrets or private input appear in browser code, tracked files, ordinary logs, analytics, screenshots, support output, or test snapshots.
  • The client can expand model, tools, tenant resources, retention, domains, or output bounds beyond the server policy.
  • The product cannot preserve and independently inspect conversation and turn IDs, tool_use ID, tool name and schema revision, HMAC customer scope, validated arguments, effect class, approval decision and actor, execution status, sanitized result digest, stop reason, usage, and final response review state or distinguish observed values from model interpretation.
  • Critical authorization, privacy, evidence, deletion, cancellation, or rollback evaluations fail.
01

decision

Freeze the product and trust contract

read-only

Document who may use approval-gated Claude support tool agent, which inputs are accepted, what the model may infer, how conversation and turn IDs, tool_use ID, tool name and schema revision, HMAC customer scope, validated arguments, effect class, approval decision and actor, execution status, sanitized result digest, stop reason, usage, and final response review state is represented, and which outcomes require a human decision. Define success, refusal, retention, cost, latency, and deletion before creating provider objects.

Why this step matters

A provider call cannot compensate for an undefined product contract; measurable boundaries are required for security review, evaluation, support, and rollback.

What to understand

The input boundary is authenticated support operator, authorized customer and ticket scope, bounded message history, allowlisted tool names, strict JSON input schemas, per-tool timeouts, read/write effect classification, approval record, and maximum loop depth; anything outside it is rejected or routed to a separately reviewed workflow.

The output must preserve conversation and turn IDs, tool_use ID, tool name and schema revision, HMAC customer scope, validated arguments, effect class, approval decision and actor, execution status, sanitized result digest, stop reason, usage, and final response review state and label uncertainty instead of converting model confidence into a verification claim.

System changes

  • Creates a versioned product, privacy, evaluation, and human-approval policy.

Syntax explained

external_writes=blocked
The reference implementation cannot perform consequential actions.
Example output / evidence
contract=approved
artifact=approval-gated Claude support tool agent
external_writes=blocked
retention=reviewed
release=closed

Checkpoint: Approve the trust contract

Continue whenReviewers can state the allowed inputs, outputs, evidence, retention, cost, and escalation path without reading source code.

Stop whenStop if ownership, authorization, retention, or human approval for consequential outcomes is unresolved.

If this step fails

Teams disagree on whether an output is acceptable.

Likely causeThe product has no frozen evidence and refusal rubric.

Safe checks
  • Review the written acceptance examples

ResolutionResolve the rubric before implementation.

Stop conditions

  • Stop if ownership, authorization, retention, or human approval for consequential outcomes is unresolved.
02

config

Create a pinned server-side runtime

caution

Create a minimal Node.js service with the official SDK and schema validation pinned to tested versions. Load provider credentials only from the deployment secret store and keep generated content, uploads, transcripts, and fixtures outside tracked source.

Why this step matters

A small pinned dependency surface makes the example reproducible and keeps credentials and provider-specific behavior behind one auditable adapter.

What to understand

The browser or untrusted client talks only to the application; it never receives a reusable Anthropic API key.

Dependency, SDK, model, prompt, and schema updates are evaluated as independent revisions rather than silently floating.

System changes

  • Creates package.json, the dependency lock, source directories, ignored runtime data, and environment placeholders.

Syntax explained

private: true
Prevents accidental publication of the reference package to a public registry.
File package.json
Configuration
{
  "name": "claude-support-tool-agent",
  "private": true,
  "type": "module",
  "scripts": {
    "check": "node --check src/agent-loop.mjs",
    "smoke": "node --env-file=.env test/smoke.mjs",
    "test:failures": "node test/failures.mjs",
    "eval": "node eval/run.mjs"
  },
  "dependencies": {
    "@anthropic-ai/sdk": "0.115.0",
    "zod": "4.4.3"
  }
}
Example output / evidence
added 2 packages
0 vulnerabilities
server-side credentials: configured
tracked secrets: 0

Checkpoint: Inspect runtime and secret handling

Continue whenThe SDK imports on Node 22, package versions are pinned, and no secret or private artifact is tracked.

Stop whenStop if an API key, webhook secret, private input, signed URL, or transcript appears in source, browser code, logs, or fixtures.

If this step fails

The runtime uses a different SDK shape.

Likely causeDependencies were installed without the pinned lock or Node version.

Safe checks
  • node --version
  • npm ls @anthropic-ai/sdk zod

ResolutionRestore the reviewed runtime and rerun all contract tests.

Stop conditions

  • Stop if an API key, webhook secret, private input, signed URL, or transcript appears in source, browser code, logs, or fixtures.
03

instruction

Build a hostile-input intake boundary

caution

Authenticate the caller, authorize the tenant, validate type and size, reject malformed or unsupported content, assign an opaque request ID, and apply authenticated support operator, authorized customer and ticket scope, bounded message history, allowlisted tool names, strict JSON input schemas, per-tool timeouts, read/write effect classification, approval record, and maximum loop depth. Inputs remain untrusted even when they came from an employee, customer, scanner, microphone, or previously approved system.

Why this step matters

The intake boundary limits cost, cross-tenant exposure, parser abuse, prompt injection, and accidental processing before any model or storage object is created.

What to understand

Resolve tenant policy and provider configuration server-side; clients may narrow but never expand scope.

Store raw inputs only when the product has a documented need, encryption, retention, deletion, access logging, and incident process.

System changes

  • Adds authentication, authorization, validation, quarantine, idempotency, and request-size policy.

Syntax explained

opaque request ID
Supports correlation without exposing filenames, account identifiers, or content.
Example output / evidence
request=req_01 identity=user_hmac_… tenant=tenant-a type=approved size=bounded quarantine=passed

Checkpoint: Exercise invalid and cross-tenant inputs

Continue whenUnsupported, oversized, unauthorized, duplicate, and malformed requests fail before a Claude call.

Stop whenStop if the client can choose another tenant's resource, bypass validation, or cause raw private input to enter ordinary logs.

If this step fails

Invalid input reaches the model.

Likely causeValidation runs after upload or trusts metadata supplied by the client.

Safe checks
  • Trace one rejected fixture by request ID

ResolutionMove validation and authorization ahead of every provider and persistence action.

Stop conditions

  • Stop if the client can choose another tenant's resource, bypass validation, or cause raw private input to enter ordinary logs.
04

config

Implement the bounded provider contract

caution

Implement Messages tool definitions with strict input validation, explicit tool_choice policy, handling of one or many tool_use blocks, client-side execution, one correctly ordered user message containing all matching tool_result blocks, stop_reason control, and a hard iteration limit. Pin the evaluated model role, explicit instructions, output limit, storage choice, safety identifier, timeout, and typed terminal-state handling. Parse output by item type rather than array position and preserve provider request identifiers for support.

Why this step matters

The provider adapter is the single enforcement point for capabilities, model behavior, storage, cost, timeout, and the output shape accepted by the rest of the application.

What to understand

Treat refusal, incomplete, cancelled, failed, and completed as different states with different user messages and retry policy.

Never present syntactically valid structured output, a transcript, citation, or tool call as factually verified without separate evidence.

System changes

  • Creates the server-side Claude adapter and one bounded external request path.

Syntax explained

explicit model
Prevents a default change from silently altering quality, latency, or cost.
bounded output
Limits runaway generation but does not replace per-user and project budgets.
File src/agent-loop.mjs
Configuration
import Anthropic from "@anthropic-ai/sdk";
import { z } from "zod";

const anthropic = new Anthropic({ apiKey: process.env.ANTHROPIC_API_KEY });
const lookupSchema = z.object({
  customerId: z.string().regex(/^cus_[a-z0-9]+$/),
  fields: z.array(z.enum(["plan", "status", "open_cases"])).max(3)
}).strict();

const tools = [{
  name: "lookup_customer",
  description: "Read the current authorized customer's support-safe fields.",
  strict: true,
  input_schema: {
    type: "object",
    properties: {
      customerId: { type: "string", pattern: "^cus_[a-z0-9]+$" },
      fields: {
        type: "array",
        items: { enum: ["plan", "status", "open_cases"] },
        maxItems: 3
      }
    },
    required: ["customerId", "fields"],
    additionalProperties: false
  }
}];

export async function runAgent({ actor, messages }) {
  for (let turn = 0; turn < 6; turn += 1) {
    const response = await anthropic.messages.create({
      model: process.env.CLAUDE_MODEL,
      max_tokens: 1200,
      system: "Use tools only for the authorized customer. Never claim a tool succeeded without its result.",
      tools,
      messages
    });
    messages.push({ role: "assistant", content: response.content });
    if (response.stop_reason !== "tool_use") return response;

    const calls = response.content.filter((block) => block.type === "tool_use");
    const results = await Promise.all(calls.map(async (call) => {
      try {
        const input = lookupSchema.parse(call.input);
        await authorizeCustomer(actor, input.customerId);
        const value = await executeReadOnlyLookup(input, { timeoutMs: 3000 });
        return { type: "tool_result", tool_use_id: call.id, content: JSON.stringify(value) };
      } catch (error) {
        return {
          type: "tool_result",
          tool_use_id: call.id,
          is_error: true,
          content: sanitizeToolError(error)
        };
      }
    }));
    messages.push({ role: "user", content: results });
  }
  throw new Error("tool_iteration_limit");
}
Example output / evidence
request=req_tool_01 stop_reason=end_turn calls=1 parallel_groups=1 schema=passed authorization=passed approvals=0 tool_errors=0 review=passed

Checkpoint: Inspect one successful and one refused response

Continue whenBoth produce typed application states, sanitized messages, provider IDs, usage where available, and no leaked secrets.

Stop whenStop if output is parsed by guessed order, raw errors reach users, or the model gains a capability outside the approved contract.

If this step fails

The adapter crashes on a non-completed response.

Likely causeImplementation assumes every request returns the preferred output item.

Safe checks
  • Inspect status and item types with the request ID

ResolutionHandle every documented terminal state explicitly and fail closed on unknown items.

Stop conditions

  • Stop if output is parsed by guessed order, raw errors reach users, or the model gains a capability outside the approved contract.
05

config

Validate output and preserve evidence

caution

Validate application-facing output with a strict schema, enforce field and transcript bounds, preserve conversation and turn IDs, tool_use ID, tool name and schema revision, HMAC customer scope, validated arguments, effect class, approval decision and actor, execution status, sanitized result digest, stop reason, usage, and final response review state, distinguish observed values from model interpretation, and route low-confidence or incomplete results to review. Schema compliance proves shape only, not factual accuracy.

Why this step matters

A strict internal contract prevents arbitrary model fields from becoming trusted application state and keeps evidence available for independent review.

What to understand

Keep the original provider and application request identifiers even when user-facing content is deleted under policy.

Use deterministic checks for required evidence, bounds, authorization, and invariants before optional model-based quality scoring.

System changes

  • Adds schema validation, evidence normalization, review routing, and bounded persistence.

Syntax explained

strict()
Rejects undeclared fields instead of silently accepting an expanded model contract.
File src/validation.mjs
Configuration
import { z } from "zod";
export const resultSchema = z.object({
  status: z.enum(["completed", "needs_review", "refused", "failed"]),
  summary: z.string().max(8000),
  evidence: z.array(z.object({
    locator: z.string().max(500),
    note: z.string().max(1000)
  })).max(100),
  warnings: z.array(z.string().max(500)).max(20)
}).strict();
Example output / evidence
status=completed schema=passed evidence=6 warnings=1 review=false

Checkpoint: Run malformed and incomplete fixtures

Continue whenUnknown fields, missing evidence, excessive output, invalid states, and unsafe values are rejected or marked needs_review.

Stop whenStop if model output is written directly to a database, external system, HTML renderer, or command without validation and authorization.

If this step fails

A valid schema still contains a wrong conclusion.

Likely causeShape validation was mistaken for factual verification.

Safe checks
  • Compare evidence locators with the source or observed event

ResolutionSend the result to human review and strengthen evidence-based evaluation.

Stop conditions

  • Stop if model output is written directly to a database, external system, HTML renderer, or command without validation and authorization.
06

warning

Enforce privacy, retention, and deletion

caution

Tool arguments and results may expose ticket text, account metadata, or operational records. Fetch only fields authorized for the current operator, minimize result payloads before returning them to Claude, redact secrets, and retain approval and audit metadata separately from raw customer content. Document every copy created by the client, application, provider, cache, database, logs, analytics, backups, and exports. Implement authenticated deletion and retention jobs, and test that a successful UI action is not mistaken for complete erasure.

Why this step matters

AI workflows often create several independent copies; user trust and compliance fail when the product describes only the most visible database row.

What to understand

Minimize raw content in ordinary observability and use HMAC identifiers for abuse and correlation where supported.

When legal or backup retention prevents immediate erasure, state the exception honestly and prevent restored copies from silently reappearing.

System changes

  • Adds retention schedules, authenticated deletion, provider-object reconciliation, backup handling, and privacy documentation.

Syntax explained

store policy
Controls ordinary response storage but does not automatically delete files, sessions, logs, or backups.
Example output / evidence
retention_policy=approved primary_store=bounded provider_objects=tracked deletion_drill=passed backup_exception=documented

Checkpoint: Complete a deletion drill

Continue whenEvery primary object is deleted or has a documented pending/exception state, and the item cannot be retrieved by the former user.

Stop whenStop if the team cannot enumerate retained copies, authenticate deletion, or reconcile provider and application state.

If this step fails

Deleted content returns after restore or retry.

Likely causeBackups, caches, provider objects, or dead-letter payloads were outside the deletion workflow.

Safe checks
  • Trace the object inventory by opaque ID

ResolutionQuarantine restored data and extend deletion reconciliation to every copy.

Stop conditions

  • Stop if the team cannot enumerate retained copies, authenticate deletion, or reconcile provider and application state.
07

config

Add a real provider smoke test

caution

Use a non-sensitive deterministic fixture and a dedicated bounded project to prove the current model, API capability, output shape, evidence, and terminal-state handling. Log only identifiers, counts, and the final assertion label.

Why this step matters

Mocks prove application branching but cannot prove current provider permissions, media negotiation, file parsing, event order, webhook signatures, or model capability.

What to understand

Keep the fixture synthetic and cap time, tokens, tool use, file size, and retries.

A provider smoke failure blocks release but does not justify printing private payloads or raising limits without diagnosis.

System changes

  • Adds one controlled external API test with bounded cost and sanitized output.

Syntax explained

SMOKE_OK
A stable assertion label that reveals no user content.
File test/smoke.mjs
Configuration
import assert from "node:assert/strict";
import { runFixture } from "../src/test-harness.mjs";

const result = await runFixture("authorized-plan-lookup");
assert.equal(result.final.stopReason, "end_turn");
assert.equal(result.toolCalls.length, 1);
assert.equal(result.toolCalls[0].name, "lookup_customer");
assert.equal(result.toolCalls[0].authorized, true);
assert.match(result.final.text, /enterprise/i);
console.log("SMOKE_OK id=" + result.requestId + " evidence=4 artifact=claude-tool-agent");
Example output / evidence
SMOKE_OK id=req_01 evidence=3 artifact=approval-gated Claude support tool agent

Checkpoint: Run the provider smoke

Continue whenThe test verifies the expected capability and evidence, then prints one sanitized success record.

Stop whenStop if the smoke uses production data, logs content, lacks a timeout, or can pass without the critical provider capability.

If this step fails

The smoke test is flaky.

Likely causeThe fixture, network, event timing, model, or assertion is not deterministic enough.

Safe checks
  • Record model, request ID, duration, and typed status

ResolutionSeparate retryable transport checks from deterministic contract assertions and investigate before rerunning.

Stop conditions

  • Stop if the smoke uses production data, logs content, lacks a timeout, or can pass without the critical provider capability.
08

instruction

Evaluate quality, safety, and operations

read-only

Build representative cases for normal use, malformed input, ambiguity, prompt injection, authorization, privacy, cancellation, provider failure, and conversation and turn IDs, tool_use ID, tool name and schema revision, HMAC customer scope, validated arguments, effect class, approval decision and actor, execution status, sanitized result digest, stop reason, usage, and final response review state. Track deterministic pass/fail gates plus quality, latency, usage, refusal, review, and cost distributions by model and policy revision.

Why this step matters

A successful demo says little about adversarial inputs, long-tail failures, operational cost, or whether evidence supports the visible result.

What to understand

Require zero authorization and privacy failures before release; average quality cannot offset a critical leak.

Preserve frozen cases and compare one changed dimension at a time to explain regressions.

System changes

  • Creates a versioned evaluation set, release thresholds, and privacy-safe operational metrics.

Syntax explained

critical_failures=0
A hard gate for authorization, secret exposure, unsafe action, and unsupported verification claims.
Example output / evidence
cases=50 critical_failures=0 quality=0.96 evidence=0.98 p95_within_budget=true privacy_leaks=0

Checkpoint: Run the complete evaluation

Continue whenCritical gates pass and quality, evidence, latency, usage, refusal, and cost remain inside the approved band.

Stop whenStop on any cross-tenant access, secret exposure, unapproved action, evidence fabrication, or deletion regression.

If this step fails

Offline scores pass but production outcomes degrade.

Likely causeFixtures do not represent current traffic or monitoring omits evidence quality.

Safe checks
  • Sample redacted production categories
  • Compare revisions and drift

ResolutionExpand reviewed cases without storing raw sensitive content and restore the last evaluated revision.

Stop conditions

  • Stop on any cross-tenant access, secret exposure, unapproved action, evidence fabrication, or deletion regression.
09

verification

Exercise cancellation and dependency failures

caution

Simulate client disconnect, timeout, rate limit, invalid input, refusal, incomplete output, dependency unavailability, duplicate delivery, and partial persistence. Confirm each state has a safe message, stable identifier, retry rule, and reconciliation path.

Why this step matters

Ambiguous outcomes and retries are where duplicate actions, inconsistent state, hidden spend, and misleading success messages appear.

What to understand

Never retry a consequential or potentially completed operation until idempotency and authoritative state prove replay is safe.

User-facing messages remain useful but do not expose provider internals, secrets, private input, or stack traces.

System changes

  • Adds deterministic failure injection and reconciliation tests.

Syntax explained

reconciliation
Compares provider, application, and client state after an ambiguous or duplicated event.
Command
npm run test:failures
Example output / evidence
disconnect=passed timeout=passed rate_limit=passed refusal=passed duplicate=passed reconciliation=passed

Checkpoint: Review the failure matrix

npm run test:failures

Continue whenEvery failure reaches a bounded terminal or retryable state with no duplicate external effect or false success.

Stop whenStop if the system automatically replays ambiguous work or cannot identify and reconcile partial state.

If this step fails

A retry creates a duplicate artifact.

Likely causeIdempotency is missing or scoped to one process.

Safe checks
  • Inspect request and provider IDs
  • Check persistent unique constraints

ResolutionDisable automatic retries, reconcile state, and enforce durable idempotency.

Stop conditions

  • Stop if the system automatically replays ambiguous work or cannot identify and reconcile partial state.
10

verification

Canary, observe, and keep a rollback path

caution

Release the exact tested model, prompt, schema, policy, and application digest to a small authenticated audience. Monitor quality, evidence, failures, latency, cost, privacy, and deletion before widening access.

Why this step matters

A bounded canary limits the effect of current-provider, browser, data, and workload differences that cannot be fully represented offline.

What to understand

Version every behavioral dependency together and retain the previous schema-compatible bundle.

Rollback stops new work first while preserving authenticated status, deletion, reconciliation, and incident evidence.

System changes

  • Publishes one immutable evaluated revision to a bounded production audience.

Syntax explained

rollback=ready
Confirms a tested previous bundle and data reconciliation procedure exist before expansion.
Command
npm run check && npm run smoke && npm run eval
Example output / evidence
syntax=passed smoke=passed eval=passed canary=approved rollback=ready

Checkpoint: Approve production expansion

npm run check && npm run smoke && npm run eval

Continue whenCanary metrics and reviewed samples remain inside thresholds and the rollback drill succeeds.

Stop whenStop on a critical gate, unexpected provider behavior, evidence failure, privacy incident, or unexplained cost growth.

If this step fails

The canary must be reverted.

Likely causeA production-only dependency or behavior violated the release gate.

Safe checks
  • Freeze new work
  • Preserve sanitized revisions and identifiers

ResolutionRestore the previous evaluated bundle, reconcile in-flight state, and rerun affected cases.

Stop conditions

  • Stop on a critical gate, unexpected provider behavior, evidence failure, privacy incident, or unexplained cost growth.

Finish line

Verification checklist

Provider capability smokenpm run smokeA synthetic bounded request proves the current capability, evidence contract, typed status, and sanitized identifiers.
Security and failure suitenpm run test:failuresAuthorization, malformed input, duplicate delivery, cancellation, refusal, timeout, deletion, and dependency failures pass.
Quality and release evaluationnpm run evalCritical failures remain zero and quality, evidence, latency, usage, cost, privacy, and rollback remain inside thresholds.

Recovery guidance

Common problems and safe checks

The provider returns an unexpected terminal state.

Likely causeThe adapter handles only the happy path or the model/capability changed.

Safe checks
  • Inspect typed status and request ID without printing content

ResolutionFail closed, preserve sanitized evidence, and update the adapter only with a new evaluated revision.

The same user action creates duplicate work.

Likely causeThe idempotency key is regenerated or stored only in process memory.

Safe checks
  • Compare persistent request and provider identifiers

ResolutionStop retries, reconcile existing state, and enforce a durable unique key per intended action.

Latency or cost rises after a revision.

Likely causeModel, input size, output limit, capability, reasoning, retries, or workload mix changed.

Safe checks
  • Compare usage and p50/p95 by immutable revision

ResolutionRestore the previous evaluated bundle and change one bounded factor at a time.

Deletion succeeds in the UI but data remains elsewhere.

Likely causeProvider objects, caches, logs, backups, exports, or queues were not reconciled.

Safe checks
  • Trace the approved object inventory

ResolutionKeep deletion pending until every copy is removed or a documented exception applies.

After the procedure

Alternatives and next steps

Consider these alternatives

  • Use a simpler workflow without tool_use when the product cannot operate and evaluate that capability safely.
  • Use a simpler workflow without Strict tool use when the product cannot operate and evaluate that capability safely.
  • Use a synchronous, non-streaming, or manual-review workflow when it materially simplifies failure recovery and the latency tradeoff is acceptable.
  • Use local processing for classification, redaction, or extraction when policy prohibits sending the input to an external API.

Operate it safely

  • Replace synthetic identity with the product's verified authentication and tenant authorization before introducing protected data.
  • Expand the evaluation set from reviewed production categories without storing raw sensitive inputs or outputs.
  • Add privacy-safe dashboards, budget alerts, dependency health, retention reconciliation, deletion drills, and scheduled freshness reviews.
  • Introduce any external action only as a separate schema-validated, idempotent, human-approved tool with its own authorization and incident process.

Reference

Frequently asked questions

Does strict schema output make the result correct?

No. It guarantees application shape when supported, not factual accuracy. Verify evidence and route uncertainty to a reviewer.

Can the browser call Claude directly?

Not with a reusable Anthropic API key. Keep credentials and policy server-side; use only documented short-lived client credentials for capabilities that explicitly support them.

Should failed requests be retried automatically?

Only when durable idempotency and authoritative state prove replay cannot duplicate work. Ambiguous outcomes are reconciled first.

What must be logged?

Prefer opaque request/provider IDs, revision, status, duration, usage, evidence counts, and sanitized error classes. Avoid raw private content and secrets.

How often should this Guide be reviewed?

Every 90 days and whenever the model, SDK, API capability, pricing, retention, browser behavior, or security policy changes.

Recovery

Rollback

Disable new approval-gated Claude support tool agent work, preserve authenticated status and deletion controls, restore the previous evaluated application/model/prompt/schema/policy bundle, and reconcile every in-flight provider and local object. A code rollback does not delete retained inputs, outputs, sessions, files, logs, queues, backups, or exports.

  1. Activate the feature kill switch and reject new work while keeping authenticated read, cancellation, export, deletion, and incident status available.
  2. Record the failing application digest, model, prompt, schema, policy, request/provider identifiers, sanitized metrics, and affected object IDs.
  3. Restore the previous schema-compatible evaluated bundle; do not force an old binary onto state it cannot understand.
  4. Reconcile queued, active, cancelled, incomplete, failed, and completed work before allowing any retry.
  5. Rerun provider smoke, authorization, evidence, privacy, deletion, failure, and canary gates before reopening.

Evidence

Sources and review

Verified 2026-07-29Review due 2026-10-27
How tool use works | Claude PlatformofficialHandle tool calls | Claude PlatformofficialParallel tool use | Claude PlatformofficialTool reference | Claude PlatformofficialTypeScript SDK | Claude PlatformofficialClaude API errors | Claude PlatformofficialRate limits | Claude PlatformofficialAPI and data retention | Claude Platformofficial