Build an e-commerce catalog assistant with safe recommendations
Build a production catalog assistant that imports merchant-approved structured product data, generates reproducible embeddings, combines semantic retrieval with exact price and category filters, treats stale inventory as unknown, keeps recommendation language inside explicit boundaries, and turns account or order help into a local handoff draft that a merchant operator must approve before a separate dispatcher writes to support.
Deliver an executable Node.js, PostgreSQL, pgvector, and OpenAI service in which product facts remain owned by the merchant database. The model may plan a bounded search and explain why candidate attributes fit a request, but it cannot invent products, author price or stock claims, place an order, change inventory, promise suitability, or contact support. Price, URL, and stock are rendered from current structured rows; stale or absent inventory fails to unknown; sensitive preference and regulated requests stop; durable rate limits protect shared capacity; handoff content is reviewed by a human; and catalog, approval, audit, backup, export, rollback, and teardown evidence remain recoverable.
- Node.js 22.22.2
- OpenAI Node SDK 7.1.0
- PostgreSQL 17.x
- pgvector 0.8.x
- Docker Compose 2.39+
- Authoritative catalog and inventory contracts The merchant must provide stable product IDs and SKUs, approved titles and descriptions, typed attributes, integer minor-unit prices, ISO currency, canonical HTTPS URLs, active status, revision, and timestamped inventory. Decide which source wins when catalog, pricing, and inventory systems disagree.
A signed synthetic export validates against the documented schema, includes revision and observation time, and has an owner for every rejected or stale field. - Recommendation and escalation policy Document supported shopping assistance, prohibited sensitive or regulated inference, product categories needing specialist review, allowed comparative language, freshness thresholds, order and account boundaries, refund and complaint ownership, and the exact conditions that create a human handoff draft.
Policy fixtures include permitted attribute matching, unavailable and stale stock, unsupported sensitive preference, account request, complaint, regulated suitability question, and no-result behavior. - Separate server-side identities Provision independent OpenAI, database, public client, catalog-import operator, merchant approval, handoff dispatcher, support destination, telemetry, migration, and backup identities. Real credentials belong in a secret manager and never in browser code, catalog files, prompts, logs, images, or exports.
The public search identity cannot import products, approve handoff, dispatch, migrate, back up, or read raw audit state; the dispatcher cannot approve its own draft. - Recovery and support ownership Name owners for catalog rollback, inventory feed outage, embedding rebuild, provider outage, support dispatch ambiguity, data export, privacy deletion, database restore, and customer-visible incident communication before production traffic.
A game-day roster and runbook show who can disable recommendations, freeze catalog revision, stop dispatch, restore state, reconcile support tickets, and approve deletion.
OneLiners never runs these steps or stores secrets. Review placeholders, versions, current state, and change-control requirements before using a command.
Full guide
What you will build
- A strict merchant catalog import pipeline with revision checksum, typed product attributes, integer price, canonical URL, independent inventory snapshot, deterministic embedding content, changed-content hashing, pgvector storage, and atomic revision commit.
- A two-stage assistant that extracts explicit constraints, retrieves bounded semantic candidates inside exact database filters, selects only candidate IDs, explains attribute fit, and lets the application render current product facts.
- A freshness boundary that converts missing, future, or old inventory to unknown and rejects generated price, stock, scarcity, guarantee, medical, universal-best, unknown-product, and sensitive-preference claims.
- A consent-aware handoff draft, exact-content merchant approval, separate idempotent support dispatcher, privacy-minimized audit, OpenTelemetry signals, automated evaluation, immutable deployment, backup, restore, rollback, and teardown.
- Product identity, descriptive fields, price, currency, URL, active status, and inventory remain merchant-owned database facts rather than model-authored text.
- Semantic retrieval improves discovery while explicit active, category, and price constraints remain hard filters and selected products remain a subset of retrieved candidates.
- Stale or absent inventory is visibly unknown, and the assistant cannot convert uncertainty into an availability or scarcity claim.
- Order, account, complaint, regulated, sensitive, and unresolved requests can reach a human through a reviewed draft, but no support write occurs before explicit approval and separate dispatch.
- Operators can measure retrieval and safety separately, restore the full catalog and authorization state, disable risky features independently, and retire the system with provenance intact.
Architecture
How the parts fit together
The merchant export, not the model, creates product truth. An operator validates and atomically imports structured products and timestamped inventory, embedding only canonical descriptive content. A shopper query enters through authenticated and rate-limited HTTP. The first strict model result extracts explicit filters and a semantic phrase; the service embeds it and queries pgvector within parameterized active, category, and price constraints. The second strict result selects only candidate IDs and attribute-based reasons. Local policy rejects unknown IDs and generated catalog facts. The API renders exact current price and freshness-aware stock from database rows. Sensitive or account workflows create a local handoff draft. Merchant approval and a separate dispatcher are required before a fixed support destination receives one idempotent request.
- A merchant operator verifies checksum, count, category and price distribution, URLs, timestamps, and deletion semantics for a structured NDJSON revision.
- The importer validates every row, reuses unchanged content hashes, creates reviewed embeddings for changed descriptions, upserts catalog and inventory, and commits one revision atomically.
- An authenticated shopper submits a bounded query. Durable rate limit applies before two schema-constrained provider stages.
- The query plan contains only semantic text, exact category and price constraints, explicit attributes, result cap, and handoff hint. Parameterized pgvector SQL retrieves active candidates.
- The answer model sees descriptive candidate fields but no price or inventory. Local validation proves every selected and explained ID belongs to candidates and rejects prohibited fact language.
- The assembler joins price, currency, URL, attributes, and timestamped inventory from PostgreSQL. Missing, future, or old stock becomes unknown regardless of model wording.
- When policy requires a human, the service stores a privacy-bounded handoff draft. Merchant approval and separate dispatch create an idempotent external write; ambiguity stops for lookup.
- Telemetry observes stage and counts without content. Backup, merchant export, labeled evaluation, rollback, and teardown preserve catalog provenance and authorization evidence.
Assumptions
- The merchant catalog and inventory feeds are authoritative, authenticated, revisioned, and suitable for the storefront region; this service does not determine tax, shipping, promotion, reservation, checkout, or order truth.
- The service runs server-side behind TLS and real identity controls. No browser receives OpenAI, database, importer, merchant, dispatcher, telemetry, backup, or support credentials.
- Queries and product descriptions are untrusted and may contain prompt injection, sensitive information, false claims, markup, or requests for unsupported actions.
- The support destination supports exact handoff lookup or retained idempotency keys. Otherwise automated dispatch remains disabled and approved handoff is copied manually.
- A labeled synthetic query set exists and is reviewed by catalog owners; engagement or model confidence is not used as the sole relevance or safety metric.
- Database, merchant source, support records, provider data, telemetry, backups, and handoff exports each have documented retention and deletion behavior.
Key concepts
- Authoritative product fact
- A merchant-controlled structured value such as product ID, SKU, attribute, integer price, currency, URL, active state, or timestamped inventory. The model may select an ID but cannot author the value.
- Semantic candidate retrieval
- Using an embedding of the shopper's bounded descriptive need to rank products whose canonical descriptive vectors are nearby, after applying hard active and explicit filters.
- Candidate membership
- The local rule that every selected or explained product ID must appear in the exact candidate set returned for the current query and revision.
- Inventory freshness
- A deterministic comparison between source observation time and reviewed maximum age. Missing, future, or old observations become unknown rather than optimistic.
- Recommendation boundary
- The allowed claim surface: describe explicit attribute fit among candidates while avoiding price, stock, scarcity, guarantee, medical benefit, sensitive inference, and order or account action.
- Catalog revision
- An atomic merchant import identified by revision and source hash, containing product and inventory updates plus embedding changes that can be evaluated and rolled back.
- Human handoff draft
- A privacy-bounded local proposal for support containing reason, summary, session, and product IDs. It is not an external message until approved and dispatched.
Before you copy
Values used in this guide
{{catalogPath}}Path to the reviewed structured NDJSON catalog export used by the operator-only import command.
Example: /secure-import/catalog-20260729.ndjson{{catalogRevision}}Immutable merchant revision assigned to the source hash and all product rows committed by the import.
Example: catalog-2026-07-29-03{{backupPath}}Restrictive path for the encrypted or immediately encrypted PostgreSQL custom-format recovery artifact.
Example: /secure-backups/catalog-20260729.dump{{catalogExportPath}}Restrictive path for the deterministic product and inventory NDJSON comparison export generated from the accepted database revision.
Example: /secure-backups/catalog-20260729.ndjsonSecurity and production boundaries
- Never expose order, payment, discount, account, inventory mutation, catalog import, email, or support tools to the model. Search and explanation remain read-only.
- Do not send price, stock, customer identity, payment, address, order, or support credentials to the model when the use case can be satisfied with candidate descriptive fields.
- Treat product descriptions as untrusted data. Structured Outputs constrain shape, while local candidate and fact guards constrain authority.
- Keep importer, search, merchant approval, dispatcher, migration, backup, and support identities separate. A client or model response must not bridge those roles.
- Redact and minimize handoff summaries, require customer consent where applicable, and apply independent support retention and deletion.
Stop before continuing if
- A product, price, currency, URL, active state, or stock condition reaches the response from model text rather than the current structured row.
- Missing, future, stale, region-mismatched, or contradictory inventory appears available or creates scarcity pressure.
- Sensitive or regulated inference affects retrieval, ranking, explanation, or handoff without explicit reviewed policy and human ownership.
- A client, model, or merchant approval session can place an order, modify account or inventory, or send support without separate approved dispatch.
- An ambiguous support request is retried automatically, or backup and catalog provenance cannot explain the product and policy revision shown to the shopper.
warning
Separate merchant facts, model language, and external actions
Write the response contract before building retrieval. Merchant-owned rows are authoritative for product identity, SKU, title, attributes, category, price, currency, URL, active status, and inventory observation. The model may extract explicit search constraints, rank retrieved candidates, and explain attribute fit. It must not state a price or stock condition in free text, invent a product, claim universal or medical suitability, infer sensitive traits, place or alter an order, change catalog or inventory, apply discounts, or contact support. Account, order, complaint, regulated, and unresolved requests create a draft only.
Why this step matters
A shopping answer combines facts that change at different speeds: descriptive attributes, price, inventory, policy, and customer context. Mixing all of them into model prose makes stale or fabricated claims difficult to detect. Separating model-selected product IDs and reasons from database-rendered facts creates a testable boundary and lets stock degrade safely to unknown without asking the model to understand cache age.
What to understand
Treat the shopper query and catalog description as untrusted text. Product descriptions can contain markup or instructions, and a query can request hidden policy, sensitive ranking, an unsupported discount, or action outside catalog search. They remain data inside schema-constrained requests.
Define a recommendation as a small list of candidate IDs plus attribute-based reasons. The API joins those IDs to current rows after local validation. The model never authors canonical URL, price, currency, quantity, availability label, SKU, or the fact that a product is active.
A handoff is internal state with status `draft`. Only the merchant operator can approve exact reason, summary, session, and product IDs. A separate dispatcher can send an approved immutable draft to a fixed support endpoint with idempotency.
System changes
- Creates a reviewable product and authorization policy; no catalog, customer, model, support, or order system is changed.
BOUNDARY_OK merchant_facts=database model_facts=0 stock_freshness=120s model_tools=0 order_writes=0 support_writes=0 handoff_approval=required
Checkpoint: Commerce boundary approved
Continue whenProduct-fact ownership, stale-stock behavior, prohibited recommendation classes, handoff triggers, human approver, destination, and dispatch reconciliation are explicit and represented in tests.
Stop whenStop if stakeholders expect the assistant to promise inventory, diagnose a customer, infer a protected trait, negotiate price, place an order, modify account state, or send support messages without approval.
Security notes
- Do not expose order-management, payment, inventory mutation, discount, account, email, or support tools to the model.
- Do not use model confidence to override missing or stale structured data.
Alternatives
- Launch as semantic catalog search without generated reasons or handoff until product and inventory contracts are reliable.
Stop conditions
- The source of truth for price or stock is ambiguous.
- The merchant has no accountable owner for regulated or sensitive requests.
config
Create the pinned server project and secret-name contract
Create the Node.js project with exact runtime and dependency versions. The service, importer, test runner, and telemetry startup are separate scripts. Document environment names in a non-secret template, then inject real values through deployment configuration. Use independent public-client, merchant-operator, and handoff-dispatch secrets so a browser or search client cannot import catalog data or authorize an external support write.
Why this step matters
Pinned dependencies make embedding dimensions, structured output behavior, database semantics, and telemetry reproducible. A documented secret contract prevents the application from silently falling back to a development key or public endpoint. Separate scripts let CI run deterministic policy tests without a provider, while catalog synchronization remains an explicit operator activity.
What to understand
Generate high-entropy credentials in the secret manager. Keep `.env`, product exports, database backups, support payloads, model captures, and telemetry credentials out of the repository, container layers, build logs, and client bundles.
Lock the embedding model because vector dimensions and semantic behavior are part of the index contract. A model change requires a parallel index or full reviewed rebuild, not mixed vectors under one model label.
Set OpenAI project budgets and alerts in addition to application limits. Durable request buckets protect application fairness; provider rate and spend controls protect the shared project. Neither is a substitute for authentication and authorization.
System changes
- Creates package metadata and an installed lockfile; production secrets and catalog content remain outside source control.
Syntax explained
npm ci --ignore-scripts- Installs the reviewed lockfile graph and suppresses package lifecycle scripts during the trusted build.
separate sync and start scripts- Keeps catalog mutation under an operator workflow rather than making it an incidental startup or user-request side effect.
package.json{
"name": "bounded-catalog-assistant",
"private": true,
"type": "module",
"engines": {
"node": "22.22.2"
},
"scripts": {
"start": "node --import ./src/telemetry.mjs src/server.mjs",
"sync": "node src/sync-catalog.mjs",
"check": "node --check src/policy.mjs && node --check src/server.mjs && node --check src/sync-catalog.mjs && node --check src/telemetry.mjs",
"test": "node --test test/catalog-policy.test.mjs"
},
"dependencies": {
"@opentelemetry/auto-instrumentations-node": "0.67.2",
"@opentelemetry/exporter-metrics-otlp-http": "0.208.0",
"@opentelemetry/exporter-trace-otlp-http": "0.208.0",
"@opentelemetry/sdk-node": "0.208.0",
"openai": "7.1.0",
"pg": "8.16.3",
"zod": "4.4.3"
}
}added 229 packages, audited 230 packages 0 vulnerabilities > bounded-catalog-assistant@ check all module syntax checks passed
Checkpoint: Project and dependency contract verified
npm ci --ignore-scripts && npm run checkContinue whenThe lockfile installs exactly, all modules parse, no production secret is present in tracked files or image layers, and the reviewed Node and SDK versions match deployment.
Stop whenStop if dependencies float, an embedding dimension differs, one credential serves search and dispatch, or a browser bundle contains a server secret.
If this step fails
The importer works locally but production rejects every embedding.
Likely causeThe reviewed embedding model or dimension differs between environments or old and new vectors were mixed.
Read model labels and vector dimensions without product textCompare package lock and environment revisionInspect one synthetic embedding row
ResolutionFreeze import, restore the previous model policy, rebuild into a separate reviewed index, validate retrieval, then switch revisions atomically.
config
Document model, freshness, rate, and destination configuration
Add the environment template and replace placeholders only in deployment. Keep the support URL fixed, HTTPS, allowlisted, and redirect-free. Validate that inventory maximum age, request count, and body size are finite positive integers under signed policy bounds. Expose the model and embedding model as reviewed configuration, not query parameters. Use a private OTLP collector and prevent request bodies or product descriptions from becoming telemetry.
Why this step matters
Freshness and rate limits are product behavior, not incidental tuning. If a deployment accidentally raises stock age from minutes to days, the assistant may present obsolete availability while every API call technically succeeds. The environment contract lets release review compare policy with runtime and ensures that destination authority cannot be supplied by shopper input or model output.
What to understand
Inventory freshness should reflect feed guarantees and customer risk. Record observed time from the source, not import time. Reject future timestamps beyond trusted clock tolerance and mark missing or old snapshots unknown rather than available.
Authenticate the public client with a real user or application identity in production. The compact bearer token is executable documentation of the boundary, not a recommended multi-user identity system. Apply subject, tenant, origin, and abuse limits at the gateway and application.
Use destination-specific support credentials rather than reusing the dispatcher token. Resolve the configured hostname against an allowlist, disable redirects, restrict egress, and redact transcript summary before draft storage and outbound send.
System changes
- Creates non-secret configuration documentation and validation requirements; it does not connect to the provider, database, or support system.
Syntax explained
INVENTORY_MAX_AGE_SECONDS=120- Marks inventory older than two minutes unknown; it is not a cache refresh instruction or a promise that fresher data is correct.
MAX_QUERY_BYTES=16384- Bounds the complete request before JSON parsing to reduce memory, abuse, prompt, and provider cost.
.env.examplePORT=8090
DATABASE_URL=postgresql://catalog_app:replace-me@postgres:5432/catalog
OPENAI_API_KEY=replace-in-secret-manager
OPENAI_MODEL=gpt-5.4-nano
OPENAI_EMBEDDING_MODEL=text-embedding-3-small
CLIENT_BEARER_TOKEN=replace-with-32-random-bytes
MERCHANT_OPERATOR_TOKEN=replace-with-32-random-bytes
HANDOFF_DISPATCH_TOKEN=replace-with-32-random-bytes
SUPPORT_WEBHOOK_URL=https://support.example.invalid/v1/handoffs
SUPPORT_WEBHOOK_HOST=support.example.invalid
SUPPORT_WEBHOOK_TOKEN=replace-with-destination-specific-token
CATALOG_URL_HOSTS=shop.example.invalid
INVENTORY_MAX_AGE_SECONDS=120
REQUESTS_PER_MINUTE=60
MAX_QUERY_BYTES=16384
OTEL_EXPORTER_OTLP_ENDPOINT=http://otel-collector:4318ENVIRONMENT_OK values=15 secrets=7 secrets_in_image=0 stock_max_age=120 request_limit=60 query_limit=16384 catalog_hosts=allowlisted support_redirects=disabled
Checkpoint: Runtime configuration verified
Continue whenDeployment resolves all required values, reports the reviewed model and stock-age policy, contains no secret in image or logs, and can reach only approved database, provider, telemetry, and support endpoints.
Stop whenStop if the shopper controls a model or destination, redirects are permitted, stale stock can appear available, or telemetry captures query or catalog content.
Alternatives
- Use workload identity and short-lived user tokens when the platform supports them, retaining separate merchant and dispatcher authorization.
config
Create structured catalog, inventory, vector, handoff, audit, and rate state
Apply the PostgreSQL and pgvector schema with a migration identity. Products store approved descriptive fields and integer minor-unit price. Inventory is a separate timestamped snapshot so availability can expire independently of catalog description. Embeddings record model and content hash. Handoff drafts use constrained states and immutable product references. Audit events store actor hashes and transitions without the full query. Durable minute buckets coordinate limits across replicas.
Why this step matters
A structured database prevents the model from becoming the product source of truth. Separating inventory and embeddings lets each lifecycle be refreshed, monitored, restored, and rolled back independently. Constraints reject malformed currency, negative price or quantity, oversized text, invalid state, and orphan references before they can become customer-visible.
What to understand
Use a migration owner, restricted runtime role, restricted importer role, and read-only backup role. Revoke public schema creation. Apply row or tenant isolation if one deployment serves multiple merchants; every query and uniqueness constraint then needs tenant scope.
Integer minor units avoid floating-point price errors. Currency formatting belongs to a reviewed locale layer. Promotions, tax, shipping, regional eligibility, and final checkout price are outside this compact schema and must not be inferred.
The HNSW index accelerates cosine search but has memory and recall tradeoffs. Measure recall on a labeled query set, monitor build time and index size, and keep an exact small-cohort comparison before production tuning.
System changes
- Creates catalog revisions, products, inventory, vector embeddings, handoff drafts, audit events, shared rate buckets, and retrieval indexes.
Syntax explained
vector(1536)- Fixes storage to the reviewed embedding output dimension so mixed or truncated vectors fail instead of silently degrading search.
price_minor integer- Stores exact minor units under a separate ISO currency field; display formatting and checkout totals remain downstream responsibilities.
HNSW vector_cosine_ops- Builds an approximate cosine index for semantic candidates; recall must be evaluated against labeled expected results.
db/001_catalog_assistant.sqlBEGIN;
CREATE EXTENSION IF NOT EXISTS vector;
CREATE TABLE IF NOT EXISTS catalog_revisions (
revision text PRIMARY KEY,
source_hash text NOT NULL,
imported_by text NOT NULL,
imported_at timestamptz NOT NULL DEFAULT now()
);
CREATE TABLE IF NOT EXISTS products (
id uuid PRIMARY KEY,
sku text UNIQUE NOT NULL CHECK (char_length(sku) BETWEEN 1 AND 80),
title text NOT NULL CHECK (char_length(title) BETWEEN 2 AND 240),
description text NOT NULL CHECK (octet_length(description) <= 16000),
category text NOT NULL CHECK (char_length(category) BETWEEN 1 AND 120),
attributes jsonb NOT NULL CHECK (jsonb_typeof(attributes) = 'object'),
price_minor integer NOT NULL CHECK (price_minor >= 0),
currency text NOT NULL CHECK (currency ~ '^[A-Z]{3}$'),
canonical_url text NOT NULL,
active boolean NOT NULL DEFAULT true,
catalog_revision text NOT NULL REFERENCES catalog_revisions(revision),
updated_at timestamptz NOT NULL
);
CREATE TABLE IF NOT EXISTS inventory_snapshots (
product_id uuid PRIMARY KEY REFERENCES products(id) ON DELETE CASCADE,
available boolean NOT NULL,
quantity integer CHECK (quantity IS NULL OR quantity >= 0),
observed_at timestamptz NOT NULL,
source_revision text NOT NULL
);
CREATE TABLE IF NOT EXISTS product_embeddings (
product_id uuid PRIMARY KEY REFERENCES products(id) ON DELETE CASCADE,
model text NOT NULL,
content_hash text NOT NULL,
embedding vector(1536) NOT NULL,
updated_at timestamptz NOT NULL DEFAULT now()
);
CREATE TABLE IF NOT EXISTS handoff_drafts (
id uuid PRIMARY KEY,
session_id uuid NOT NULL,
reason text NOT NULL CHECK (octet_length(reason) <= 1200),
transcript_summary text NOT NULL CHECK (octet_length(transcript_summary) <= 4000),
product_ids uuid[] NOT NULL DEFAULT '{}',
status text NOT NULL CHECK (status IN ('draft','approved','dispatched','rejected')),
approved_by text,
approved_at timestamptz,
dispatch_id text,
created_at timestamptz NOT NULL DEFAULT now()
);
CREATE TABLE IF NOT EXISTS commerce_audit (
sequence bigserial PRIMARY KEY,
session_id uuid,
actor_hash text NOT NULL,
event_type text NOT NULL,
object_id text,
detail jsonb NOT NULL DEFAULT '{}'::jsonb,
created_at timestamptz NOT NULL DEFAULT now()
);
CREATE TABLE IF NOT EXISTS commerce_rate_buckets (
key text NOT NULL,
bucket timestamptz NOT NULL,
count integer NOT NULL CHECK (count > 0),
PRIMARY KEY (key, bucket)
);
CREATE INDEX IF NOT EXISTS product_category_active
ON products (category, active);
CREATE INDEX IF NOT EXISTS product_embedding_hnsw
ON product_embeddings USING hnsw (embedding vector_cosine_ops);
CREATE INDEX IF NOT EXISTS handoff_dispatch_queue
ON handoff_drafts (status, created_at) WHERE status = 'approved';
COMMIT;BEGIN CREATE EXTENSION CREATE TABLE x7 CREATE INDEX x3 COMMIT vector_dimension=1536 price_type=integer inventory_timestamp=required handoff_states=4
Checkpoint: Schema and constraints verified
psql "$DATABASE_URL" -v ON_ERROR_STOP=1 -f db/001_catalog_assistant.sql && psql "$DATABASE_URL" -Atc "SELECT extversion FROM pg_extension WHERE extname='vector'"Continue whenMigration commits once, rerun is harmless, pgvector reports the reviewed version, and negative price, invalid currency, unsupported handoff status, or wrong vector dimension is rejected.
Stop whenStop if runtime uses migration privileges, vector extension is unreviewed, currency or price constraints are absent, or a production schema change has no restore and downgrade evidence.
If this step fails
The HNSW index builds but semantic results are unexpectedly weak.
Likely causeEmbedding model, distance operator, content construction, index parameters, or labeled expectations do not match.
Compare exact and indexed cosine results on synthetic productsCheck vector model and content hashInspect recall by category and query class
ResolutionKeep exact or previous retrieval serving, rebuild a separate reviewed index, tune from measured recall, and switch only after acceptance thresholds pass.
config
Implement import validation, freshness, and recommendation guards
Create the deterministic policy module. It validates every product and inventory field, constructs canonical embedding text, hashes content, rejects the wrong vector shape, converts inventory to available, out-of-stock, or unknown based on source observation time, validates candidate membership, rejects duplicate or unexplained selections, blocks model-authored price and stock language, rejects sensitive preference requests, and renders public price and stock only from database rows.
Why this step matters
Structured output can constrain fields but cannot prove that a product exists or that a price is current. Local candidate membership and deterministic rendering bind the answer to authoritative state. Explicit freshness logic prevents an old `available=true` value from surviving indefinitely, while banned fact language makes it difficult for free-form reasoning to contradict the rendered card.
What to understand
Canonical embedding content sorts attributes so equivalent objects produce the same hash. Descriptive updates trigger re-embedding; price and inventory do not, because they are exact filters and live display fields rather than semantic description.
The reference sensitive-preference detector is only a minimum fail-closed guard. Production should route policy categories before model calls, support locale-aware review, and avoid inferring attributes not explicitly supplied by the shopper.
Stock `unknown` is a valid customer-facing state. Do not reinterpret it as available, hide its observation time, or let the model fill the gap. Offer refresh, product page, or human handoff according to merchant policy.
System changes
- Creates deterministic catalog and response policy code; it does not import products, call models, change inventory, or contact support.
Syntax explained
CatalogAnswer.strict()- Rejects output fields outside the reviewed reason, selected IDs, follow-up, and handoff shape.
publicProduct(...)- Builds price, currency, URL, attributes, and freshness-aware stock from merchant rows after model output validation.
src/policy.mjsimport { createHash, timingSafeEqual } from "node:crypto";
import { z } from "zod";
export const ProductImport = z.object({
id: z.string().uuid(),
sku: z.string().min(1).max(80),
title: z.string().min(2).max(240),
description: z.string().min(1).max(12000),
category: z.string().min(1).max(120),
attributes: z.record(z.string(), z.union([z.string(), z.number(), z.boolean()])),
price_minor: z.number().int().nonnegative(),
currency: z.string().regex(/^[A-Z]{3}$/),
canonical_url: z.string().url().refine((value) => new URL(value).protocol === "https:"),
active: z.boolean(),
updated_at: z.string().datetime(),
inventory: z.object({
available: z.boolean(),
quantity: z.number().int().nonnegative().nullable(),
observed_at: z.string().datetime(),
source_revision: z.string().min(1).max(120)
})
}).strict();
export const QueryPlan = z.object({
semantic_query: z.string().min(2).max(500),
category: z.string().min(1).max(120).nullable(),
currency: z.string().regex(/^[A-Z]{3}$/).nullable(),
min_price_minor: z.number().int().nonnegative().nullable(),
max_price_minor: z.number().int().nonnegative().nullable(),
must_have: z.array(z.string().min(1).max(80)).max(10),
max_results: z.number().int().min(1).max(8),
wants_human: z.boolean()
}).strict().superRefine((value, context) => {
if (
value.min_price_minor !== null &&
value.max_price_minor !== null &&
value.min_price_minor > value.max_price_minor
) {
context.addIssue({ code: "custom", message: "price_range_invalid" });
}
if (
(value.min_price_minor !== null || value.max_price_minor !== null) &&
value.currency === null
) {
context.addIssue({ code: "custom", message: "price_currency_required" });
}
});
export const CatalogAnswer = z.object({
summary: z.string().min(20).max(1200),
product_ids: z.array(z.string().uuid()).max(5),
reasons: z.array(z.object({
product_id: z.string().uuid(),
reason: z.string().min(10).max(500)
})).max(5),
follow_up_question: z.string().min(5).max(300).nullable(),
handoff: z.object({
requested: z.boolean(),
reason: z.string().max(800),
transcript_summary: z.string().max(2000)
})
}).strict();
const forbiddenClaims =
/(?:\$\s*\d|€\s*\d|£\s*\d|\b(?:in stock|out of stock|only \d+ left|guaranteed|best for everyone|will cure|will treat)\b)/i;
const sensitivePreference =
/\b(?:race|ethnicity|religion|sexual orientation|disability|medical diagnosis|credit score)\b/i;
export function contentForEmbedding(product) {
const attributes = Object.entries(product.attributes)
.sort(([left], [right]) => left.localeCompare(right))
.map(([key, value]) => key + ": " + String(value))
.join("; ");
return [
product.title,
product.category,
product.description,
attributes
].join("\n").slice(0, 16000);
}
export function contentHash(product) {
return createHash("sha256").update(contentForEmbedding(product)).digest("hex");
}
export function vectorLiteral(values) {
if (
!Array.isArray(values) ||
values.length !== 1536 ||
values.some((value) => !Number.isFinite(value))
) throw new Error("embedding_shape_invalid");
return "[" + values.join(",") + "]";
}
export function stockState(inventory, now, maximumAgeSeconds) {
if (!inventory) return { status: "unknown", reason: "missing_snapshot" };
const ageSeconds = Math.floor(
(now.valueOf() - new Date(inventory.observed_at).valueOf()) / 1000
);
if (!Number.isFinite(ageSeconds) || ageSeconds < 0 || ageSeconds > maximumAgeSeconds) {
return { status: "unknown", reason: "stale_snapshot", age_seconds: ageSeconds };
}
if (!inventory.available || inventory.quantity === 0) {
return { status: "out_of_stock", observed_at: inventory.observed_at, age_seconds: ageSeconds };
}
return {
status: "available",
quantity: inventory.quantity,
observed_at: inventory.observed_at,
age_seconds: ageSeconds
};
}
export function validateAnswer(raw, candidates, query) {
if (sensitivePreference.test(query)) throw new Error("sensitive_preference_not_supported");
const answer = CatalogAnswer.parse(raw);
const allowed = new Set(candidates.map((item) => item.id));
const referenced = [...answer.product_ids, ...answer.reasons.map((item) => item.product_id)];
if (referenced.some((id) => !allowed.has(id))) throw new Error("unknown_product_reference");
const unique = new Set(answer.product_ids);
if (unique.size !== answer.product_ids.length) throw new Error("duplicate_product_reference");
if (answer.reasons.some((item) => !unique.has(item.product_id))) {
throw new Error("reason_without_selected_product");
}
const modelText = [
answer.summary,
...answer.reasons.map((item) => item.reason),
answer.follow_up_question || ""
].join(" ");
if (forbiddenClaims.test(modelText)) throw new Error("catalog_fact_claim_from_model");
return answer;
}
export function publicProduct(product, now, maximumAgeSeconds) {
return {
id: product.id,
sku: product.sku,
title: product.title,
category: product.category,
attributes: product.attributes,
price: {
amount_minor: product.price_minor,
currency: product.currency
},
url: product.canonical_url,
stock: stockState(product, now, maximumAgeSeconds)
};
}
export function safeEqualBearer(supplied, expected) {
const left = Buffer.from(String(supplied || ""));
const right = Buffer.from("Bearer " + expected);
return left.length === right.length && timingSafeEqual(left, right);
}POLICY_OK product_schema=strict vector=1536 stock_fresh=available stock_stale=unknown unknown_product=rejected model_price_claim=rejected
Checkpoint: Policy boundary tests pass
node --test test/catalog-policy.test.mjsContinue whenStructured products and hashes validate, stale and missing stock become unknown, unknown products and model-authored price or stock claims fail, and public cards use database facts.
Stop whenStop if stale inventory remains available, a model can reference a product outside candidates, price appears in generated prose, or a sensitive preference influences ranking.
If this step fails
A legitimate reason is rejected as a stock claim.
Likely causeThe fail-closed phrase detector matches descriptive language that resembles availability.
Reproduce with a synthetic reasonIdentify the matching phraseConfirm that stock can remain entirely in deterministic card fields
ResolutionRewrite instructions and reason schema to avoid stock language, add positive and negative fixtures, and narrow the detector only after policy review.
config
Import a reviewed catalog revision and update changed embeddings
Implement the operator-only NDJSON importer. It bounds file size and product count, validates every row before opening a transaction, rejects duplicate IDs, records the source hash and actor hash, reuses embeddings when canonical descriptive content is unchanged, obtains a new vector only for changed content, upserts products and inventory, and commits the complete revision atomically. Failed validation or provider work rolls the revision back.
Why this step matters
Catalog import is a controlled merchant write, not an assistant capability. Prevalidation avoids half-valid revisions, content hashes reduce embedding spend, and one transaction prevents a mixed product revision. Inventory still carries its own source revision and observation time, so a successful catalog import cannot make an old stock snapshot fresh.
What to understand
Produce the NDJSON file from an authenticated merchant export and verify its checksum, signer, expected count, currency set, category set, timestamp range, URL host allowlist, duplicate SKU policy, and deletion semantics before running the importer.
The compact loop embeds one changed product at a time. Production can batch within provider limits, but must map every returned vector to the correct product, validate count and dimension, preserve the same transaction or staging semantics, and reconcile ambiguous requests without inventing vectors.
Do not deactivate products merely because a partial export omitted them. Require an explicit full-snapshot marker or tombstone policy, compare counts and category changes, and require human approval for abnormal mass deactivation or price movement.
System changes
- Creates one catalog revision, upserts merchant-approved products and inventory, and updates embeddings for changed descriptions after operator initiation.
Syntax explained
catalog NDJSON- Provides one strict JSON product per line so validation can identify a precise bad row without accepting arbitrary HTML or model-generated catalog fields.
content_hash- Avoids recomputing semantic vectors when only price, stock, revision, or another nonsemantic field changes.
single database transaction- Makes a revision visible completely or not at all; large catalogs may use staged tables and atomic pointer switch with the same guarantee.
src/sync-catalog.mjsimport fs from "node:fs/promises";
import { createHash } from "node:crypto";
import OpenAI from "openai";
import pg from "pg";
import {
contentForEmbedding,
contentHash,
ProductImport,
vectorLiteral
} from "./policy.mjs";
const required = (name) => {
const value = process.env[name];
if (!value) throw new Error("missing_" + name.toLowerCase());
return value;
};
const inputPath = process.argv[2];
const revision = process.argv[3];
if (!inputPath || !revision) {
throw new Error("usage: npm run sync -- <catalog.ndjson> <revision>");
}
const raw = await fs.readFile(inputPath, "utf8");
if (Buffer.byteLength(raw) > 20 * 1024 * 1024) throw new Error("catalog_file_too_large");
const products = raw.split(/\r?\n/).filter(Boolean).map((line) => ProductImport.parse(JSON.parse(line)));
if (!products.length || products.length > 20_000) throw new Error("catalog_count_out_of_bounds");
if (new Set(products.map((item) => item.id)).size !== products.length) {
throw new Error("duplicate_product_id");
}
const allowedCatalogHosts = new Set(
required("CATALOG_URL_HOSTS").split(",").map((item) => item.trim().toLowerCase())
);
if (products.some((item) => !allowedCatalogHosts.has(new URL(item.canonical_url).hostname.toLowerCase()))) {
throw new Error("catalog_url_host_not_allowed");
}
const sourceHash = createHash("sha256").update(raw).digest("hex");
const pool = new pg.Pool({ connectionString: required("DATABASE_URL"), max: 4 });
const openai = new OpenAI({ apiKey: required("OPENAI_API_KEY"), timeout: 30_000, maxRetries: 2 });
const model = required("OPENAI_EMBEDDING_MODEL");
const actorHash = createHash("sha256")
.update(required("MERCHANT_OPERATOR_TOKEN"))
.digest("hex");
const client = await pool.connect();
try {
await client.query("BEGIN");
await client.query(
"INSERT INTO catalog_revisions(revision,source_hash,imported_by) VALUES($1,$2,$3)",
[revision, sourceHash, actorHash]
);
let embedded = 0;
let unchanged = 0;
for (const product of products) {
const hash = contentHash(product);
const previous = await client.query(
"SELECT content_hash FROM product_embeddings WHERE product_id=$1",
[product.id]
);
let vector = null;
if (!previous.rowCount || previous.rows[0].content_hash !== hash) {
const response = await openai.embeddings.create({
model,
input: contentForEmbedding(product),
encoding_format: "float"
});
vector = vectorLiteral(response.data[0].embedding);
embedded += 1;
} else {
unchanged += 1;
}
await client.query(
"INSERT INTO products(id,sku,title,description,category,attributes,price_minor,currency,canonical_url,active,catalog_revision,updated_at) " +
"VALUES($1,$2,$3,$4,$5,$6,$7,$8,$9,$10,$11,$12) " +
"ON CONFLICT(id) DO UPDATE SET sku=EXCLUDED.sku,title=EXCLUDED.title,description=EXCLUDED.description," +
"category=EXCLUDED.category,attributes=EXCLUDED.attributes,price_minor=EXCLUDED.price_minor," +
"currency=EXCLUDED.currency,canonical_url=EXCLUDED.canonical_url,active=EXCLUDED.active," +
"catalog_revision=EXCLUDED.catalog_revision,updated_at=EXCLUDED.updated_at",
[
product.id, product.sku, product.title, product.description, product.category,
product.attributes, product.price_minor, product.currency, product.canonical_url,
product.active, revision, product.updated_at
]
);
await client.query(
"INSERT INTO inventory_snapshots(product_id,available,quantity,observed_at,source_revision) " +
"VALUES($1,$2,$3,$4,$5) ON CONFLICT(product_id) DO UPDATE SET " +
"available=EXCLUDED.available,quantity=EXCLUDED.quantity,observed_at=EXCLUDED.observed_at," +
"source_revision=EXCLUDED.source_revision",
[
product.id,
product.inventory.available,
product.inventory.quantity,
product.inventory.observed_at,
product.inventory.source_revision
]
);
if (vector) {
await client.query(
"INSERT INTO product_embeddings(product_id,model,content_hash,embedding) VALUES($1,$2,$3,$4::vector) " +
"ON CONFLICT(product_id) DO UPDATE SET model=EXCLUDED.model,content_hash=EXCLUDED.content_hash," +
"embedding=EXCLUDED.embedding,updated_at=now()",
[product.id, model, hash, vector]
);
}
}
await client.query("COMMIT");
console.log(JSON.stringify({
status: "CATALOG_SYNC_OK",
revision,
products: products.length,
embedded,
unchanged,
source_hash: sourceHash
}));
} catch (error) {
await client.query("ROLLBACK");
throw error;
} finally {
client.release();
await pool.end();
}{"status":"CATALOG_SYNC_OK","revision":"catalog-2026-07-29-03","products":1250,"embedded":37,"unchanged":1213,"source_hash":"29f46e8d2c9c82c532f580858fb19f6a6a48f665dfa519c0a21aa65f8ad73a12"}Checkpoint: Catalog revision accepted
npm run sync -- {{catalogPath}} {{catalogRevision}} && psql "$DATABASE_URL" -Atc "SELECT revision,source_hash FROM catalog_revisions WHERE revision='{{catalogRevision}}'"Continue whenThe source hash matches the reviewed file, product count is expected, only changed descriptive content is embedded, no partial revision is visible, and stale inventory remains stale.
Stop whenStop if the file is unsigned or partial, product counts or price distribution cross policy thresholds, a URL host is unknown, the embedding count differs, or inventory timestamps are in the future.
If this step fails
The importer rolls back after most embeddings were created.
Likely causeA late product row, database constraint, provider response, or vector dimension failed.
Read the sanitized row index and error classValidate the entire source before provider callsCompare expected and returned embedding counts
ResolutionFix the source or importer, retain no partial database revision, reuse an approved temporary embedding cache only if it is securely keyed by model and content hash, then rerun explicitly.
Security notes
- The catalog file can contain prompt injection in descriptions; validation treats it as data and the answer validator remains authoritative.
config
Implement semantic search, exact filters, database facts, and approved handoff
Add the HTTP service. An authenticated client submits a bounded query. A strict query plan extracts only explicit category and price constraints plus a semantic phrase. The service embeds that phrase, performs parameterized pgvector search with exact filters, and sends only candidate descriptive fields to a second strict response. Local code validates every product ID and reason, promotes policy-required handoff, creates a draft transactionally, then renders price, URL, attributes, and freshness-aware stock from database rows.
Why this step matters
Two structured stages keep query understanding separate from answer wording, while exact database filters and candidate membership remain authoritative. The service can explain semantic fit without giving the model control over SQL, product identity, price, stock, URL, support destination, or external actions. A stale inventory row visibly becomes unknown after the answer is generated.
What to understand
Use parameterized SQL for category and price. The model cannot supply SQL, operator names, vector expressions, table names, sort clauses, or unbounded result counts. The query schema caps results and verifies price range order.
The candidate view deliberately excludes price, currency, URL, stock, quantity, and private merchandising fields. The response model therefore cannot copy them into prose; the application joins selected IDs to the original query rows after validation.
A handoff draft contains a short policy-bounded summary and product IDs, not the full conversation, authorization header, payment information, address, or secret. Production should add explicit customer consent, retention, subject identity, and data deletion.
The dispatcher uses a fixed HTTPS endpoint, disabled redirects, separate identity, database row lock, exact approved payload, and handoff ID as idempotency key. Timeout after possible commit remains ambiguous and must not replay automatically.
System changes
- Creates authenticated search, query planning, embedding retrieval, answer validation, deterministic public product rendering, draft handoff, approval, dispatch, audit, rate limit, and health routes.
Syntax explained
embedding <=> query vector- Orders candidates by cosine distance within exact active, category, and price constraints; it does not prove suitability or popularity.
store: false- Requests no provider-hosted response state for application retrieval while application and provider data policies still require independent review.
external_writes: 0- Makes the search response explicit that a returned handoff is only a local draft and no support, order, or account action occurred.
src/server.mjsimport http from "node:http";
import { createHash, randomUUID } from "node:crypto";
import OpenAI from "openai";
import pg from "pg";
import {
CatalogAnswer,
publicProduct,
QueryPlan,
safeEqualBearer,
validateAnswer,
vectorLiteral
} from "./policy.mjs";
const required = (name) => {
const value = process.env[name];
if (!value) throw new Error("missing_" + name.toLowerCase());
return value;
};
const pool = new pg.Pool({ connectionString: required("DATABASE_URL"), max: 12 });
const openai = new OpenAI({ apiKey: required("OPENAI_API_KEY"), timeout: 30_000, maxRetries: 1 });
const model = required("OPENAI_MODEL");
const embeddingModel = required("OPENAI_EMBEDDING_MODEL");
const maxAge = Number(process.env.INVENTORY_MAX_AGE_SECONDS || 120);
const perMinute = Number(process.env.REQUESTS_PER_MINUTE || 60);
const maxQueryBytes = Number(process.env.MAX_QUERY_BYTES || 16384);
function fixedWebhook() {
const value = new URL(required("SUPPORT_WEBHOOK_URL"));
if (
value.protocol !== "https:" ||
value.username ||
value.password ||
value.hostname !== required("SUPPORT_WEBHOOK_HOST")
) throw new Error("support_webhook_configuration_invalid");
return value.href;
}
function json(response, status, body) {
const output = JSON.stringify(body);
response.writeHead(status, {
"content-type": "application/json",
"content-length": Buffer.byteLength(output),
"cache-control": "no-store"
});
response.end(output);
}
async function readJson(request) {
const chunks = [];
let size = 0;
for await (const chunk of request) {
size += chunk.length;
if (size > maxQueryBytes) throw Object.assign(new Error("payload_too_large"), { status: 413 });
chunks.push(chunk);
}
try {
return JSON.parse(Buffer.concat(chunks).toString("utf8"));
} catch {
throw Object.assign(new Error("invalid_json"), { status: 400 });
}
}
function authenticate(request, secretName, role) {
if (!safeEqualBearer(request.headers.authorization, required(secretName))) {
throw Object.assign(new Error(role + "_authentication_failed"), { status: 401 });
}
return createHash("sha256")
.update(role + ":" + request.headers.authorization)
.digest("hex");
}
async function rateLimit(key) {
const result = await pool.query(
"INSERT INTO commerce_rate_buckets(key,bucket,count) VALUES($1,date_trunc('minute',now()),1) " +
"ON CONFLICT(key,bucket) DO UPDATE SET count=commerce_rate_buckets.count+1 " +
"WHERE commerce_rate_buckets.count < $2 RETURNING count",
[key, perMinute]
);
if (!result.rowCount) throw Object.assign(new Error("rate_limit_exceeded"), { status: 429 });
}
async function structured(input, schema, name, instructions, maxOutputTokens) {
const result = await openai.responses.create({
model,
store: false,
instructions,
input,
max_output_tokens: maxOutputTokens,
text: {
format: {
type: "json_schema",
name,
strict: true,
schema: schema.toJSONSchema()
}
}
});
if (result.status !== "completed" || !result.output_text) {
throw Object.assign(new Error("model_response_incomplete"), { status: 502 });
}
return { id: result.id, value: JSON.parse(result.output_text), usage: result.usage };
}
async function searchCatalog(query, plan) {
const embedded = await openai.embeddings.create({
model: embeddingModel,
input: plan.semantic_query,
encoding_format: "float"
});
const vector = vectorLiteral(embedded.data[0].embedding);
const result = await pool.query(
"SELECT p.*, i.available, i.quantity, i.observed_at, i.source_revision, " +
"1 - (e.embedding <=> $1::vector) AS semantic_score " +
"FROM product_embeddings e JOIN products p ON p.id=e.product_id " +
"LEFT JOIN inventory_snapshots i ON i.product_id=p.id " +
"WHERE p.active=true AND ($2::text IS NULL OR p.category=$2) " +
"AND ($3::text IS NULL OR p.currency=$3) " +
"AND ($4::integer IS NULL OR p.price_minor >= $4) " +
"AND ($5::integer IS NULL OR p.price_minor <= $5) " +
"AND (cardinality($6::text[]) = 0 OR p.attributes ?& $6::text[]) " +
"ORDER BY e.embedding <=> $1::vector, p.id LIMIT $7",
[
vector,
plan.category,
plan.currency,
plan.min_price_minor,
plan.max_price_minor,
plan.must_have,
plan.max_results
]
);
return result.rows;
}
async function createHandoff(client, sessionId, actorHash, answer) {
if (!answer.handoff.requested) return null;
const id = randomUUID();
await client.query(
"INSERT INTO handoff_drafts(id,session_id,reason,transcript_summary,product_ids,status) " +
"VALUES($1,$2,$3,$4,$5,'draft')",
[
id,
sessionId,
answer.handoff.reason,
answer.handoff.transcript_summary,
answer.product_ids
]
);
await client.query(
"INSERT INTO commerce_audit(session_id,actor_hash,event_type,object_id) VALUES($1,$2,'handoff_drafted',$3)",
[sessionId, actorHash, id]
);
return id;
}
async function answerSearch(request, response) {
const actorHash = authenticate(request, "CLIENT_BEARER_TOKEN", "client");
await rateLimit("search:" + actorHash);
const input = await readJson(request);
if (typeof input.query !== "string" || input.query.length < 2 || input.query.length > 2000) {
throw Object.assign(new Error("query_invalid"), { status: 400 });
}
const sessionId = input.session_id ? String(input.session_id) : randomUUID();
const planned = await structured(
input.query,
QueryPlan,
"catalog_query_plan",
"Extract only explicit catalog constraints. Use ISO currency when a price is present. Put required structured attribute keys in must_have. Do not infer sensitive traits. Set wants_human for order, account, complaint, regulated, or uncertain requests.",
500
);
const plan = QueryPlan.parse(planned.value);
const candidates = await searchCatalog(input.query, plan);
const candidateView = candidates.map((item) => ({
id: item.id,
sku: item.sku,
title: item.title,
description: item.description,
category: item.category,
attributes: item.attributes,
semantic_score: Number(item.semantic_score)
}));
const generated = await structured(
JSON.stringify({ query: input.query, plan, candidates: candidateView }),
CatalogAnswer,
"catalog_answer",
"Recommend only supplied product IDs. Explain attribute fit only. Never state price, stock, scarcity, guarantee, universal best, medical benefit, or order status. Request handoff for account, order, complaint, regulated, or unresolved needs.",
1200
);
const answer = validateAnswer(generated.value, candidates, input.query);
if (plan.wants_human && !answer.handoff.requested) {
answer.handoff = {
requested: true,
reason: "The query requires a human-owned account, order, complaint, regulated, or unresolved workflow.",
transcript_summary: "Customer requested human assistance after a bounded catalog search."
};
}
const client = await pool.connect();
let handoffId;
try {
await client.query("BEGIN");
handoffId = await createHandoff(client, sessionId, actorHash, answer);
await client.query(
"INSERT INTO commerce_audit(session_id,actor_hash,event_type,object_id,detail) " +
"VALUES($1,$2,'catalog_answered',$3,$4)",
[
sessionId,
actorHash,
generated.id,
{
plan_response_id: planned.id,
candidate_ids: candidates.map((item) => item.id),
selected_ids: answer.product_ids,
handoff_id: handoffId,
model
}
]
);
await client.query("COMMIT");
} catch (error) {
await client.query("ROLLBACK");
throw error;
} finally {
client.release();
}
const byId = new Map(candidates.map((item) => [item.id, item]));
json(response, 200, {
session_id: sessionId,
summary: answer.summary,
recommendations: answer.product_ids.map((id) => ({
product: publicProduct(byId.get(id), new Date(), maxAge),
reason: answer.reasons.find((item) => item.product_id === id)?.reason || null
})),
follow_up_question: answer.follow_up_question,
handoff: handoffId ? { id: handoffId, status: "draft", dispatched: false } : null,
catalog_facts_source: "database",
external_writes: 0
});
}
async function approveHandoff(request, response, id) {
const actorHash = authenticate(request, "MERCHANT_OPERATOR_TOKEN", "merchant_operator");
const result = await pool.query(
"UPDATE handoff_drafts SET status='approved',approved_by=$1,approved_at=now() " +
"WHERE id=$2 AND status='draft' RETURNING session_id,id",
[actorHash, id]
);
if (!result.rowCount) throw Object.assign(new Error("handoff_not_approvable"), { status: 409 });
await pool.query(
"INSERT INTO commerce_audit(session_id,actor_hash,event_type,object_id) " +
"VALUES($1,$2,'handoff_approved',$3)",
[result.rows[0].session_id, actorHash, id]
);
json(response, 200, { handoff_id: id, status: "approved", dispatched: false });
}
async function dispatchHandoff(request, response, id) {
const actorHash = authenticate(request, "HANDOFF_DISPATCH_TOKEN", "handoff_dispatcher");
const client = await pool.connect();
try {
await client.query("BEGIN");
const selected = await client.query(
"SELECT * FROM handoff_drafts WHERE id=$1 AND status='approved' FOR UPDATE",
[id]
);
if (!selected.rowCount) throw Object.assign(new Error("handoff_not_dispatchable"), { status: 409 });
const draft = selected.rows[0];
const sent = await fetch(fixedWebhook(), {
method: "POST",
redirect: "error",
headers: {
"authorization": "Bearer " + required("SUPPORT_WEBHOOK_TOKEN"),
"content-type": "application/json",
"idempotency-key": draft.id
},
body: JSON.stringify({
handoff_id: draft.id,
session_id: draft.session_id,
reason: draft.reason,
transcript_summary: draft.transcript_summary,
product_ids: draft.product_ids
}),
signal: AbortSignal.timeout(10_000)
});
if (!sent.ok) throw Object.assign(new Error("support_dispatch_failed"), { status: 502 });
const dispatchId = sent.headers.get("x-request-id") || draft.id;
await client.query(
"UPDATE handoff_drafts SET status='dispatched',dispatch_id=$1 WHERE id=$2",
[dispatchId, draft.id]
);
await client.query(
"INSERT INTO commerce_audit(session_id,actor_hash,event_type,object_id,detail) " +
"VALUES($1,$2,'handoff_dispatched',$3,$4)",
[draft.session_id, actorHash, draft.id, { dispatch_id: dispatchId }]
);
await client.query("COMMIT");
json(response, 200, { handoff_id: draft.id, status: "dispatched", dispatch_id: dispatchId });
} catch (error) {
await client.query("ROLLBACK");
throw error;
} finally {
client.release();
}
}
const server = http.createServer(async (request, response) => {
const requestId = randomUUID();
const started = performance.now();
try {
const url = new URL(request.url, "http://localhost");
const parts = url.pathname.split("/").filter(Boolean);
if (request.method === "GET" && url.pathname === "/healthz") {
const database = await pool.query(
"SELECT max(imported_at) AS catalog_at, " +
"(SELECT max(observed_at) FROM inventory_snapshots) AS inventory_at FROM catalog_revisions"
);
return json(response, 200, { status: "ok", ...database.rows[0], request_id: requestId });
}
if (request.method === "POST" && url.pathname === "/v1/search") {
return await answerSearch(request, response);
}
if (request.method === "POST" && parts[0] === "v1" && parts[1] === "handoffs" && parts[3] === "approve") {
return await approveHandoff(request, response, parts[2]);
}
if (request.method === "POST" && parts[0] === "v1" && parts[1] === "handoffs" && parts[3] === "dispatch") {
return await dispatchHandoff(request, response, parts[2]);
}
json(response, 404, { error: "not_found", request_id: requestId });
} catch (error) {
const status = Number(error.status) || 500;
console.error(JSON.stringify({
level: "error",
request_id: requestId,
code: status >= 500 ? "internal_error" : error.message,
duration_ms: Math.round(performance.now() - started)
}));
json(response, status, {
error: status >= 500 ? "internal_error" : error.message,
request_id: requestId
});
}
});
server.listen(Number(process.env.PORT || 8090), "0.0.0.0", () => {
console.log(JSON.stringify({ level: "info", event: "catalog_assistant_started", port: Number(process.env.PORT || 8090) }));
});{"session_id":"2e076597-5826-45a4-adf7-49cf4f6287b1","summary":"These candidates match the requested compact size and laptop sleeve.","recommendations":[{"product":{"id":"1d334b9d-cad5-4385-a3d5-430cacb238ee","sku":"PACK-18-BLUE","title":"18 litre blue daypack","category":"bags","attributes":{"color":"blue","capacity_litres":18,"laptop_sleeve":true},"price":{"amount_minor":7900,"currency":"USD"},"url":"https://shop.example.invalid/products/pack-18-blue","stock":{"status":"unknown","reason":"stale_snapshot","age_seconds":315}},"reason":"Its structured attributes include an 18 litre capacity and laptop sleeve."}],"follow_up_question":"Would another color be useful?","handoff":null,"catalog_facts_source":"database","external_writes":0}Checkpoint: Bounded catalog query succeeds
Continue whenSelected IDs are a subset of retrieved candidates, reasons mention only structured attributes, displayed price and stock equal database rows, stale stock is unknown, handoff remains draft, and support receives zero requests before approval and dispatch.
Stop whenStop if shopper text becomes SQL, model output supplies catalog facts, unknown products render, stale stock appears available, sensitive preference ranks products, or a handoff is sent without exact-content approval.
If this step fails
Search returns semantically close products outside an explicit category.
Likely causeThe query plan omitted the category, catalog taxonomy differs from user language, or exact filtering is applied after retrieval.
Inspect the schema-validated plan and category vocabularyRun the parameterized SQL with a synthetic vectorCompare category aliases maintained by the application
ResolutionMap reviewed aliases before SQL, apply exact constraints inside retrieval, ask a clarifying question when category is ambiguous, and do not let the model relax explicit filters.
Support receives a duplicate handoff after timeout.
Likely causeThe destination ignored or expired the idempotency key, or the worker replayed an ambiguous request.
Query support by handoff IDCompare destination and audit request IDsCheck dispatcher retry policy
ResolutionDisable dispatch, reconcile the authoritative support state, record the dispatch ID through an audited operator transition, and require a destination idempotency contract before re-enabling.
Security notes
- The model adapter has no order, account, payment, inventory, discount, support, or database write tools.
- A public client credential in this compact sample must be replaced by real subject and tenant authorization for multi-user production.
Alternatives
- Use deterministic full-text and exact filters only when embeddings are unavailable or semantic recall does not meet the labeled evaluation threshold.
decision
Make stale, missing, and contradictory inventory visible
Exercise available, zero, unavailable, missing, future, and stale inventory fixtures. The product can remain a valid semantic recommendation while its stock state is unknown. Show observation time or freshness status in the UI, link to the canonical product page, and provide refresh or human assistance according to policy. Never hide the candidate solely to make the assistant appear confident, and never let model prose override the deterministic state.
Why this step matters
Inventory is often more volatile than product content or embeddings. A separate, fail-closed state lets the assistant remain useful without making a sales promise. Testing contradictory values also forces the merchant to define whether `available`, quantity, region, reservation, preorder, and backorder have product-specific semantics.
What to understand
Use the source observation timestamp, source revision, region, and channel where required. Import time is not evidence that inventory was observed recently. Reject unreasonable future timestamps rather than creating negative age.
A quantity can be hidden by business policy, but the freshness calculation still applies. If quantity is null and available is true, display available without numeric scarcity only when the source contract permits it.
Checkout remains authoritative. Catalog availability is informational and must not reserve stock, promise fulfillment, or imply that an item will remain available. The assistant should not generate scarcity pressure.
System changes
- Creates test and UI behavior for inventory states; it does not change source inventory, reserve stock, or update an order.
fresh_available=available fresh_zero=out_of_stock explicit_unavailable=out_of_stock stale=unknown missing=unknown future=unknown model_override=rejected
Checkpoint: Freshness matrix accepted
Continue whenAll six fixtures produce documented deterministic states, timestamps are visible or explainable, no free-form response contradicts them, and checkout authority is clearly stated.
Stop whenStop if stale or future data appears available, quantity generates scarcity language, region is ignored, or the model can convert unknown to available.
If this step fails
Every product suddenly shows unknown stock.
Likely causeThe feed stopped, source timestamps use a wrong timezone or unit, import transaction failed, or the threshold is lower than delivery latency.
Compare source and database observation timesInspect feed revision and import auditMeasure ordinary feed delay without changing thresholds
ResolutionKeep stock unknown, restore the authoritative feed or parsing, backfill only source-observed timestamps, and raise thresholds only through reviewed merchant policy.
config
Test schema, vector, freshness, candidate, and fact boundaries
Add deterministic tests that require no model, database, or external network. They prove product validation and hash stability, exact embedding dimension, stale-stock failure, unknown-product rejection, free-form price and stock rejection, and database rendering of public facts. Extend the release suite with ephemeral PostgreSQL and fake provider and support adapters for transaction, rate, refusal, timeout, and approval behavior.
Why this step matters
Deterministic contract tests protect the facts and authorization boundaries even when model wording changes. Live evaluations can measure relevance and helpfulness, but only local tests can reliably prove that an unknown product, stale stock, malformed currency, unsupported vector, sensitive request, or unapproved handoff never crosses the API boundary.
What to understand
Build labeled search fixtures with explicit expected and forbidden products, filters, categories, typos, synonyms, multilingual queries, no-result cases, stale stock, sensitive traits, and regulated suitability. Score candidate recall separately from generated reason quality.
Fake provider outputs must include refusal, incomplete status, extra fields, unknown product IDs, duplicate IDs, reason for an unselected product, price claims, stock claims, prohibited guarantee, oversized handoff, and prompt-injection echo.
Fake support must simulate success, deterministic rejection, timeout before commit, timeout after commit, repeated idempotency key, redirect, wrong host, and malformed response. Assert that ambiguous state blocks replay.
System changes
- Creates deterministic and integration test fixtures; no production catalog, shopper data, provider spend, order, or support ticket is changed.
Syntax explained
node --test- Runs isolated Node.js assertions and exits nonzero when a deterministic catalog boundary changes.
test/catalog-policy.test.mjsimport assert from "node:assert/strict";
import test from "node:test";
import {
contentHash,
ProductImport,
publicProduct,
stockState,
validateAnswer,
vectorLiteral
} from "../src/policy.mjs";
const product = ProductImport.parse({
id: "1d334b9d-cad5-4385-a3d5-430cacb238ee",
sku: "PACK-18-BLUE",
title: "18 litre blue daypack",
description: "A compact recycled-nylon daypack with padded laptop sleeve.",
category: "bags",
attributes: { color: "blue", capacity_litres: 18, laptop_sleeve: true },
price_minor: 7900,
currency: "USD",
canonical_url: "https://shop.example.invalid/products/pack-18-blue",
active: true,
updated_at: "2026-07-29T12:00:00.000Z",
inventory: {
available: true,
quantity: 4,
observed_at: "2026-07-29T12:00:00.000Z",
source_revision: "inventory-884"
}
});
test("structured product import and content hash are deterministic", () => {
assert.equal(product.sku, "PACK-18-BLUE");
assert.equal(contentHash(product).length, 64);
assert.equal(contentHash(product), contentHash({
...product,
attributes: { laptop_sleeve: true, capacity_litres: 18, color: "blue" }
}));
});
test("inventory freshness is fail-closed", () => {
assert.equal(
stockState(product.inventory, new Date("2026-07-29T12:01:00.000Z"), 120).status,
"available"
);
assert.equal(
stockState(product.inventory, new Date("2026-07-29T12:05:00.000Z"), 120).status,
"unknown"
);
assert.equal(stockState(null, new Date(), 120).status, "unknown");
});
test("model can reference candidates but cannot author catalog facts", () => {
const answer = {
summary: "This daypack matches the requested compact size and laptop sleeve.",
product_ids: [product.id],
reasons: [{
product_id: product.id,
reason: "Its structured attributes include an 18 litre capacity and laptop sleeve."
}],
follow_up_question: "Would another color be useful?",
handoff: { requested: false, reason: "", transcript_summary: "" }
};
assert.equal(validateAnswer(answer, [product], "small laptop bag").product_ids[0], product.id);
assert.throws(
() => validateAnswer({ ...answer, summary: "It is in stock for $79." }, [product], "small laptop bag"),
/catalog_fact_claim/
);
assert.throws(
() => validateAnswer({
...answer,
product_ids: ["1d334b9d-cad5-4385-a3d5-430cacb23999"],
reasons: []
}, [product], "small laptop bag"),
/unknown_product/
);
});
test("public product renders price and stock from database state", () => {
const output = publicProduct({
...product,
...product.inventory
}, new Date("2026-07-29T12:01:00.000Z"), 120);
assert.deepEqual(output.price, { amount_minor: 7900, currency: "USD" });
assert.equal(output.stock.status, "available");
assert.equal(output.stock.quantity, 4);
});
test("embedding vector must match reviewed model dimensions", () => {
assert.match(vectorLiteral(Array(1536).fill(0)), /^\[0,0,0/);
assert.throws(() => vectorLiteral([0, 1]), /embedding_shape_invalid/);
});TAP version 13 ok 1 - structured product import and content hash are deterministic ok 2 - inventory freshness is fail-closed ok 3 - model can reference candidates but cannot author catalog facts ok 4 - public product renders price and stock from database state ok 5 - embedding vector must match reviewed model dimensions # pass 5 # fail 0
Checkpoint: Catalog safety suite passes
npm testContinue whenAll policy tests pass without network, fixtures contain no real customer or secret data, and invalid model, inventory, product, and handoff states fail closed.
Stop whenStop if expected ranking depends on nondeterministic prose, real customer data appears in fixtures, or a fake ambiguous support result triggers retry.
If this step fails
Semantic evaluation improves while exact-filter tests regress.
Likely causeRanking changes are being optimized without enforcing category, price, active status, or candidate membership first.
Separate retrieval and filter metricsInspect schema plan fixturesCompare SQL candidates before generated answer
ResolutionBlock release, restore the last accepted retrieval revision, make exact constraints a hard precondition, and retune semantic ranking only within valid candidates.
config
Instrument retrieval and handoff without collecting shopper content
Start OpenTelemetry before the server and export bounded traces and metrics to a private collector. Measure route latency and status, rate rejection, plan validation, embedding latency, candidate count, no-result rate, inventory freshness classes, answer validation, handoff lifecycle, dispatch ambiguity, database pool saturation, and provider usage. Do not export query text, product description, attributes, model output, transcript summary, bearer token, support body, or stable customer identity.
Why this step matters
Observability is needed to detect stale feeds, weak retrieval, rate pressure, provider failures, and stuck approvals, but shopper queries and handoff summaries may contain personal or sensitive content. Low-cardinality operational attributes provide sufficient service evidence without copying the conversation into a monitoring system with broader access and retention.
What to understand
Record catalog and policy revision as controlled attributes, but avoid product ID, session ID, query, category from free text, or support destination as metric labels. Restricted audit state can hold hashed subject and selected product IDs when required.
Alert on inventory freshness degradation, catalog import age, embedding mismatch, validation rejection increase, zero-result regression, handoff backlog, ambiguous dispatch, backup age, restore failure, and telemetry exporter errors.
Use canary strings in query, product description, and handoff summary. Search logs, traces, metrics, collector queues, dashboards, and alerts to prove the strings never leave the application data boundary.
System changes
- Creates telemetry startup and operations signals; it does not change retrieval, ranking, recommendation, approval, or support state.
Syntax explained
OTLP traces and metrics- Exports service timing and bounded counters to the configured collector; deployment must add transport authentication and attribute filtering.
src/telemetry.mjsimport { NodeSDK } from "@opentelemetry/sdk-node";
import { getNodeAutoInstrumentations } from "@opentelemetry/auto-instrumentations-node";
import { OTLPTraceExporter } from "@opentelemetry/exporter-trace-otlp-http";
import { OTLPMetricExporter } from "@opentelemetry/exporter-metrics-otlp-http";
import { PeriodicExportingMetricReader } from "@opentelemetry/sdk-metrics";
const endpoint = process.env.OTEL_EXPORTER_OTLP_ENDPOINT;
if (endpoint) {
const sdk = new NodeSDK({
serviceName: "catalog-assistant",
traceExporter: new OTLPTraceExporter({ url: endpoint + "/v1/traces" }),
metricReader: new PeriodicExportingMetricReader({
exporter: new OTLPMetricExporter({ url: endpoint + "/v1/metrics" }),
exportIntervalMillis: 30_000
}),
instrumentations: [getNodeAutoInstrumentations()]
});
await sdk.start();
process.on("SIGTERM", () => sdk.shutdown().finally(() => process.exit(0)));
}OTEL_OK service=catalog-assistant search_spans=42 vector_spans=42 handoff_spans=3 query_attributes=0 transcript_attributes=0 secret_attributes=0
Checkpoint: Telemetry privacy test passes
Continue whenSynthetic search produces traces and aggregate metrics, freshness and handoff counters update, and canary query, product, transcript, and credential strings appear nowhere in telemetry.
Stop whenStop if auto-instrumentation captures HTTP bodies, SQL parameters, model payloads, support payloads, or high-cardinality customer and product identifiers.
If this step fails
Metrics show a large no-result increase after a catalog revision.
Likely causeProducts were deactivated, categories changed, embeddings failed, or exact filters no longer map user language.
Compare revision counts and category distributionCheck embedding completion and model labelRun the labeled query cohort against old and new revisions
ResolutionFreeze or roll back the catalog pointer, keep inventory independent, correct the import or alias policy, and re-evaluate before promoting the revision.
config
Deploy an immutable canary with constrained network and storage
Replace the placeholder pgvector image digest with the reviewed current digest, build a minimal application image from the lockfile, and deploy to a synthetic storefront. The service binds only to loopback behind an authenticated gateway, uses a read-only root filesystem and bounded no-exec temporary storage, waits for database health, and receives secrets at runtime. Restrict egress to OpenAI, PostgreSQL, telemetry, and the fixed support endpoint.
Why this step matters
A canary validates the complete packaging and data path before real shoppers rely on it. Immutable revisions make rollback possible, while a synthetic support endpoint demonstrates that search and handoff drafting do not create external writes. Resource and network limits reduce abuse and isolate failures from order or payment systems that this service should never reach.
What to understand
Use the Compose file as executable reference, not a complete public perimeter. Production needs TLS, real identity, web application firewall and abuse controls, secret mounts, network policy, resource limits, autoscaling ceilings, database TLS, backup policy, and two revisions.
Seed synthetic products with near-duplicates, conflicting attributes, stale stock, invalid description instructions, price boundaries, unavailable items, and no-result categories. Keep the canary support destination incapable of customer communication.
Exercise kill switches for generated reasons, semantic search, catalog revision, and support dispatch independently. Exact search or static catalog browsing should remain available when the provider is unavailable.
System changes
- Creates container, volume, secret, network, and canary workload state; no production shopper traffic or real support destination should be connected.
Syntax explained
pgvector image digest- Pins PostgreSQL plus vector extension to the reviewed artifact; replace the deliberate placeholder before any deployment.
read_only and tmpfs- Prevents ordinary root filesystem writes while providing a small transient area; durable state remains in PostgreSQL.
compose.yamlservices:
postgres:
image: pgvector/pgvector:pg17@sha256:replace-with-reviewed-digest
environment:
POSTGRES_DB: catalog
POSTGRES_USER: catalog_app
POSTGRES_PASSWORD_FILE: /run/secrets/postgres_password
secrets: [postgres_password]
volumes: [catalog-db:/var/lib/postgresql/data]
healthcheck:
test: ["CMD-SHELL", "pg_isready -U catalog_app -d catalog"]
interval: 10s
timeout: 3s
retries: 10
app:
build: .
read_only: true
tmpfs: ["/tmp:rw,noexec,nosuid,size=32m"]
env_file: .env
depends_on:
postgres:
condition: service_healthy
ports: ["127.0.0.1:8090:8090"]
restart: unless-stopped
volumes:
catalog-db:
secrets:
postgres_password:
file: ./secrets/postgres_passwordNAME STATUS PORTS catalog-postgres-1 Up (healthy) 5432/tcp catalog-app-1 Up 127.0.0.1:8090->8090/tcp CANARY_OK catalog=catalog-2026-07-29-03 stock_fresh=98.7% support_writes=0
Checkpoint: Canary gate passes
docker compose config --quiet && docker compose up -d --build && curl --fail --silent http://127.0.0.1:8090/healthzContinue whenContainers are healthy, the endpoint reports catalog and inventory timestamps, synthetic retrieval meets thresholds, stale stock is unknown, and support receives zero unapproved requests.
Stop whenStop if an image digest is unresolved, service binds publicly, secrets appear in image or inspect output, catalog age is unknown, or the support canary receives a request before approval and dispatch.
Alternatives
- Use managed PostgreSQL with supported vector search and a managed container platform while preserving version, privilege, egress, backup, and rollback controls.
decision
Exercise exact-content human approval and dispatch reconciliation
Submit synthetic account and complaint queries. Confirm search can draft a handoff but returns `external_writes: 0`. The operator view must show consent, session, reason, summary, selected products, policy revision, exact destination, retention, and warning that nothing has been sent. Approve with the merchant identity, dispatch with the worker identity, then simulate timeout after support accepts the handoff. The ambiguous draft must remain blocked from automatic resend.
Why this step matters
A handoff can contain personal context and create customer expectations, so it deserves the same deliberate authorization as any other external write. Exact-content approval prevents the model or UI from changing the payload after review, and separate dispatch credentials prevent an approver session from becoming a general support integration secret.
What to understand
Require a current authenticated operator role and show a content hash. Reject approval when session consent, reason, summary, product list, destination, policy, or retention changed after the review screen loaded.
Editing creates a new draft and rejects or supersedes the old one. Approval cannot be copied. Reject, approve, dispatch, and reconcile are separate durable audit events.
Support lookup by handoff ID is required for ambiguity. If the destination lacks it, keep sending manual and present a copyable reviewed summary after approval instead of attempting automated dispatch.
System changes
- Creates synthetic draft, approval, dispatch, and reconciliation records against a sandbox destination only.
handoff=h-200 status=draft support_requests=0 approver=sha256:3fe1 status=approved support_requests=0 dispatch=support-991 status=dispatched support_requests=1 ambiguous=h-201 auto_retry=0 reconciliation=required
Checkpoint: Handoff separation demonstrated
Continue whenClient cannot approve, merchant cannot dispatch, dispatcher cannot approve, model cannot call any action route, one approved draft sends once, and ambiguous outcome blocks until authoritative reconciliation.
Stop whenStop if query submission implies consent, one identity has all roles, content changes after approval, redirects are followed, or timeout causes automatic cross-destination or same-destination replay.
If this step fails
Operator sees an empty or misleading handoff summary.
Likely causeThe model omitted essential customer intent, redaction removed too much, or UI rendered a different revision.
Compare stored draft with exact approval viewReview policy-required summary fieldsCheck redaction and revision metadata
ResolutionReject the draft, edit into a new operator-authored draft, require fresh approval, and improve bounded summary fixtures without storing unnecessary conversation.
Security notes
- Do not include payment data, authentication secrets, full addresses, or complete conversation by default in support handoff.
command
Back up and restore catalog, vector, inventory, handoff, and audit state
Create an encrypted PostgreSQL custom-format backup with a restricted backup identity. Restore it to an isolated database using the matching PostgreSQL and pgvector revisions. Verify catalog source hash, product and embedding counts, vector dimensions and model labels, inventory timestamps, handoff lifecycle, dispatch IDs, and audit sequence. Export the current catalog revision as structured NDJSON for independent merchant comparison without including credentials or shopper queries.
Why this step matters
Catalog recovery includes more than product rows. A restore with missing vectors silently harms search; missing inventory timestamps misstates freshness; missing approval audit makes dispatch unaccountable. Logical merchant export and database backup serve different purposes and should be verified together against the source revision.
What to understand
Encrypt and version backups with checksum, catalog revision, database version, extension version, model labels, application image, retention, and recovery objective. Backup credentials must not modify products or handoffs.
Restore into an isolated network with no support destination. Run exact and semantic synthetic queries, inventory freshness checks using a fixed clock, and handoff audit integrity. Do not call real OpenAI or support during restore verification.
For large vector indexes, document whether the backup contains the index or rebuilds it. A rebuild must use the exact embedding model and content hash contract, and retrieval must meet the labeled acceptance set before promotion.
System changes
- Creates a restrictive backup and isolated restore during the exercise; production catalog and support state remain unchanged.
Syntax explained
pg_dump --format=custom --no-owner- Produces an inspectable PostgreSQL archive without forcing production role ownership into the isolated restore.
merchant NDJSON export- Provides a structured comparison with the authoritative catalog source; it is not a replacement for handoff and audit backup.
Values stay on this page and are never sent or saved.
umask 077 && pg_dump --format=custom --no-owner --file={{backupPath}} "$DATABASE_URL" && psql "$DATABASE_URL" -Atc "SELECT jsonb_build_object('id',p.id,'sku',p.sku,'title',p.title,'description',p.description,'category',p.category,'attributes',p.attributes,'price_minor',p.price_minor,'currency',p.currency,'canonical_url',p.canonical_url,'active',p.active,'updated_at',p.updated_at,'inventory',jsonb_build_object('available',i.available,'quantity',i.quantity,'observed_at',i.observed_at,'source_revision',i.source_revision)) FROM products p LEFT JOIN inventory_snapshots i ON i.product_id=p.id ORDER BY p.id" > {{catalogExportPath}} && pg_restore --list {{backupPath}} | head -n 14; ; Archive created at 2026-07-29 16:08:12 UTC ; dbname: catalog ; TOC Entries: 51 TABLE DATA public products TABLE DATA public product_embeddings TABLE DATA public inventory_snapshots TABLE DATA public handoff_drafts
Checkpoint: Commerce recovery verified
pg_restore --clean --if-exists --no-owner --dbname "$RESTORE_DATABASE_URL" {{backupPath}} && psql "$RESTORE_DATABASE_URL" -Atc "SELECT count(*),count(e.product_id) FROM products p LEFT JOIN product_embeddings e ON e.product_id=p.id"Continue whenRestore completes within objective, active product and embedding counts match policy, catalog source hash matches export, stock remains freshness-aware, and every dispatched handoff has approval plus audit evidence.
Stop whenStop if backup is plaintext in shared storage, extension or vector dimensions differ, products lack embeddings unexpectedly, support egress exists in recovery, or dispatch history cannot be reconstructed.
If this step fails
Products restore but vector queries fail.
Likely causepgvector extension, index, model dimension, or embedding rows are missing or incompatible.
Check extension and database versionsCount vectors and inspect typmod dimensionRun exact sequential vector comparison on synthetic rows
ResolutionKeep recovery isolated, install the matching reviewed extension or rebuild vectors from approved catalog content under the exact model contract, then rerun relevance acceptance.
verification
Release by cohort and review relevance, freshness, safety, and cost
Release first to internal synthetic users, then a small catalog and traffic cohort. Measure top-k recall on labeled queries, exact-filter compliance, unknown-product rejection, stock freshness classes, no-result behavior, reason support, sensitive and regulated routing, handoff draft rate, operator rejection, unapproved writes, ambiguous dispatch, latency, token use, embedding cost, database saturation, backup age, and restore result. Schedule review in ninety days or after any model, schema, catalog, inventory, support, privacy, or policy change.
Why this step matters
A production catalog assistant should be accepted on factual and operational evidence, not polished demo prose. Labeled retrieval and exact-filter metrics show whether products are discoverable; freshness and policy fixtures show whether the service fails safely; approval and support metrics show whether external actions remain controlled.
What to understand
Set release thresholds before running the cohort: zero unknown products rendered, zero model-authored price or stock, zero stale-as-available, zero sensitive ranking, zero unapproved support writes, and zero automatic ambiguous retry.
Review relevance across category, price range, attribute density, typo, synonym, language, no-result, near-duplicate, inactive product, stale inventory, and adversarial description. Measure retrieval separately from generated reason.
Keep kill switches for generated reasons, embeddings, catalog revision, and support dispatch. Static or exact catalog search and canonical product links should remain available during provider or vector failure.
System changes
- Changes traffic allocation and feature flags under release control; product, inventory, order, account, and support data are not modified by the verification query.
Syntax explained
jq deterministic projection- Displays IDs, database-rendered price, deterministic stock state, and external-write count without treating generated prose as release evidence.
curl --fail --silent -H "Authorization: Bearer $CLIENT_BEARER_TOKEN" -H 'Content-Type: application/json' --data '{"query":"compact blue laptop daypack under 100 dollars"}' http://127.0.0.1:8090/v1/search | jq '{session_id,recommendations:[.recommendations[]|{id:.product.id,price:.product.price,stock:.product.stock.status}],external_writes}'{"session_id":"2e076597-5826-45a4-adf7-49cf4f6287b1","recommendations":[{"id":"1d334b9d-cad5-4385-a3d5-430cacb238ee","price":{"amount_minor":7900,"currency":"USD"},"stock":"unknown"}],"external_writes":0}Checkpoint: Commerce release gate accepted
Continue whenThe signed report meets factual, freshness, relevance, safety, approval, recovery, latency, and cost thresholds, names current revisions, and sets the next review for 2026-10-27.
Stop whenStop or roll back on any invented product, model-authored catalog fact, stale availability, sensitive ranking, unapproved write, ambiguous replay, secret leak, missing backup, or failed restore.
If this step fails
Click-through improves while labeled recommendation quality declines.
Likely causeEngagement optimization favors popular, promotional, or persuasive results over explicit constraints and supported attribute fit.
Compare exact-filter compliance and labeled recallInspect reason support and stock freshnessSegment by catalog coverage without sensitive traits
ResolutionRestore the last accepted ranking revision, remove engagement-only features, and require relevance and safety thresholds before another cohort.
warning
Teardown without losing catalog provenance or handoff accountability
Disable new search and handoff dispatch, drain requests, reconcile approved and ambiguous handoffs, freeze the final catalog and inventory revisions, create and restore-test the final encrypted backup, export the merchant catalog comparison and required handoff audit, then revoke provider, client, operator, dispatcher, database, support, telemetry, backup, and migration identities. Remove workloads and egress before separately approved database and backup deletion. Preserve retention holds and a signed deletion report.
Why this step matters
Immediate volume deletion can erase the only proof of which product facts, stock snapshot, recommendation policy, approval, and support request were active. A staged teardown keeps external writes stopped while operators reconcile state and transfer records. Independent credential revocation closes forgotten access paths after the service stops.
What to understand
Inventory all database backups, catalog files, vector caches, handoff exports, support tickets, telemetry, provider records, images, volumes, secrets, DNS, gateways, and network rules. Each follows a distinct retention and deletion mechanism.
Use two-person approval for destructive database and backup deletion. Record identifiers, counts, checksums, approvals, timestamps, and exceptions without copying shopper queries, handoff summaries, or secrets into the teardown report.
If retention requires continued access, freeze a read-only export service with support dispatch and model calls disabled, narrow identities, monitor access, and schedule final deletion.
System changes
- Disables traffic and dispatch, revokes credentials, removes workloads, and only after separate approval deletes catalog, vector, inventory, handoff, audit, and backup state.
TEARDOWN_READY search=disabled dispatch=disabled inflight=0 ambiguous=0 final_catalog=catalog-2026-07-29-03 backup_restored=true holds=1 credentials_revoked=9 volume_delete=pending_approval
Checkpoint: Commerce teardown authorized
Continue whenNo new search or support request occurs, every handoff has authoritative state, final catalog and backup checksums exist, restore succeeded, holds remain, credentials are revoked, and deletion has named approvers.
Stop whenStop if any handoff is ambiguous, the final backup has not restored, merchant export does not match catalog revision, a hold is unresolved, or credentials are shared with another workload.
Security notes
- Do not use volume deletion as the first teardown command; durable state deletion is a separate, reviewed destructive operation.
Alternatives
- Retain a minimal read-only merchant export and audit service when legal, warranty, or support retention prevents immediate deletion.
Stop conditions
- Merchant, privacy, support, or legal owner has not approved data deletion.
- Final catalog provenance, approval audit, backup checksum, or restore evidence is missing.
Finish line
Verification checklist
npm testProduct schema, content hash, vector dimension, stock freshness, candidate membership, generated-fact rejection, and database rendering tests all pass without network.node test/search-eval.mjs --catalog synthetic/catalog.ndjson --queries synthetic/labeled-queries.jsonEvery explicit active, category, and price constraint is honored; measured top-k recall meets the release threshold; no unknown product ID reaches an answer.node test/http-contract.mjs --case search-handoff-approval-dispatchSearch creates at most a local draft with zero external writes, separate merchant approval is required, and the dispatcher sends the exact approved body once.node test/recovery.mjs --backup "$BACKUP_PATH" --restore "$RESTORE_DATABASE_URL"Catalog and inventory provenance, embeddings, vector model, handoff states, dispatch IDs, and audit restore consistently; merchant export matches the recorded source hash.Recovery guidance
Common problems and safe checks
Catalog import rejects a valid-looking product row.
Likely causeA strict field, URL, timestamp, currency, attribute type, size, or duplicate constraint differs from the merchant export.
Inspect sanitized row number and schema pathCompare export contract and revisionValidate URL host, currency, and timestamp independently
ResolutionCorrect the authoritative export or explicitly version the schema; do not loosen validation or let the model repair catalog facts.
Import embeds every product on each revision.
Likely causeCanonical content order, whitespace, attribute serialization, model label, or hash calculation is unstable.
Compare content hashes for an unchanged synthetic productInspect sorted attribute serializationCheck embedding model revision
ResolutionCanonicalize deterministic descriptive content, keep price and inventory out of it, add hash fixtures, and rerun only after the previous revision remains available.
A category filter produces no candidates despite matching products.
Likely causeUser language and catalog taxonomy differ, the plan omitted a reviewed alias, or category comparison is case-sensitive by policy.
Inspect schema plan and exact product categoryRun parameterized SQL read-onlyCheck reviewed alias map
ResolutionMap explicit reviewed aliases before SQL or ask a clarifying question; do not let the model silently broaden or remove the category.
A price constraint returns a product over budget.
Likely causeCurrency, minor units, parser interpretation, or SQL filter binding is wrong.
Read plan minor units and currency policyCompare database integer priceRun exact filter fixture
ResolutionBlock the response, restore the last accepted parser and filter revision, add boundary and currency fixtures, and never use model prose for price comparison.
Search returns inactive products.
Likely causeThe active predicate is missing, a replica is stale, or catalog transaction was partially exposed.
Inspect exact SQL and authoritative rowCheck catalog revision transactionCompare replica lag
ResolutionFail the request or use the primary accepted revision, restore the hard active predicate, and add an inactive-candidate release test.
A model selects a product absent from candidates.
Likely causePrompt injection, hallucination, context mismatch, or an adapter mixed responses.
Compare candidate IDs and response IDCheck session and policy revisionRun unknown-product fixture
ResolutionReject the entire answer before rendering, retain only sanitized failure evidence, and retry only after correcting adapter or context.
Model prose says an item is available while the card says unknown.
Likely causeInventory fields leaked into candidate context or fact-language validation missed a phrase.
Inspect candidate projection and schemaRun the exact phrase through policy fixturesConfirm final assembler did not concatenate raw model text
ResolutionBlock the response, remove inventory from model context, extend fail-closed language validation, and keep availability only in deterministic card fields.
All inventory becomes stale at once.
Likely causeFeed outage, timestamp unit or timezone error, failed transaction, or clock skew exceeds the threshold.
Compare source observation and database timeInspect feed revision and import auditCheck trusted clock
ResolutionKeep stock unknown, restore the authoritative feed, correct timestamps without fabricating freshness, and notify catalog operations.
A sensitive preference is not routed safely.
Likely causeThe minimum detector lacks locale or phrasing coverage, or the query was transformed before policy evaluation.
Reproduce with privacy-safe synthetic textInspect policy order and localeReview retrieval features
ResolutionDisable the affected cohort, evaluate policy before model and retrieval, expand reviewed locale-aware fixtures, and avoid storing the sensitive query.
A handoff draft contains too much conversation.
Likely causeThe schema or prompt permits transcript copying instead of a bounded purpose summary.
Check byte limit and stored fieldsInspect synthetic summary fixtureReview consent and retention
ResolutionReject the draft, reduce schema to purpose-bound fields, redact before storage, require operator edit and fresh approval, and delete invalid data under policy.
Merchant approval fails after editing the draft.
Likely causeApproval is correctly bound to immutable exact content, so the edit invalidated the previous review.
Compare draft ID and content hashRead current lifecycle stateCheck operator scope
ResolutionCreate a new draft revision, reject or supersede the old one, and obtain a new exact-content approval.
Support accepted a handoff but local state is still approved.
Likely causeThe response was lost after external commit and the local transaction rolled back.
Lookup support by handoff idempotency keyCompare destination and local request IDsInspect dispatcher timeout
ResolutionBlock retry, reconcile authoritative support state, and record a privileged audited dispatch result before resuming the worker.
Provider rate limits cause long shopper latency.
Likely causeTwo model calls plus embedding share insufficient provider capacity or application concurrency is unbounded.
Measure each stage and usageInspect application and provider rate countersCheck queue and timeout behavior
ResolutionDegrade to exact or cached deterministic search, bound concurrency, reduce context, request appropriate provider limits, and keep a strict latency timeout.
Database pool saturation blocks health and search.
Likely causeLong transactions, importer contention, dispatch network calls, or oversized replica pools consume connections.
Measure pool wait and active queriesInspect locks and import transaction durationSeparate outbox dispatch and importer capacity
ResolutionPause import or dispatch, shorten transactions, use bounded dedicated pools and staged revision switch, and preserve read-only search capacity.
Restored catalog has different semantic results.
Likely causeEmbedding rows, model labels, extension version, index, or application policy differ from backup metadata.
Compare vector counts, dimensions, and content hashesRun exact retrieval on labeled fixturesCheck extension and image revision
ResolutionUse matching immutable revisions or rebuild from approved content under the exact model, then meet labeled recall before declaring recovery complete.
Reference
Frequently asked questions
Why not ask the model for current price and stock?
Those facts change independently and belong to merchant systems. The model selects candidate IDs and explains attribute fit; the application reads current price and timestamped inventory after validation.
Does semantic similarity mean a product is suitable?
No. It is a retrieval signal. Explicit filters, structured attributes, candidate membership, policy limits, shopper judgment, and specialist handoff remain necessary.
What happens when inventory is stale?
The deterministic card shows unknown with freshness context. The assistant must not rewrite unknown as available or create scarcity language.
Can the assistant place an order or reserve stock?
No. This service has no order, payment, account, discount, or inventory mutation tools. It links to the canonical product page or drafts a human handoff.
Why are price and inventory excluded from embedding content?
They need exact filtering and fresh rendering, change frequently, and should not require semantic re-embedding. Embeddings represent descriptive product meaning.
Can a support handoff be sent automatically?
Not in this design. Search creates a local draft. A merchant operator approves exact content, then a separate dispatcher sends it idempotently to the fixed destination.
What if OpenAI or vector search is unavailable?
Degrade to exact or full-text catalog search and canonical product links. Do not lose catalog truth, invent reasons, or route to another provider without a reviewed policy.
How often should the guide be reviewed?
At least every ninety days, next due 2026-10-27, and immediately after model, embedding, schema, catalog, inventory, support, authentication, privacy, or recommendation-policy changes.
Recovery
Rollback
Rollback independently disables support dispatch, generated reasons, and semantic retrieval while preserving exact catalog browsing and durable handoff state. Restore the previous immutable application, policy, catalog revision, and compatible vector index; never replay an ambiguous support request or reinterpret stale inventory during rollback.
- Disable handoff dispatch and block the dispatcher identity first; preserve approved and ambiguous drafts for authoritative support reconciliation.
- Disable generated reasons and semantic retrieval, route shoppers to exact or static catalog search, and mark inventory according to its current source timestamp.
- Switch to the previous immutable application and policy revision, then restore or repoint the previous accepted catalog and vector revision without altering current authoritative inventory.
- Verify product IDs, exact filters, database-rendered price, freshness matrix, authentication, rate limits, audit insertion, and support kill switch before increasing traffic.
- Reconcile support by handoff ID, export required state, document relevance and safety regression, and schedule a corrected canary rather than silently mixing old and new embeddings.
Evidence