OneLinersCommand workbench
AI
Back to skills
SKILL.md

Test gap analysis

An evidence-first workflow to find missing behavioral coverage around a proposed change.

Revision
2
Verified
2026-10-20
Save or explore
Save to collectionCreate a collection in the sidebar first.

Compatibility and paths

Codexskills/test-gap-analysis/SKILL.md
Claude Code.claude/skills/test-gap-analysis/SKILL.md
VS Code.github/skills/test-gap-analysis/SKILL.md

Trust and provenance

Curated record reviewed 2026-10-20. Results still depend on the supplied context and target environment.

Generated assetReady to copy or download
---
name: test-gap-analysis
description: Finds missing behavioral distinctions in tests for a concrete change. Use with requirements, diff and current tests; produce a prioritized counterexample that fails the proposed regression, not a generic request for more coverage.
license: CC-BY-4.0
compatibility: Offline review and the included owned Node22.13+ exercises need no credentials, network or package installation. Untrusted repository execution requires separate review.
metadata:
  author: oneliners
  version: "2.0.0"
---

# Test gap analysis

## Tie every test to a behavior and changed boundary
Request the requirement/version, exact diff, affected callers, current tests/fixtures and known incidents. Identify branch decisions, permission boundaries, data invariants, error paths and external effects touched by the change. Separate confirmed assertions from test names and comments: a test named rejects foreign tenant proves nothing unless it exercises and asserts that behavior. A green aggregate report or a coverage percentage does not establish that the right counterexample was tested.

## Worked precedence regression
This requirement deliberately scopes the public flag to one tenant; do not apply it blindly to a product with globally public files. The code is only a Boolean teaching function, not a complete access-control implementation.

```json
{
  "provenance": "synthetic Boolean policy; not a production authorization implementation",
  "requirement": "A tenant-scoped file may be read iff sameTenant AND (isOwner OR isPublicWithinTenant). Public here never means cross-tenant public internet access.",
  "approved": "return sameTenant && (isOwner || isPublicWithinTenant);",
  "candidate": "return sameTenant && isOwner || isPublicWithinTenant;",
  "existingTests": [
    {
      "sameTenant": true,
      "isOwner": true,
      "isPublicWithinTenant": false
    },
    {
      "sameTenant": true,
      "isOwner": false,
      "isPublicWithinTenant": true
    },
    {
      "sameTenant": true,
      "isOwner": true,
      "isPublicWithinTenant": true
    }
  ]
}
```

## Gap matrix and highest-value regression
Finding: existing3 tests all exercise allowed same-tenant outcomes. All3 pass under both approved and defective candidate logic, so a green run cannot distinguish this change. This is3 of the8 explicitly modeled Boolean combinations, not a claim of measured statement/branch coverage or complete application coverage.

| sameTenant | isOwner | tenant-public | Required | Candidate |
| --- | --- | --- | --- | --- |
| false | false | false | DENY | DENY |
| false | false | true | DENY | ALLOW — regression |
| false | true | false | DENY | DENY |
| false | true | true | DENY | ALLOW — regression |
| true | false | false | DENY | DENY |
| true | false | true | ALLOW | ALLOW |
| true | true | false | ALLOW | ALLOW |
| true | true | true | ALLOW | ALLOW |

Priority P1 test: sameTenant=false,isOwner=false,isPublicWithinTenant=true must DENY. It fails the candidate and passes the approved expression. Add the other tenant-public cross-tenant case and same-tenant non-owner/private denial, plus retain all3 allowed cases. JavaScript && binds before ||, so removing the parentheses changes the gate to (sameTenant && isOwner) || isPublicWithinTenant. The minimal proposed correction is to preserve the tenant gate around BOTH alternatives, not add a test that merely snapshots the defective output.

Additional evidence needed before production approval: verified identity-to-tenant mapping, missing/deleted file handling, authoritative server-side enforcement, other authorization paths and tests at the real boundary. Booleans supplied to a toy function are not authentication. The gap analysis identifies a concrete missing distinction; it did not query accounts, expose a file or patch a live authorization implementation.

## Turn a gap into a discriminating test
State exact input, expected output, why the requirement demands it, and how it differs from an existing passing case. First demonstrate that the proposed bad implementation fails that assertion in an owned fixture; then verify the intended implementation passes while existing positives remain intact. Do not weaken the assertion, alter the fixture to match the bug or update a snapshot without reviewing intent. The complete8-case matrix here is cheap; a larger real domain needs risk-selected boundaries, not an unexplained combinatorial explosion.

## Beyond the Boolean example
Add meaningful boundary values, invalid/missing input, partial failures, repeated operations, stale/concurrent state and side-effect non-occurrence where the change touches those risks. For a permission denial, assert no downstream write/response disclosure, not just an error message. For UI changes, use an actual user-observable state/accessible control assertion rather than implementation classes; isolate fixtures so another test cannot supply the needed state. Preserve the existing application's supported platforms and old behaviors rather than inventing new acceptance criteria.

## Output and verification limits
Return requirement→changed decision→existing assertion→missing case→expected result→risk/priority→test layer→owner. Identify duplicate tests and unresolved requirements separately. Report an actual failing/passing reproduction only when run; otherwise label it proposed. This example's matrix is computed from explicit inputs, not a browser/server/database test or proof of overall security. Do not install dependencies or execute unreviewed repository hooks during analysis. [Playwright's assertions and isolation guidance](https://playwright.dev/docs/writing-tests) is useful for browser checks; the fixture here is plain offline JavaScript.

Real example

Input

Find the missing behavior that distinguishes these synthetic implementations; do not access any real tenant/file.

{
  "provenance": "synthetic Boolean policy; not a production authorization implementation",
  "requirement": "A tenant-scoped file may be read iff sameTenant AND (isOwner OR isPublicWithinTenant). Public here never means cross-tenant public internet access.",
  "approved": "return sameTenant && (isOwner || isPublicWithinTenant);",
  "candidate": "return sameTenant && isOwner || isPublicWithinTenant;",
  "existingTests": [
    {
      "sameTenant": true,
      "isOwner": true,
      "isPublicWithinTenant": false
    },
    {
      "sameTenant": true,
      "isOwner": false,
      "isPublicWithinTenant": true
    },
    {
      "sameTenant": true,
      "isOwner": true,
      "isPublicWithinTenant": true
    }
  ]
}

Expected result

Finding: existing3 tests all exercise allowed same-tenant outcomes. All3 pass under both approved and defective candidate logic, so a green run cannot distinguish this change. This is3 of the8 explicitly modeled Boolean combinations, not a claim of measured statement/branch coverage or complete application coverage.

| sameTenant | isOwner | tenant-public | Required | Candidate |
| --- | --- | --- | --- | --- |
| false | false | false | DENY | DENY |
| false | false | true | DENY | ALLOW — regression |
| false | true | false | DENY | DENY |
| false | true | true | DENY | ALLOW — regression |
| true | false | false | DENY | DENY |
| true | false | true | ALLOW | ALLOW |
| true | true | false | ALLOW | ALLOW |
| true | true | true | ALLOW | ALLOW |

Priority P1 test: sameTenant=false,isOwner=false,isPublicWithinTenant=true must DENY. It fails the candidate and passes the approved expression. Add the other tenant-public cross-tenant case and same-tenant non-owner/private denial, plus retain all3 allowed cases. JavaScript && binds before ||, so removing the parentheses changes the gate to (sameTenant && isOwner) || isPublicWithinTenant. The minimal proposed correction is to preserve the tenant gate around BOTH alternatives, not add a test that merely snapshots the defective output.

Additional evidence needed before production approval: verified identity-to-tenant mapping, missing/deleted file handling, authoritative server-side enforcement, other authorization paths and tests at the real boundary. Booleans supplied to a toy function are not authentication. The gap analysis identifies a concrete missing distinction; it did not query accounts, expose a file or patch a live authorization implementation.

Source evidence

Agent Skills specificationofficialPlaywright test documentationofficial