psf-memo/doc/quality-level-proposal.md

Proposal: Configurable quality level per project

Status: proposal pending implementation — working document. Origin: evaluation of the saas-prototype run (see the project's REPORT.md): fixed gates (CRAP≤6, mutation run, DRY, soft Gherkin) cost tokens that not every project needs. This proposal adds a configurable quality axis per project. Note: the two-pack review (section 5) narrows the proposal — the real value is depth within a pack, not replacing pack choice.

1. The 3 levels

Level	Relative cost	Typical use
minimal	~1x	Prototypes, spikes, throwaway code
standard	~2x	Reasonable default for product features
maximum	~3-4x	Current rigor — critical libraries, security/payments, code consumed by others

maximum = what the pipeline already does today (nothing beyond that for now).

2. Gates by level

Gate	minimal	standard	maximum (current)
TDD + unit tests	✓	✓	✓
Acceptance Gherkin	optional	✓ (without full APS pipeline)	✓ full APS pipeline
CRAP	—	improve what is reasonable, no hard gate	≤6
DRY	—	reduce reasonable duplication	tooling, strict
Mutation scan + split >100 sites	—	✓ (count only — cheap)	✓
Full mutation run	—	—	✓ differential, kill non-equivalents
Soft Gherkin mutation	—	—	✓
Property tests	—	—	support
Written report	—	optional	✓

3. Role responsibility by level (four-pack)

Role	minimal	standard	maximum
specifier	Scoping + human gate only (no mandatory Gherkin)	Gherkin ✓	Gherkin + QA suite
coder	Implements with TDD — required	✓	✓
refactorer	No gates → no real work	Reasonable CRAP + DRY + mutation scan	full gates
architect	No gates → no real work	light structural review only	full gates

Conclusion: level and workflow are correlated. At minimal, refactorer/architect have no gates to apply — configuring 4 roles would waste tokens with no benefit.

4. Mechanism (prompts/articles only, zero code)

1. PROJECT (project.prompt):         "## Quality Level → Quality level: standard"
2. SHARED (quality.prompt):          the ON/OFF gate table by level
3. EACH ROLE PROMPT (one line):      "Apply only the gates that are ON for
                                     the project's level (see quality article)"

The agent reads the level in project.prompt, the table in quality.prompt, and its role prompt tells it to apply only the ON gates → consistent interpretation across roles.

The quality.prompt article also includes the role mapping by level: "at minimal, configure only specifier+coder (2 windows in swarmforge.conf); at standard, add refactorer; at maximum, all 4".

Honest nuance: the adjustment is prompt-soft — agents follow the level by instruction. Hard enforcement would need a tools/quality-check (validate level artifacts), an optional later step.

5. Review: does two-pack already solve part of this?

Result of reviewing two-pack's real scope (original project):

• coder (two-pack): TDD + unit tests ONLY — explicitly excludes acceptance, Gherkin, IR, Gherkin mutation, property tests, CRAP, DRY, and language mutation.
• cleaner (two-pack, batch): coverage, CRAP≤6, DRY, structure/encapsulation/dependencies and mutation run on uncovered behavior + tests to kill mutants.

two-pack quality profile: unit tests ✓ · CRAP≤6 ✓ · DRY ✓ · mutation run ✓ · structure ✓ (inside cleaner) · acceptance/Gherkin ✗ · property ✗ · separate QA ✗.

Conclusion

1. two-pack is NOT minimal: it keeps the hard hardening gates (CRAP≤6, DRY, mutation run). It is "full hardening without specification" — not the cheap option on the depth axis.
2. Packs already encode a quality axis: which gates EXIST (two-pack: no spec; four-pack: spec + architecture; six-pack: + hardender + QA).
3. The level axis adds what packs do NOT cover: the DEPTH of each active gate (CRAP≤6 vs "improve reasonably"; mutation run vs scan-only; soft Gherkin on/off).
4. Practical implication: for "cheap", choosing two-pack already drops the expensive layers (spec/architecture) — the main cost lever is the pack. Level is for scaling depth WITHIN a pack (e.g. two-pack without mutation run, four-pack without Gherkin mutation). The run confirmed it: the main waste was four-pack for a login form, not gate depth.

Verdict: the level proposal remains valid but is narrower than it first seemed: its real value is depth within a pack, not replacing pack choice. Possible simplification: start with only two levels (standard = current, light = no mutation run or Gherkin mutation) and let pack choice do the rest.

6. Analysis: spec vs hardening priority (the critique of two-pack)

two-pack's logic: TDD already specifies behavior at the unit-test level; Gherkin is a second layer (reviewable contract + end-to-end acceptance) that is expensive (APS pipeline); hardening gates (CRAP≤6, DRY, mutation run) are the code quality floor.

The critique (valid): for a small task the priority is inverted — mutation run is expensive and protects code that may be thrown away in a prototype; cheap spec ensures the RIGHT thing is built. A well-hardened but wrong feature is still wrong. Logical order: first WHAT (spec), then HOW (gates).

	two-pack	spec-first variant (proposal)
Base spec	unit tests (TDD)	Light Gherkin (reviewable contract + human approval) + TDD
Code protection	CRAP≤6 + DRY + mutation run	Reasonable CRAP/DRY, no mutation run
Cost	~2-3x	~1.5-2x
Risk covered	dirty/unchangeable code	building the wrong thing

The gap it reveals: two-pack assumes Gherkin comes with the full APS pipeline cost (parser + entrypoint generator + runtime + step handlers). It offers no "spec-lite" variant: write Gherkin as a reviewable contract without building the pipeline or running mutation.

Refinement of the light level:

light = Gherkin written as contract + human approval + TDD + reasonable CRAP/DRY — no APS pipeline, no mutation run, no Gherkin mutation, no property tests.

This variant covers the most important risk (is it the right thing? does a human approve?) at lower cost than "mutation run without spec".

7. Spec-light: Gherkin or other alternatives?

Comparison of options for a cheap, reviewable contract:

Option	Cost	Human pre-code contract	Real enforcement	Risk	Upgrade to spec-full
TDD tests as spec (two-pack)	~1x	❌	✅	user sees behavior at the end	—
Prose criteria (markdown)	~1x	✅ imprecise	❌	ambiguity	rewrite
Gherkin written-only (light)	~1.5x	✅ precise	❌	spec drift	✅ zero rewrite
Given/When/Then scenarios in markdown	~1x	✅	❌	no standard format	medium rewrite

Key point: in light, real enforcement comes from the coder's TDD — it turns each approved scenario into unit tests. Gherkin remains a human contract + guide, not verification. Spec drift risk is mitigated by a light workflow rule: "the coder maps each approved scenario to unit tests; the handoff/report confirms the scenario→tests mapping".

Recommendation: in four-pack, light = natural degradation — the specifier already writes Gherkin and asks for approval; the back half of the pipeline is cut (the coder does not build entrypoint generator/runtime/step handlers, implements with TDD mapping scenarios). If you later scale to spec-full, the .feature files are already there — you only build the pipeline around them.

Recommended cheap add-on: in light, the coder runs gherkin-parser ONLY to validate that the spec parses (seconds, no pipeline build) — prevents broken Gherkin syntax from passing as a contract.

When to choose each: never going to scale → prose or TDD-only; may scale → Gherkin written-only (the format IS the upgrade path); human must approve before coding → Gherkin.