mirror of
https://github.com/Permissionless-Software-Foundation/psf-memo.git
synced 2026-09-21 16:52:01 -07:00
Guard mutation runs against differential under-selection
The default differential run can select fewer covered sites than exist after a function is added or removed, forcing a manual --mutate-all rerun. Add mutate-file.sh (auto-reruns when Selected < Covered) and clean-builds.sh (removes worker-copy bloat), and document them plus the intrinsic upper-bound Gherkin survivors in the architect cheat-sheet. By specifier.
This commit is contained in:
@@ -20,11 +20,13 @@ the last run. If you add tests to kill survivors but the SOURCE is unchanged, a
|
||||
plain re-run reports `Killed: 0, Survived: 0, Uncovered: 0` and silently skips
|
||||
the survivors.
|
||||
|
||||
**Always pass `--mutate-all` when re-running mutation on a file that already has
|
||||
a manifest** (especially right after adding hardening tests). New files (no
|
||||
manifest) run fully on the first pass.
|
||||
|
||||
Canonical invocation (run from the component dir so `npm test` works):
|
||||
Prefer `swarmforge/scripts/mutate-file.sh`, which detects differential
|
||||
under-selection automatically (`Selected < Covered`) and reruns once with
|
||||
`--mutate-all`. Run it from the component directory:
|
||||
```bash
|
||||
../swarmforge/scripts/mutate-file.sh src/<file>.js --max-workers 8
|
||||
```
|
||||
Manual fallback when you must run the binary directly:
|
||||
```bash
|
||||
node ../psf-memo-client/node_modules/mutate4javascript/bin/mutate4javascript.js \
|
||||
src/<file>.js --max-workers 8 --mutate-all
|
||||
@@ -44,6 +46,14 @@ bb gherkin-mutator \
|
||||
Parse survivors with a small python one-liner over `d['results']` (filter
|
||||
`Status == 'survived'`).
|
||||
|
||||
### Equivalent upper-bound survivors (no project filter hook)
|
||||
APS `gherkin-mutator` exposes no project-specific mutation-filter hook, so
|
||||
upper-bound steps such as `the store was read at most <max_entries> entries`
|
||||
leave survivors when the bound is mutated upward (a larger bound can never
|
||||
fail). Treat these as intrinsic equivalents, document them in the review
|
||||
summary, and do not chase them. Prefer exact counts or independently-tied
|
||||
fixture values when a bound must itself be mutatable.
|
||||
|
||||
## Long runs
|
||||
- Mutation and gherkin-mutator runs take 60-180s each. Use `--max-workers 8` /
|
||||
`--workers 8` and `--status-interval` so progress is visible.
|
||||
@@ -56,8 +66,7 @@ Stale `tmp/acceptance` (LevelDB dirs) and `target/mutation-workers` can bloat to
|
||||
hang. `architect-startup.sh` now flags any of these over 100MB as `[FAIL]`.
|
||||
Clean them before any mutation run:
|
||||
```bash
|
||||
rm -rf psf-memo-db/tmp/acceptance psf-memo-db/target/mutation-workers \
|
||||
psf-memo-client/tmp/acceptance psf-memo-client/target/mutation-workers
|
||||
swarmforge/scripts/clean-builds.sh
|
||||
```
|
||||
This cut a notifications-query mutation run from 20+ minutes to ~2 minutes.
|
||||
|
||||
|
||||
Executable
+16
@@ -0,0 +1,16 @@
|
||||
#!/usr/bin/env bash
|
||||
# Remove gitignored build directories that bloat mutate4javascript worker
|
||||
# copies and make mutation runs appear to hang.
|
||||
#
|
||||
# Usage: clean-builds.sh
|
||||
set -euo pipefail
|
||||
|
||||
SCRIPT_DIR="$(cd "$(dirname "$0")" && pwd)"
|
||||
ROOT="$(cd "$SCRIPT_DIR/../.." && pwd)"
|
||||
|
||||
for component in psf-memo-client psf-memo-db psf-memo-indexer; do
|
||||
rm -rf "$ROOT/$component/tmp/acceptance" \
|
||||
"$ROOT/$component/target/mutation-workers"
|
||||
done
|
||||
|
||||
echo "clean-builds: removed tmp/acceptance and target/mutation-workers for all components"
|
||||
Executable
+53
@@ -0,0 +1,53 @@
|
||||
#!/usr/bin/env bash
|
||||
# Run mutate4javascript on one source file and guard against silent
|
||||
# differential under-selection.
|
||||
#
|
||||
# The default differential run can select fewer covered mutations than exist
|
||||
# after a function is added or removed. When Selected < Covered, this wrapper
|
||||
# reruns once with --mutate-all so new mutations are not skipped.
|
||||
#
|
||||
# Run from the component directory (so the tool's `npm test` baseline works):
|
||||
# swarmforge/scripts/mutate-file.sh src/<file>.js [options...]
|
||||
set -uo pipefail
|
||||
|
||||
if [ "$#" -lt 1 ]; then
|
||||
echo "usage: mutate-file.sh <source-file.js> [mutate4javascript options...]" >&2
|
||||
exit 2
|
||||
fi
|
||||
|
||||
SCRIPT_DIR="$(cd "$(dirname "$0")" && pwd)"
|
||||
ROOT="$(cd "$SCRIPT_DIR/../.." && pwd)"
|
||||
BIN="${MUTATE4JS_BIN:-$ROOT/psf-memo-client/node_modules/mutate4javascript/bin/mutate4javascript.js}"
|
||||
|
||||
if [ ! -f "$BIN" ]; then
|
||||
echo "mutate-file: cannot find mutate4javascript at $BIN" >&2
|
||||
exit 1
|
||||
fi
|
||||
|
||||
SOURCE="$1"
|
||||
shift
|
||||
|
||||
already_all=0
|
||||
for arg in "$@"; do
|
||||
[ "$arg" = "--mutate-all" ] && already_all=1
|
||||
done
|
||||
|
||||
mkdir -p "$ROOT/tmp"
|
||||
OUT="$(mktemp "$ROOT/tmp/mutate-file.XXXXXX")"
|
||||
trap 'rm -f "$OUT"' EXIT
|
||||
|
||||
node "$BIN" "$SOURCE" "$@" 2>&1 | tee "$OUT"
|
||||
status=${PIPESTATUS[0]}
|
||||
|
||||
if [ "$already_all" -eq 0 ]; then
|
||||
covered="$(grep -oE 'Covered mutation sites: [0-9]+' "$OUT" | grep -oE '[0-9]+' | tail -1)"
|
||||
selected="$(grep -oE 'Selected mutation sites: [0-9]+' "$OUT" | grep -oE '[0-9]+' | tail -1)"
|
||||
if [ -n "${covered:-}" ] && [ -n "${selected:-}" ] && [ "$selected" -lt "$covered" ]; then
|
||||
echo "" >&2
|
||||
echo "mutate-file: selected $selected of $covered covered sites (differential under-selection); rerunning with --mutate-all" >&2
|
||||
node "$BIN" "$SOURCE" "$@" --mutate-all
|
||||
status=$?
|
||||
fi
|
||||
fi
|
||||
|
||||
exit "$status"
|
||||
Reference in New Issue
Block a user