Update backlog and briefing after feed-query-performance merge

By specifier.
This commit is contained in:
Chris Troutner
2026-09-04 19:29:29 -07:00
parent 2bcc965efb
commit 13201d4923
2 changed files with 41 additions and 28 deletions
+21 -11
View File
@@ -366,6 +366,15 @@ that a single user-facing feature may require specs in more than one component.
address on `:` and yielded just `"bitcoincash"` — strip the trailing
`:<followeePkHash>` suffix via `key.slice(0, key.lastIndexOf(':'))` instead.
Spec: `psf-memo-client/specs/mute-feed-filtering.feature`.
21. **The recent-feed `total` is now capped, not exact.** Since the
feed-query-performance job (`2bcc965`), `GET /posts/recent` computes
`total`/`hasMore` from a capped scan of the last `TOTAL_SCAN_CAP` (10)
top-level posts rather than a full `postHeights` walk. `total` is therefore
`min(actual, TOTAL_SCAN_CAP)` and `hasMore` is only reliable for the first
few pages; deep pagination past the cap may report `hasMore: false` even
when older posts exist. Specs for the recent feed should assert `total`
against the cap (e.g. `10`) and only assert `hasMore` for the first pages.
Spec: `psf-memo-db/specs/feed-query-performance.feature`.
---
@@ -403,14 +412,15 @@ At the end of each session, update this file:
- Note the current `master` HEAD commit.
- State the next feature to work on.
Current `master` HEAD: `3992395` (merged architect's mute-feed-filtering job —
muting a profile now hides its content from the viewer's recent feed, topic feed,
search results, and notifications. Server-side filtering keyed by the viewer's
address passed as a `viewer` query param; shared `loadMutedAddrs`/`isMutedPost`
helper in `psf-memo-db/src/adapters/lib/muted-posts.js`. Verified DB 343 unit +
40 property + 9 acceptance passing + lint clean; client build OK + 284 unit +
24 acceptance passing + lint clean, including the new `mute-feed-filtering`
suite).
Next action: **spec the feed query performance work** (capped scan for the
`total`/`hasMore` computation + per-page reply counting) — see
`specs/feature-backlog.md` "Next up: feed query performance".
Current `master` HEAD: `2bcc965` (merged architect's feed-query-performance job —
`GET /posts/recent` no longer does two full scans per request. Reply counts are
computed per returned post (`countRepliesForTxids`) instead of a global
`buildReplyCountMap()` scan, and the `total`/`hasMore` computation is a capped
scan of the last `TOTAL_SCAN_CAP` (10) top-level posts instead of walking the
whole `postHeights` index. `list-recent-posts.js` uses
`scanRecentPostTxidsAndCount()` which returns page txids plus a capped total in
one bounded scan. Verified DB unit + 10 acceptance passing + lint clean,
including the new `feed-query-performance` suite).
Next action: **TBD** — current direction is front-end improvements to
`psf-memo-client` (UI/UX polish, accessibility, performance, responsiveness,
state handling, error surfacing). See `specs/feature-backlog.md`.
+20 -17
View File
@@ -61,6 +61,16 @@ focus is **front-end improvements** to `psf-memo-client` (the React SPA).
of the raw URL; surrounding text is preserved; non-embeddable URLs stay plain
text. Client-only rendering feature. Spec:
`psf-memo-client/specs/youtube-embed.feature`. Merged to `master` at `b63019c`.
- **Feed query performance (2026-09-05):** `GET /posts/recent` no longer does
two full scans per request. Reply counts are computed per returned post
(`countRepliesForTxids`) instead of a global `buildReplyCountMap()` scan, and
the `total`/`hasMore` computation is a capped scan of the last
`TOTAL_SCAN_CAP` (10) top-level posts instead of walking the whole
`postHeights` index. `list-recent-posts.js` now uses
`scanRecentPostTxidsAndCount()` which returns the page txids plus a capped
total in one bounded scan. Spec:
`psf-memo-db/specs/feed-query-performance.feature`. Merged to `master` at
`2bcc965`.
---
@@ -137,25 +147,18 @@ Reference: https://memo.sv/protocol (Wayback snapshot 2025-12-15)
---
## Next up: feed query performance
## Next up: feed query performance — DONE (2026-09-05)
`GET /posts/recent` (and the other paginated feeds) is slow at 1.3M posts because
`list-recent-posts.js` does two full scans on every request:
`GET /posts/recent` previously did two full scans per request:
`countTopLevelPosts()` walked the entire `postHeights` index for
`total`/`hasMore`, and `buildReplyCountMap()` scanned all `postChildren` entries.
Both are now bounded: reply counts are per-page (`countRepliesForTxids`) and the
total scan is capped to the last `TOTAL_SCAN_CAP` (10) top-level posts. Merged
to `master` at `2bcc965`.
- `countTopLevelPosts()` iterates the ENTIRE `postHeights` index to compute the
`total`/`hasMore` pagination field.
- `buildReplyCountMap()` scans ALL `postChildren` entries to build a global
reply-count map.
Planned optimization (decision: **capped scan**, keep it simple):
- Replace the global `buildReplyCountMap()` with per-page-txid reply counting
(like `countLikesForTxids` already does) — only count replies for the ~50
posts on the page.
- Cap the `total` scan to the last N posts / last N blocks so `hasMore` still
works for the first pages without walking all 1.3M entries.
Affected components: `psf-memo-db` (feed use cases + post-query adapter).
Next feature: TBD — current direction is front-end improvements to
`psf-memo-client` (UI/UX polish, accessibility, performance, responsiveness,
state handling, error surfacing).
## Notes for future cycles