mirror of
https://github.com/Permissionless-Software-Foundation/psf-memo.git
synced 2026-09-23 01:32:00 -07:00
Update backlog and briefing after feed-query-performance merge
By specifier.
This commit is contained in:
+21
-11
@@ -366,6 +366,15 @@ that a single user-facing feature may require specs in more than one component.
|
||||
address on `:` and yielded just `"bitcoincash"` — strip the trailing
|
||||
`:<followeePkHash>` suffix via `key.slice(0, key.lastIndexOf(':'))` instead.
|
||||
Spec: `psf-memo-client/specs/mute-feed-filtering.feature`.
|
||||
21. **The recent-feed `total` is now capped, not exact.** Since the
|
||||
feed-query-performance job (`2bcc965`), `GET /posts/recent` computes
|
||||
`total`/`hasMore` from a capped scan of the last `TOTAL_SCAN_CAP` (10)
|
||||
top-level posts rather than a full `postHeights` walk. `total` is therefore
|
||||
`min(actual, TOTAL_SCAN_CAP)` and `hasMore` is only reliable for the first
|
||||
few pages; deep pagination past the cap may report `hasMore: false` even
|
||||
when older posts exist. Specs for the recent feed should assert `total`
|
||||
against the cap (e.g. `10`) and only assert `hasMore` for the first pages.
|
||||
Spec: `psf-memo-db/specs/feed-query-performance.feature`.
|
||||
|
||||
---
|
||||
|
||||
@@ -403,14 +412,15 @@ At the end of each session, update this file:
|
||||
- Note the current `master` HEAD commit.
|
||||
- State the next feature to work on.
|
||||
|
||||
Current `master` HEAD: `3992395` (merged architect's mute-feed-filtering job —
|
||||
muting a profile now hides its content from the viewer's recent feed, topic feed,
|
||||
search results, and notifications. Server-side filtering keyed by the viewer's
|
||||
address passed as a `viewer` query param; shared `loadMutedAddrs`/`isMutedPost`
|
||||
helper in `psf-memo-db/src/adapters/lib/muted-posts.js`. Verified DB 343 unit +
|
||||
40 property + 9 acceptance passing + lint clean; client build OK + 284 unit +
|
||||
24 acceptance passing + lint clean, including the new `mute-feed-filtering`
|
||||
suite).
|
||||
Next action: **spec the feed query performance work** (capped scan for the
|
||||
`total`/`hasMore` computation + per-page reply counting) — see
|
||||
`specs/feature-backlog.md` "Next up: feed query performance".
|
||||
Current `master` HEAD: `2bcc965` (merged architect's feed-query-performance job —
|
||||
`GET /posts/recent` no longer does two full scans per request. Reply counts are
|
||||
computed per returned post (`countRepliesForTxids`) instead of a global
|
||||
`buildReplyCountMap()` scan, and the `total`/`hasMore` computation is a capped
|
||||
scan of the last `TOTAL_SCAN_CAP` (10) top-level posts instead of walking the
|
||||
whole `postHeights` index. `list-recent-posts.js` uses
|
||||
`scanRecentPostTxidsAndCount()` which returns page txids plus a capped total in
|
||||
one bounded scan. Verified DB unit + 10 acceptance passing + lint clean,
|
||||
including the new `feed-query-performance` suite).
|
||||
Next action: **TBD** — current direction is front-end improvements to
|
||||
`psf-memo-client` (UI/UX polish, accessibility, performance, responsiveness,
|
||||
state handling, error surfacing). See `specs/feature-backlog.md`.
|
||||
|
||||
+20
-17
@@ -61,6 +61,16 @@ focus is **front-end improvements** to `psf-memo-client` (the React SPA).
|
||||
of the raw URL; surrounding text is preserved; non-embeddable URLs stay plain
|
||||
text. Client-only rendering feature. Spec:
|
||||
`psf-memo-client/specs/youtube-embed.feature`. Merged to `master` at `b63019c`.
|
||||
- **Feed query performance (2026-09-05):** `GET /posts/recent` no longer does
|
||||
two full scans per request. Reply counts are computed per returned post
|
||||
(`countRepliesForTxids`) instead of a global `buildReplyCountMap()` scan, and
|
||||
the `total`/`hasMore` computation is a capped scan of the last
|
||||
`TOTAL_SCAN_CAP` (10) top-level posts instead of walking the whole
|
||||
`postHeights` index. `list-recent-posts.js` now uses
|
||||
`scanRecentPostTxidsAndCount()` which returns the page txids plus a capped
|
||||
total in one bounded scan. Spec:
|
||||
`psf-memo-db/specs/feed-query-performance.feature`. Merged to `master` at
|
||||
`2bcc965`.
|
||||
|
||||
---
|
||||
|
||||
@@ -137,25 +147,18 @@ Reference: https://memo.sv/protocol (Wayback snapshot 2025-12-15)
|
||||
|
||||
---
|
||||
|
||||
## Next up: feed query performance
|
||||
## Next up: feed query performance — DONE (2026-09-05)
|
||||
|
||||
`GET /posts/recent` (and the other paginated feeds) is slow at 1.3M posts because
|
||||
`list-recent-posts.js` does two full scans on every request:
|
||||
`GET /posts/recent` previously did two full scans per request:
|
||||
`countTopLevelPosts()` walked the entire `postHeights` index for
|
||||
`total`/`hasMore`, and `buildReplyCountMap()` scanned all `postChildren` entries.
|
||||
Both are now bounded: reply counts are per-page (`countRepliesForTxids`) and the
|
||||
total scan is capped to the last `TOTAL_SCAN_CAP` (10) top-level posts. Merged
|
||||
to `master` at `2bcc965`.
|
||||
|
||||
- `countTopLevelPosts()` iterates the ENTIRE `postHeights` index to compute the
|
||||
`total`/`hasMore` pagination field.
|
||||
- `buildReplyCountMap()` scans ALL `postChildren` entries to build a global
|
||||
reply-count map.
|
||||
|
||||
Planned optimization (decision: **capped scan**, keep it simple):
|
||||
|
||||
- Replace the global `buildReplyCountMap()` with per-page-txid reply counting
|
||||
(like `countLikesForTxids` already does) — only count replies for the ~50
|
||||
posts on the page.
|
||||
- Cap the `total` scan to the last N posts / last N blocks so `hasMore` still
|
||||
works for the first pages without walking all 1.3M entries.
|
||||
|
||||
Affected components: `psf-memo-db` (feed use cases + post-query adapter).
|
||||
Next feature: TBD — current direction is front-end improvements to
|
||||
`psf-memo-client` (UI/UX polish, accessibility, performance, responsiveness,
|
||||
state handling, error surfacing).
|
||||
|
||||
## Notes for future cycles
|
||||
|
||||
|
||||
Reference in New Issue
Block a user