From 11199880099e0abb280f2f04c1f30b42d5cde0a1 Mon Sep 17 00:00:00 2001 From: Chris Troutner Date: Sun, 20 Sep 2026 14:12:13 -0700 Subject: [PATCH] Specify profile ordering by last post Add indexer profileRecency index, recent-profile ordering, and backfill specs; seed profileRecency in the recent-profile-identity fixture; mark the feature in progress in the backlog. By specifier. --- .../specs/backfill-profile-recency.feature | 58 ++++++++ .../specs/recent-profile-identity.feature | 16 ++- .../specs/recent-profile-ordering.feature | 65 +++++++++ .../specs/profile-recency-indexing.feature | 128 ++++++++++++++++++ specs/feature-backlog.md | 16 ++- 5 files changed, 277 insertions(+), 6 deletions(-) create mode 100644 psf-memo-db/specs/backfill-profile-recency.feature create mode 100644 psf-memo-db/specs/recent-profile-ordering.feature create mode 100644 psf-memo-indexer/specs/profile-recency-indexing.feature diff --git a/psf-memo-db/specs/backfill-profile-recency.feature b/psf-memo-db/specs/backfill-profile-recency.feature new file mode 100644 index 0000000..bdb57a6 --- /dev/null +++ b/psf-memo-db/specs/backfill-profile-recency.feature @@ -0,0 +1,58 @@ +# Scenarios: Backfill profile recency - 1, Backfill profile recency - 2, Backfill profile recency - 3 +# +# The profile recency backfill builds the profileRecency store from existing +# data so the read side can serve /profile/recent without a reindex. For each +# profile address it finds the newest confirmed qualifying post from the +# addrPostHeights index, using the posts store for the post's seen time and +# the postParents and polls stores to exclude replies and poll creations. +# Entries above status.chainBlockHeight are unconfirmed and ignored. Addresses +# with no qualifying post get no record. Running the backfill twice is +# idempotent. +# +# Fixture "profiles-with-post-history": +# profiles store: +# bitcoincash:qaddr-alice { text: alice bio, txid: profile-alice } +# bitcoincash:qaddr-bob { text: bob bio, txid: profile-bob } +# bitcoincash:qaddr-nopost { text: nopost bio, txid: profile-nopost } +# posts store: +# post-a1 { addr: bitcoincash:qaddr-alice, seen: 100, blockHeight: 600100 } +# reply-a1 { addr: bitcoincash:qaddr-alice, seen: 150, blockHeight: 600300 } +# post-b1 { addr: bitcoincash:qaddr-bob, seen: 200, blockHeight: 600400 } +# post-b2 { addr: bitcoincash:qaddr-bob, seen: 250, blockHeight: 600500 } +# addrPostHeights store: +# bitcoincash:qaddr-alice:post-a1 at 600100 +# bitcoincash:qaddr-alice:reply-a1 at 600300 +# bitcoincash:qaddr-alice:poll-a1 at 600200 +# bitcoincash:qaddr-bob:post-b1 at 600400 +# bitcoincash:qaddr-bob:post-b2 at 600500 +# postParents store: reply-a1 -> post-a1 +# polls store: poll-a1 +# status store: chainBlockHeight 600450 (post-b2 is above it and unconfirmed) +Feature: Backfill profile recency + + Background: + Given a psf-memo-db instance with profiles, posts, addrPostHeights, postParents, polls, status, and profileRecency stores + Given the fixture "profiles-with-post-history" is loaded into the profiles, posts, addrPostHeights, postParents, polls, status, and profileRecency stores + + Scenario Outline: Backfill profile recency - 1 backfill records each profile's newest confirmed qualifying post + When the profile recency backfill utility is run + Then the profileRecency store records at block height seen at + + Examples: + | addr | height | seen | + | bitcoincash:qaddr-alice | 600100 | 100 | + | bitcoincash:qaddr-bob | 600400 | 200 | + + Scenario: Backfill profile recency - 2 a profile with no qualifying post gets no recency record + When the profile recency backfill utility is run + Then the profileRecency store has no record for bitcoincash:qaddr-nopost + + Scenario Outline: Backfill profile recency - 3 backfill is idempotent + When the profile recency backfill utility is run + And the profile recency backfill utility is run again + Then the profileRecency store records at block height seen at + + Examples: + | addr | height | seen | + | bitcoincash:qaddr-alice | 600100 | 100 | + | bitcoincash:qaddr-bob | 600400 | 200 | diff --git a/psf-memo-db/specs/recent-profile-identity.feature b/psf-memo-db/specs/recent-profile-identity.feature index 4b48d23..39def67 100644 --- a/psf-memo-db/specs/recent-profile-identity.feature +++ b/psf-memo-db/specs/recent-profile-identity.feature @@ -11,14 +11,20 @@ # setProfilePic, 0x6d0a) are keyed by address, so the route joins each record # into its profile by address. A profile with no name record reports a null # display name, and a profile with no picture record reports a null avatar URL. -# The join does not change the profile order (block height descending) or the -# pagination metadata. +# The join does not change the profile order or the pagination metadata. # -# Fixture "profiles-with-identities" (profiles, names, and profilePics stores): +# Fixture "profiles-with-identities" (profiles, names, profilePics, and +# profileRecency stores). The profileRecency entries make each profile +# eligible for the recent list; their values are irrelevant to this identity +# join: # profiles: # bitcoincash:qaddr-alice { text: alice bio, txid: profile-alice, blockHeight: 600300, seen: 3 } # bitcoincash:qaddr-bob { text: bob bio, txid: profile-bob, blockHeight: 600200, seen: 2 } # bitcoincash:qaddr-carol { text: carol bio, txid: profile-carol, blockHeight: 600100, seen: 1 } +# profileRecency: +# bitcoincash:qaddr-alice at 600300 seen 3 +# bitcoincash:qaddr-bob at 600200 seen 2 +# bitcoincash:qaddr-carol at 600100 seen 1 # names: # bitcoincash:qaddr-alice { name: alice, txid: name-alice, blockHeight: 600250 } # bitcoincash:qaddr-carol { name: carol, txid: name-carol, blockHeight: 600050 } @@ -28,8 +34,8 @@ Feature: Recent Profile Identity Background: - Given a psf-memo-db instance with profiles, names, and profilePics stores - Given the fixture "profiles-with-identities" is loaded into the profiles, names, and profilePics stores + Given a psf-memo-db instance with profiles, names, profilePics, and profileRecency stores + Given the fixture "profiles-with-identities" is loaded into the profiles, names, profilePics, and profileRecency stores Scenario Outline: Recent Profile Identity - 1 GET /profile/recent returns each profile's display name and avatar When the client requests /profile/recent diff --git a/psf-memo-db/specs/recent-profile-ordering.feature b/psf-memo-db/specs/recent-profile-ordering.feature new file mode 100644 index 0000000..3e0e61c --- /dev/null +++ b/psf-memo-db/specs/recent-profile-ordering.feature @@ -0,0 +1,65 @@ +# Scenarios: Recent Profile Ordering - 1, Recent Profile Ordering - 2, Recent Profile Ordering - 3, Recent Profile Ordering - 4 +# +# GET /profile/recent lists one row per profile address that has at least one +# confirmed qualifying post. Rows are ordered by the most recent qualifying +# post's block height descending, then that post's seen descending, then the +# address ascending. Each row reports the profile's text, provenance txid, +# display name, and avatar; its Block and Seen columns report the most recent +# qualifying post's blockHeight and seen, not the set-profile transaction's. +# A profile with no qualifying post is omitted and does not count toward +# pagination.total. +# +# The page is served from the profileRecency index plus per-page profile +# lookups, so it never scans the addrPostHeights store and never sorts every +# profile to find the page. +# +# Fixture "profiles-with-post-recency": +# profiles store (the set-profile records; their own blockHeight/seen differ +# from the recency values so the two cannot be confused): +# bitcoincash:qaddr-alice { text: alice bio, txid: profile-alice, blockHeight: 600010, seen: 10 } +# bitcoincash:qaddr-bob { text: bob bio, txid: profile-bob, blockHeight: 600020, seen: 20 } +# bitcoincash:qaddr-erin { text: erin bio, txid: profile-erin, blockHeight: 600030, seen: 30 } +# bitcoincash:qaddr-carol { text: carol bio, txid: profile-carol, blockHeight: 600040, seen: 40 } +# bitcoincash:qaddr-dave { text: dave bio, txid: profile-dave, blockHeight: 600050, seen: 50 } +# profileRecency store (the newest qualifying post per profile): +# bitcoincash:qaddr-alice at 600300 seen 300 +# bitcoincash:qaddr-bob at 600300 seen 200 +# bitcoincash:qaddr-erin at 600300 seen 200 +# bitcoincash:qaddr-carol at 600200 seen 400 +# (bitcoincash:qaddr-dave has never posted, so it has no recency record) +Feature: Recent Profile Ordering + + Background: + Given a psf-memo-db instance with profiles and profileRecency stores + Given the fixture "profiles-with-post-recency" is loaded into the profiles and profileRecency stores + + Scenario Outline: Recent Profile Ordering - 1 GET /profile/recent orders profiles by their most recent post + When the client requests /profile/recent with limit and offset + Then the response lists profiles in order () + And the response pagination shows total and hasMore + + Examples: + | limit | offset | expected_addrs | total | hasMore | + | 2 | 0 | bitcoincash:qaddr-alice,bitcoincash:qaddr-bob | 4 | true | + | 2 | 2 | bitcoincash:qaddr-erin,bitcoincash:qaddr-carol | 4 | false | + | 1 | 1 | bitcoincash:qaddr-bob | 4 | true | + | 5 | 0 | bitcoincash:qaddr-alice,bitcoincash:qaddr-bob,bitcoincash:qaddr-erin,bitcoincash:qaddr-carol | 4 | false | + + Scenario Outline: Recent Profile Ordering - 2 each row reports the most recent post's block and seen + When the client requests /profile/recent with limit 5 and offset 0 + Then the response profile for has block height and seen + + Examples: + | addr | height | seen | + | bitcoincash:qaddr-alice | 600300 | 300 | + | bitcoincash:qaddr-bob | 600300 | 200 | + | bitcoincash:qaddr-carol | 600200 | 400 | + + Scenario: Recent Profile Ordering - 3 a profile with no qualifying post is omitted + When the client requests /profile/recent with limit 5 and offset 0 + Then the response lists profiles in order (bitcoincash:qaddr-alice,bitcoincash:qaddr-bob,bitcoincash:qaddr-erin,bitcoincash:qaddr-carol) + And the response does not list bitcoincash:qaddr-dave + + Scenario: Recent Profile Ordering - 4 the read does not iterate addrPostHeights + When the client requests /profile/recent with limit 5 and offset 0 + Then the addrPostHeights store was not iterated diff --git a/psf-memo-indexer/specs/profile-recency-indexing.feature b/psf-memo-indexer/specs/profile-recency-indexing.feature new file mode 100644 index 0000000..5406bc2 --- /dev/null +++ b/psf-memo-indexer/specs/profile-recency-indexing.feature @@ -0,0 +1,128 @@ +# Scenarios: Profile Recency Indexing - 1, Profile Recency Indexing - 2, Profile Recency Indexing - 3, Profile Recency Indexing - 4, Profile Recency Indexing - 5, Profile Recency Indexing - 6, Profile Recency Indexing - 7, Profile Recency Indexing - 8, Profile Recency Indexing - 9, Profile Recency Indexing - 10 +# +# The indexer maintains a profileRecency store so the read side can list +# profiles by their author's most recent post without scanning every post or +# sorting every profile. profileRecency holds one record per profile address +# that has at least one qualifying post, carrying that post's blockHeight and +# seen. +# +# A qualifying post is a top-level post (0x6d02) or a topic message (0x6d0c). +# Replies (0x6d03) and poll creations (0x6d10) do not qualify. Recency is only +# written for addresses that have a set-profile record (0x6d05), so an address +# that has posted but never set a profile is absent. +# +# Recency reflects confirmed blocks only: a post first seen in the mempool does +# not create or change a profileRecency record, and the record is written when +# the block containing that post is indexed. Processing qualifying posts in any +# order converges on the greatest height, with seen as the tie-breaker at equal +# heights, so a replay never regresses a profile's recency. +Feature: Profile Recency Indexing + + Background: + Given a psf-memo-db instance with profiles and profileRecency stores + Given a psf-memo-indexer configured to write to that database + + Scenario Outline: Profile Recency Indexing - 1 a confirmed top-level post records the author's recency + Given the psf-memo-db stores a profile for + When the indexer processes a Memo post transaction from at block height with text "" seen at + Then the profileRecency store records at block height seen at + + Examples: + | addr | txid | height | text | seen | + | bitcoincash:qaddr-a | post-a1 | 600100 | hello | 1000 | + | bitcoincash:qaddr-b | post-b1 | 600250 | world | 2000 | + + Scenario Outline: Profile Recency Indexing - 2 a confirmed topic post records the author's recency + Given the psf-memo-db stores a profile for + When the indexer processes a Memo topic message in room "" from at block height with text "" seen at + Then the profileRecency store records at block height seen at + + Examples: + | addr | txid | room | height | text | seen | + | bitcoincash:qaddr-a | topic-a1 | bitcoin | 600300 | gm | 3000 | + | bitcoincash:qaddr-b | topic-b1 | cash | 600150 | hi | 1500 | + + Scenario Outline: Profile Recency Indexing - 3 the greatest height wins regardless of processing order + Given the psf-memo-db stores a profile for + When the indexer processes a Memo post transaction from at block height with text "first" seen at + And the indexer processes a Memo post transaction from at block height with text "second" seen at + Then the profileRecency store records at block height seen at + + Examples: + | addr | firstTxid | firstHeight | firstSeen | secondTxid | secondHeight | secondSeen | height | seen | + | bitcoincash:qaddr-a | post-a1 | 600100 | 100 | post-a2 | 600200 | 200 | 600200 | 200 | + | bitcoincash:qaddr-b | post-b1 | 600200 | 100 | post-b2 | 600100 | 200 | 600200 | 100 | + + Scenario Outline: Profile Recency Indexing - 4 a reply does not change the recency + Given the psf-memo-db stores a profile for + When the indexer processes a Memo post transaction from at block height with text "root" seen at 100 + And the indexer processes a Memo reply transaction to parent from at block height with text "reply" + Then the profileRecency store records at block height seen at 100 + + Examples: + | addr | postTxid | postHeight | replyTxid | replyHeight | + | bitcoincash:qaddr-a | post-a1 | 600100 | reply-a1 | 600200 | + | bitcoincash:qaddr-b | post-b1 | 600300 | reply-b1 | 600400 | + + Scenario Outline: Profile Recency Indexing - 5 a poll creation does not change the recency + Given the psf-memo-db stores a profile for bitcoincash:qaddr-a + When the indexer processes a Memo post transaction from bitcoincash:qaddr-a at block height with text "root" seen at 100 + And the indexer processes a create-poll transaction with the question "" and 2 options + Then the profileRecency store records bitcoincash:qaddr-a at block height seen at 100 + + Examples: + | postTxid | postHeight | question | + | post-a1 | 600050 | best coin? | + | post-a2 | 600070 | lunch? | + + Scenario Outline: Profile Recency Indexing - 6 an author without a profile gets no recency record + When the indexer processes a Memo post transaction from at block height with text "hi" seen at 100 + Then the profileRecency store has no record for + + Examples: + | addr | txid | height | + | bitcoincash:qaddr-noprofile | post-n1 | 600100 | + | bitcoincash:qaddr-other | post-o1 | 600200 | + + Scenario Outline: Profile Recency Indexing - 7 setting a profile after posting establishes the recency + When the indexer processes a Memo post transaction from at block height with text "old post" seen at 100 + And the indexer processes a Memo reply transaction to parent from at block height with text "newer reply" + And the indexer processes a set-profile transaction for with text "my bio" + Then the profileRecency store records at block height seen at 100 + + Examples: + | addr | postTxid | postHeight | replyTxid | replyHeight | + | bitcoincash:qaddr-a | post-a1 | 600100 | reply-a1 | 600300 | + | bitcoincash:qaddr-b | post-b1 | 600200 | reply-b1 | 600250 | + + Scenario Outline: Profile Recency Indexing - 8 setting a profile with no qualifying post creates no recency record + When the indexer processes a set-profile transaction for with text "my bio" + Then the profileRecency store has no record for + + Examples: + | addr | + | bitcoincash:qaddr-a | + | bitcoincash:qaddr-b | + + Scenario Outline: Profile Recency Indexing - 9 an unconfirmed post does not affect recency until its block is indexed + Given the psf-memo-db stores a profile for + When the transaction indexer sees a Memo post transaction from at block height with text "mempool" + Then the profileRecency store has no record for + When the indexer processes a Memo post transaction from at block height with text "mempool" seen at + Then the profileRecency store records at block height seen at + + Examples: + | addr | txid | height | seen | + | bitcoincash:qaddr-a | post-a1 | 600100 | 1000 | + | bitcoincash:qaddr-b | post-b1 | 600400 | 4000 | + + Scenario Outline: Profile Recency Indexing - 10 reprocessing a qualifying post is idempotent + Given the psf-memo-db stores a profile for + When the indexer processes a Memo post transaction from at block height with text "hi" seen at 100 + And the indexer processes the same Memo post transaction again + Then the profileRecency store records at block height seen at 100 + + Examples: + | addr | txid | height | + | bitcoincash:qaddr-a | post-a1 | 600100 | + | bitcoincash:qaddr-b | post-b1 | 600200 | diff --git a/specs/feature-backlog.md b/specs/feature-backlog.md index 4f8b960..913d4f5 100644 --- a/specs/feature-backlog.md +++ b/specs/feature-backlog.md @@ -31,7 +31,21 @@ focus is **front-end improvements** to `psf-memo-client` (the React SPA). ## In progress -- None. +- **Profile ordering by last post (2026-09-20):** `/profile/recent` should + list only profiles that have posted, ordered by their most recent qualifying + post instead of their most recent profile update. A qualifying post is a + top-level post (`0x6d02`) or a topic message (`0x6d0c`); replies (`0x6d03`) + and poll creations (`0x6d10`) do not qualify. Profiles that have never posted + are dropped. Ordering is by last post's block height descending, then seen + descending, then address ascending, considering confirmed blocks only. The + Block and Seen columns report the last post's block height and timestamp + rather than the set-profile transaction's. The indexer maintains a + `profileRecency` index (mirroring `topicRecency`) so the read never scans + `addrPostHeights` or sorts every profile; a backfill builds it from existing + data. Indexer + DB (the client renders the returned block/seen unchanged). + Specs: `psf-memo-indexer/specs/profile-recency-indexing.feature`, + `psf-memo-db/specs/recent-profile-ordering.feature`, + `psf-memo-db/specs/backfill-profile-recency.feature`. ## Recently completed