mirror of
https://github.com/Permissionless-Software-Foundation/psf-memo.git
synced 2026-09-21 16:52:01 -07:00
Specify profile ordering by last post
Add indexer profileRecency index, recent-profile ordering, and backfill specs; seed profileRecency in the recent-profile-identity fixture; mark the feature in progress in the backlog. By specifier.
This commit is contained in:
Chris Troutner
@@ -0,0 +1,58 @@
|
||||
# Scenarios: Backfill profile recency - 1, Backfill profile recency - 2, Backfill profile recency - 3
|
||||
#
|
||||
# The profile recency backfill builds the profileRecency store from existing
|
||||
# data so the read side can serve /profile/recent without a reindex. For each
|
||||
# profile address it finds the newest confirmed qualifying post from the
|
||||
# addrPostHeights index, using the posts store for the post's seen time and
|
||||
# the postParents and polls stores to exclude replies and poll creations.
|
||||
# Entries above status.chainBlockHeight are unconfirmed and ignored. Addresses
|
||||
# with no qualifying post get no record. Running the backfill twice is
|
||||
# idempotent.
|
||||
#
|
||||
# Fixture "profiles-with-post-history":
|
||||
# profiles store:
|
||||
# bitcoincash:qaddr-alice { text: alice bio, txid: profile-alice }
|
||||
# bitcoincash:qaddr-bob { text: bob bio, txid: profile-bob }
|
||||
# bitcoincash:qaddr-nopost { text: nopost bio, txid: profile-nopost }
|
||||
# posts store:
|
||||
# post-a1 { addr: bitcoincash:qaddr-alice, seen: 100, blockHeight: 600100 }
|
||||
# reply-a1 { addr: bitcoincash:qaddr-alice, seen: 150, blockHeight: 600300 }
|
||||
# post-b1 { addr: bitcoincash:qaddr-bob, seen: 200, blockHeight: 600400 }
|
||||
# post-b2 { addr: bitcoincash:qaddr-bob, seen: 250, blockHeight: 600500 }
|
||||
# addrPostHeights store:
|
||||
# bitcoincash:qaddr-alice:post-a1 at 600100
|
||||
# bitcoincash:qaddr-alice:reply-a1 at 600300
|
||||
# bitcoincash:qaddr-alice:poll-a1 at 600200
|
||||
# bitcoincash:qaddr-bob:post-b1 at 600400
|
||||
# bitcoincash:qaddr-bob:post-b2 at 600500
|
||||
# postParents store: reply-a1 -> post-a1
|
||||
# polls store: poll-a1
|
||||
# status store: chainBlockHeight 600450 (post-b2 is above it and unconfirmed)
|
||||
Feature: Backfill profile recency
|
||||
|
||||
Background:
|
||||
Given a psf-memo-db instance with profiles, posts, addrPostHeights, postParents, polls, status, and profileRecency stores
|
||||
Given the fixture "profiles-with-post-history" is loaded into the profiles, posts, addrPostHeights, postParents, polls, status, and profileRecency stores
|
||||
|
||||
Scenario Outline: Backfill profile recency - 1 backfill records each profile's newest confirmed qualifying post
|
||||
When the profile recency backfill utility is run
|
||||
Then the profileRecency store records <addr> at block height <height> seen at <seen>
|
||||
|
||||
Examples:
|
||||
| addr | height | seen |
|
||||
| bitcoincash:qaddr-alice | 600100 | 100 |
|
||||
| bitcoincash:qaddr-bob | 600400 | 200 |
|
||||
|
||||
Scenario: Backfill profile recency - 2 a profile with no qualifying post gets no recency record
|
||||
When the profile recency backfill utility is run
|
||||
Then the profileRecency store has no record for bitcoincash:qaddr-nopost
|
||||
|
||||
Scenario Outline: Backfill profile recency - 3 backfill is idempotent
|
||||
When the profile recency backfill utility is run
|
||||
And the profile recency backfill utility is run again
|
||||
Then the profileRecency store records <addr> at block height <height> seen at <seen>
|
||||
|
||||
Examples:
|
||||
| addr | height | seen |
|
||||
| bitcoincash:qaddr-alice | 600100 | 100 |
|
||||
| bitcoincash:qaddr-bob | 600400 | 200 |
|
||||
@@ -11,14 +11,20 @@
|
||||
# setProfilePic, 0x6d0a) are keyed by address, so the route joins each record
|
||||
# into its profile by address. A profile with no name record reports a null
|
||||
# display name, and a profile with no picture record reports a null avatar URL.
|
||||
# The join does not change the profile order (block height descending) or the
|
||||
# pagination metadata.
|
||||
# The join does not change the profile order or the pagination metadata.
|
||||
#
|
||||
# Fixture "profiles-with-identities" (profiles, names, and profilePics stores):
|
||||
# Fixture "profiles-with-identities" (profiles, names, profilePics, and
|
||||
# profileRecency stores). The profileRecency entries make each profile
|
||||
# eligible for the recent list; their values are irrelevant to this identity
|
||||
# join:
|
||||
# profiles:
|
||||
# bitcoincash:qaddr-alice { text: alice bio, txid: profile-alice, blockHeight: 600300, seen: 3 }
|
||||
# bitcoincash:qaddr-bob { text: bob bio, txid: profile-bob, blockHeight: 600200, seen: 2 }
|
||||
# bitcoincash:qaddr-carol { text: carol bio, txid: profile-carol, blockHeight: 600100, seen: 1 }
|
||||
# profileRecency:
|
||||
# bitcoincash:qaddr-alice at 600300 seen 3
|
||||
# bitcoincash:qaddr-bob at 600200 seen 2
|
||||
# bitcoincash:qaddr-carol at 600100 seen 1
|
||||
# names:
|
||||
# bitcoincash:qaddr-alice { name: alice, txid: name-alice, blockHeight: 600250 }
|
||||
# bitcoincash:qaddr-carol { name: carol, txid: name-carol, blockHeight: 600050 }
|
||||
@@ -28,8 +34,8 @@
|
||||
Feature: Recent Profile Identity
|
||||
|
||||
Background:
|
||||
Given a psf-memo-db instance with profiles, names, and profilePics stores
|
||||
Given the fixture "profiles-with-identities" is loaded into the profiles, names, and profilePics stores
|
||||
Given a psf-memo-db instance with profiles, names, profilePics, and profileRecency stores
|
||||
Given the fixture "profiles-with-identities" is loaded into the profiles, names, profilePics, and profileRecency stores
|
||||
|
||||
Scenario Outline: Recent Profile Identity - 1 GET /profile/recent returns each profile's display name and avatar
|
||||
When the client requests /profile/recent
|
||||
|
||||
@@ -0,0 +1,65 @@
|
||||
# Scenarios: Recent Profile Ordering - 1, Recent Profile Ordering - 2, Recent Profile Ordering - 3, Recent Profile Ordering - 4
|
||||
#
|
||||
# GET /profile/recent lists one row per profile address that has at least one
|
||||
# confirmed qualifying post. Rows are ordered by the most recent qualifying
|
||||
# post's block height descending, then that post's seen descending, then the
|
||||
# address ascending. Each row reports the profile's text, provenance txid,
|
||||
# display name, and avatar; its Block and Seen columns report the most recent
|
||||
# qualifying post's blockHeight and seen, not the set-profile transaction's.
|
||||
# A profile with no qualifying post is omitted and does not count toward
|
||||
# pagination.total.
|
||||
#
|
||||
# The page is served from the profileRecency index plus per-page profile
|
||||
# lookups, so it never scans the addrPostHeights store and never sorts every
|
||||
# profile to find the page.
|
||||
#
|
||||
# Fixture "profiles-with-post-recency":
|
||||
# profiles store (the set-profile records; their own blockHeight/seen differ
|
||||
# from the recency values so the two cannot be confused):
|
||||
# bitcoincash:qaddr-alice { text: alice bio, txid: profile-alice, blockHeight: 600010, seen: 10 }
|
||||
# bitcoincash:qaddr-bob { text: bob bio, txid: profile-bob, blockHeight: 600020, seen: 20 }
|
||||
# bitcoincash:qaddr-erin { text: erin bio, txid: profile-erin, blockHeight: 600030, seen: 30 }
|
||||
# bitcoincash:qaddr-carol { text: carol bio, txid: profile-carol, blockHeight: 600040, seen: 40 }
|
||||
# bitcoincash:qaddr-dave { text: dave bio, txid: profile-dave, blockHeight: 600050, seen: 50 }
|
||||
# profileRecency store (the newest qualifying post per profile):
|
||||
# bitcoincash:qaddr-alice at 600300 seen 300
|
||||
# bitcoincash:qaddr-bob at 600300 seen 200
|
||||
# bitcoincash:qaddr-erin at 600300 seen 200
|
||||
# bitcoincash:qaddr-carol at 600200 seen 400
|
||||
# (bitcoincash:qaddr-dave has never posted, so it has no recency record)
|
||||
Feature: Recent Profile Ordering
|
||||
|
||||
Background:
|
||||
Given a psf-memo-db instance with profiles and profileRecency stores
|
||||
Given the fixture "profiles-with-post-recency" is loaded into the profiles and profileRecency stores
|
||||
|
||||
Scenario Outline: Recent Profile Ordering - 1 GET /profile/recent orders profiles by their most recent post
|
||||
When the client requests /profile/recent with limit <limit> and offset <offset>
|
||||
Then the response lists profiles in order (<expected_addrs>)
|
||||
And the response pagination shows total <total> and hasMore <hasMore>
|
||||
|
||||
Examples:
|
||||
| limit | offset | expected_addrs | total | hasMore |
|
||||
| 2 | 0 | bitcoincash:qaddr-alice,bitcoincash:qaddr-bob | 4 | true |
|
||||
| 2 | 2 | bitcoincash:qaddr-erin,bitcoincash:qaddr-carol | 4 | false |
|
||||
| 1 | 1 | bitcoincash:qaddr-bob | 4 | true |
|
||||
| 5 | 0 | bitcoincash:qaddr-alice,bitcoincash:qaddr-bob,bitcoincash:qaddr-erin,bitcoincash:qaddr-carol | 4 | false |
|
||||
|
||||
Scenario Outline: Recent Profile Ordering - 2 each row reports the most recent post's block and seen
|
||||
When the client requests /profile/recent with limit 5 and offset 0
|
||||
Then the response profile for <addr> has block height <height> and seen <seen>
|
||||
|
||||
Examples:
|
||||
| addr | height | seen |
|
||||
| bitcoincash:qaddr-alice | 600300 | 300 |
|
||||
| bitcoincash:qaddr-bob | 600300 | 200 |
|
||||
| bitcoincash:qaddr-carol | 600200 | 400 |
|
||||
|
||||
Scenario: Recent Profile Ordering - 3 a profile with no qualifying post is omitted
|
||||
When the client requests /profile/recent with limit 5 and offset 0
|
||||
Then the response lists profiles in order (bitcoincash:qaddr-alice,bitcoincash:qaddr-bob,bitcoincash:qaddr-erin,bitcoincash:qaddr-carol)
|
||||
And the response does not list bitcoincash:qaddr-dave
|
||||
|
||||
Scenario: Recent Profile Ordering - 4 the read does not iterate addrPostHeights
|
||||
When the client requests /profile/recent with limit 5 and offset 0
|
||||
Then the addrPostHeights store was not iterated
|
||||
@@ -0,0 +1,128 @@
|
||||
# Scenarios: Profile Recency Indexing - 1, Profile Recency Indexing - 2, Profile Recency Indexing - 3, Profile Recency Indexing - 4, Profile Recency Indexing - 5, Profile Recency Indexing - 6, Profile Recency Indexing - 7, Profile Recency Indexing - 8, Profile Recency Indexing - 9, Profile Recency Indexing - 10
|
||||
#
|
||||
# The indexer maintains a profileRecency store so the read side can list
|
||||
# profiles by their author's most recent post without scanning every post or
|
||||
# sorting every profile. profileRecency holds one record per profile address
|
||||
# that has at least one qualifying post, carrying that post's blockHeight and
|
||||
# seen.
|
||||
#
|
||||
# A qualifying post is a top-level post (0x6d02) or a topic message (0x6d0c).
|
||||
# Replies (0x6d03) and poll creations (0x6d10) do not qualify. Recency is only
|
||||
# written for addresses that have a set-profile record (0x6d05), so an address
|
||||
# that has posted but never set a profile is absent.
|
||||
#
|
||||
# Recency reflects confirmed blocks only: a post first seen in the mempool does
|
||||
# not create or change a profileRecency record, and the record is written when
|
||||
# the block containing that post is indexed. Processing qualifying posts in any
|
||||
# order converges on the greatest height, with seen as the tie-breaker at equal
|
||||
# heights, so a replay never regresses a profile's recency.
|
||||
Feature: Profile Recency Indexing
|
||||
|
||||
Background:
|
||||
Given a psf-memo-db instance with profiles and profileRecency stores
|
||||
Given a psf-memo-indexer configured to write to that database
|
||||
|
||||
Scenario Outline: Profile Recency Indexing - 1 a confirmed top-level post records the author's recency
|
||||
Given the psf-memo-db stores a profile for <addr>
|
||||
When the indexer processes a Memo post transaction <txid> from <addr> at block height <height> with text "<text>" seen at <seen>
|
||||
Then the profileRecency store records <addr> at block height <height> seen at <seen>
|
||||
|
||||
Examples:
|
||||
| addr | txid | height | text | seen |
|
||||
| bitcoincash:qaddr-a | post-a1 | 600100 | hello | 1000 |
|
||||
| bitcoincash:qaddr-b | post-b1 | 600250 | world | 2000 |
|
||||
|
||||
Scenario Outline: Profile Recency Indexing - 2 a confirmed topic post records the author's recency
|
||||
Given the psf-memo-db stores a profile for <addr>
|
||||
When the indexer processes a Memo topic message <txid> in room "<room>" from <addr> at block height <height> with text "<text>" seen at <seen>
|
||||
Then the profileRecency store records <addr> at block height <height> seen at <seen>
|
||||
|
||||
Examples:
|
||||
| addr | txid | room | height | text | seen |
|
||||
| bitcoincash:qaddr-a | topic-a1 | bitcoin | 600300 | gm | 3000 |
|
||||
| bitcoincash:qaddr-b | topic-b1 | cash | 600150 | hi | 1500 |
|
||||
|
||||
Scenario Outline: Profile Recency Indexing - 3 the greatest height wins regardless of processing order
|
||||
Given the psf-memo-db stores a profile for <addr>
|
||||
When the indexer processes a Memo post transaction <firstTxid> from <addr> at block height <firstHeight> with text "first" seen at <firstSeen>
|
||||
And the indexer processes a Memo post transaction <secondTxid> from <addr> at block height <secondHeight> with text "second" seen at <secondSeen>
|
||||
Then the profileRecency store records <addr> at block height <height> seen at <seen>
|
||||
|
||||
Examples:
|
||||
| addr | firstTxid | firstHeight | firstSeen | secondTxid | secondHeight | secondSeen | height | seen |
|
||||
| bitcoincash:qaddr-a | post-a1 | 600100 | 100 | post-a2 | 600200 | 200 | 600200 | 200 |
|
||||
| bitcoincash:qaddr-b | post-b1 | 600200 | 100 | post-b2 | 600100 | 200 | 600200 | 100 |
|
||||
|
||||
Scenario Outline: Profile Recency Indexing - 4 a reply does not change the recency
|
||||
Given the psf-memo-db stores a profile for <addr>
|
||||
When the indexer processes a Memo post transaction <postTxid> from <addr> at block height <postHeight> with text "root" seen at 100
|
||||
And the indexer processes a Memo reply transaction <replyTxid> to parent <postTxid> from <addr> at block height <replyHeight> with text "reply"
|
||||
Then the profileRecency store records <addr> at block height <postHeight> seen at 100
|
||||
|
||||
Examples:
|
||||
| addr | postTxid | postHeight | replyTxid | replyHeight |
|
||||
| bitcoincash:qaddr-a | post-a1 | 600100 | reply-a1 | 600200 |
|
||||
| bitcoincash:qaddr-b | post-b1 | 600300 | reply-b1 | 600400 |
|
||||
|
||||
Scenario Outline: Profile Recency Indexing - 5 a poll creation does not change the recency
|
||||
Given the psf-memo-db stores a profile for bitcoincash:qaddr-a
|
||||
When the indexer processes a Memo post transaction <postTxid> from bitcoincash:qaddr-a at block height <postHeight> with text "root" seen at 100
|
||||
And the indexer processes a create-poll transaction with the question "<question>" and 2 options
|
||||
Then the profileRecency store records bitcoincash:qaddr-a at block height <postHeight> seen at 100
|
||||
|
||||
Examples:
|
||||
| postTxid | postHeight | question |
|
||||
| post-a1 | 600050 | best coin? |
|
||||
| post-a2 | 600070 | lunch? |
|
||||
|
||||
Scenario Outline: Profile Recency Indexing - 6 an author without a profile gets no recency record
|
||||
When the indexer processes a Memo post transaction <txid> from <addr> at block height <height> with text "hi" seen at 100
|
||||
Then the profileRecency store has no record for <addr>
|
||||
|
||||
Examples:
|
||||
| addr | txid | height |
|
||||
| bitcoincash:qaddr-noprofile | post-n1 | 600100 |
|
||||
| bitcoincash:qaddr-other | post-o1 | 600200 |
|
||||
|
||||
Scenario Outline: Profile Recency Indexing - 7 setting a profile after posting establishes the recency
|
||||
When the indexer processes a Memo post transaction <postTxid> from <addr> at block height <postHeight> with text "old post" seen at 100
|
||||
And the indexer processes a Memo reply transaction <replyTxid> to parent <postTxid> from <addr> at block height <replyHeight> with text "newer reply"
|
||||
And the indexer processes a set-profile transaction for <addr> with text "my bio"
|
||||
Then the profileRecency store records <addr> at block height <postHeight> seen at 100
|
||||
|
||||
Examples:
|
||||
| addr | postTxid | postHeight | replyTxid | replyHeight |
|
||||
| bitcoincash:qaddr-a | post-a1 | 600100 | reply-a1 | 600300 |
|
||||
| bitcoincash:qaddr-b | post-b1 | 600200 | reply-b1 | 600250 |
|
||||
|
||||
Scenario Outline: Profile Recency Indexing - 8 setting a profile with no qualifying post creates no recency record
|
||||
When the indexer processes a set-profile transaction for <addr> with text "my bio"
|
||||
Then the profileRecency store has no record for <addr>
|
||||
|
||||
Examples:
|
||||
| addr |
|
||||
| bitcoincash:qaddr-a |
|
||||
| bitcoincash:qaddr-b |
|
||||
|
||||
Scenario Outline: Profile Recency Indexing - 9 an unconfirmed post does not affect recency until its block is indexed
|
||||
Given the psf-memo-db stores a profile for <addr>
|
||||
When the transaction indexer sees a Memo post transaction <txid> from <addr> at block height <height> with text "mempool"
|
||||
Then the profileRecency store has no record for <addr>
|
||||
When the indexer processes a Memo post transaction <txid> from <addr> at block height <height> with text "mempool" seen at <seen>
|
||||
Then the profileRecency store records <addr> at block height <height> seen at <seen>
|
||||
|
||||
Examples:
|
||||
| addr | txid | height | seen |
|
||||
| bitcoincash:qaddr-a | post-a1 | 600100 | 1000 |
|
||||
| bitcoincash:qaddr-b | post-b1 | 600400 | 4000 |
|
||||
|
||||
Scenario Outline: Profile Recency Indexing - 10 reprocessing a qualifying post is idempotent
|
||||
Given the psf-memo-db stores a profile for <addr>
|
||||
When the indexer processes a Memo post transaction <txid> from <addr> at block height <height> with text "hi" seen at 100
|
||||
And the indexer processes the same Memo post transaction <txid> again
|
||||
Then the profileRecency store records <addr> at block height <height> seen at 100
|
||||
|
||||
Examples:
|
||||
| addr | txid | height |
|
||||
| bitcoincash:qaddr-a | post-a1 | 600100 |
|
||||
| bitcoincash:qaddr-b | post-b1 | 600200 |
|
||||
@@ -31,7 +31,21 @@ focus is **front-end improvements** to `psf-memo-client` (the React SPA).
|
||||
|
||||
## In progress
|
||||
|
||||
- None.
|
||||
- **Profile ordering by last post (2026-09-20):** `/profile/recent` should
|
||||
list only profiles that have posted, ordered by their most recent qualifying
|
||||
post instead of their most recent profile update. A qualifying post is a
|
||||
top-level post (`0x6d02`) or a topic message (`0x6d0c`); replies (`0x6d03`)
|
||||
and poll creations (`0x6d10`) do not qualify. Profiles that have never posted
|
||||
are dropped. Ordering is by last post's block height descending, then seen
|
||||
descending, then address ascending, considering confirmed blocks only. The
|
||||
Block and Seen columns report the last post's block height and timestamp
|
||||
rather than the set-profile transaction's. The indexer maintains a
|
||||
`profileRecency` index (mirroring `topicRecency`) so the read never scans
|
||||
`addrPostHeights` or sorts every profile; a backfill builds it from existing
|
||||
data. Indexer + DB (the client renders the returned block/seen unchanged).
|
||||
Specs: `psf-memo-indexer/specs/profile-recency-indexing.feature`,
|
||||
`psf-memo-db/specs/recent-profile-ordering.feature`,
|
||||
`psf-memo-db/specs/backfill-profile-recency.feature`.
|
||||
|
||||
## Recently completed
|
||||
|
||||
|
||||
Reference in New Issue
Block a user