mirror of
https://github.com/Permissionless-Software-Foundation/psf-memo.git
synced 2026-09-21 16:52:01 -07:00
Spec topic recency ordering and pagination
Add Gherkin for efficient GET /topics ordering by most recent post, the topicSummaries and topicRecency index stores, an idempotent backfill, and client topics-page pagination. By specifier.
This commit is contained in:
Chris Troutner
@@ -0,0 +1,27 @@
|
||||
# Scenarios: Topic Pagination - 1, Topic Pagination - 2
|
||||
#
|
||||
# The topics page loads topics in pages of 50 and can move to a later page. The
|
||||
# page reports whether more topics are available.
|
||||
Feature: Topic Pagination
|
||||
|
||||
Scenario Outline: Topic Pagination - 1 the topics page loads 50 topics per page
|
||||
Given the psf-memo-db API serves <count> topics
|
||||
When I open the topics page
|
||||
Then the topics page shows <shown> topics
|
||||
And the topics page can load more topics
|
||||
|
||||
Examples:
|
||||
| count | shown |
|
||||
| 60 | 50 |
|
||||
| 70 | 50 |
|
||||
|
||||
Scenario Outline: Topic Pagination - 2 the topics page can load a later page
|
||||
Given the psf-memo-db API serves <count> topics
|
||||
When I open the topics page at offset <offset>
|
||||
Then the topics page shows <shown> topics
|
||||
And the topics page has no more topics
|
||||
|
||||
Examples:
|
||||
| count | offset | shown |
|
||||
| 60 | 50 | 10 |
|
||||
| 70 | 50 | 20 |
|
||||
@@ -0,0 +1,51 @@
|
||||
# Scenarios: Backfill topic indexes - 1, Backfill topic indexes - 2, Backfill topic indexes - 3
|
||||
#
|
||||
# The topic backfill utility builds topicSummaries and topicRecency from the
|
||||
# existing rooms store so the read side can serve /topics by recency without
|
||||
# scanning posts. Rooms with posts get postCount and lastHeight; follow-only
|
||||
# rooms get postCount 0 and lastHeight 0. Running the backfill twice is
|
||||
# idempotent.
|
||||
#
|
||||
# Fixture "rooms-with-topics-and-follows":
|
||||
# rooms store:
|
||||
# bitcoin:post-100 { room: bitcoin, txid: post-100, type: post, blockHeight: 600100 }
|
||||
# bitcoin:post-200 { room: bitcoin, txid: post-200, type: post, blockHeight: 600200 }
|
||||
# bitcoin:addr-f { room: bitcoin, addr: addr-f, type: follow, unfollow: false }
|
||||
# cash:post-250 { room: cash, txid: post-250, type: post, blockHeight: 600250 }
|
||||
# lone:addr-f { room: lone, addr: addr-f, type: follow, unfollow: false }
|
||||
Feature: Backfill topic indexes
|
||||
|
||||
Background:
|
||||
Given a psf-memo-db instance with rooms, topicSummaries, and topicRecency stores
|
||||
Given the fixture "rooms-with-topics-and-follows" is loaded into the rooms store
|
||||
|
||||
Scenario Outline: Backfill topic indexes - 1 backfill summarizes every room
|
||||
When the topic backfill utility is run
|
||||
Then the topicSummaries store contains the room "<room>" with postCount <postCount> and lastHeight <height>
|
||||
|
||||
Examples:
|
||||
| room | postCount | height |
|
||||
| bitcoin | 2 | 600200 |
|
||||
| cash | 1 | 600250 |
|
||||
| lone | 0 | 0 |
|
||||
|
||||
Scenario Outline: Backfill topic indexes - 2 backfill builds the recency index
|
||||
When the topic backfill utility is run
|
||||
Then the topicRecency store records the room "<room>" at block height <height>
|
||||
|
||||
Examples:
|
||||
| room | height |
|
||||
| bitcoin | 600200 |
|
||||
| cash | 600250 |
|
||||
| lone | 0 |
|
||||
|
||||
Scenario Outline: Backfill topic indexes - 3 backfill is idempotent
|
||||
When the topic backfill utility is run
|
||||
And the topic backfill utility is run again
|
||||
Then the topicSummaries store contains the room "<room>" with postCount <postCount> and lastHeight <height>
|
||||
And the topicRecency store records the room "<room>" at block height <height>
|
||||
|
||||
Examples:
|
||||
| room | postCount | height |
|
||||
| bitcoin | 2 | 600200 |
|
||||
| cash | 1 | 600250 |
|
||||
@@ -0,0 +1,60 @@
|
||||
# Scenarios: Topic Pagination - 1, Topic Pagination - 2, Topic Pagination - 3
|
||||
#
|
||||
# GET /topics returns a page of topics ordered by their most recent post's
|
||||
# block height descending, with rooms at the same height ordered by room name
|
||||
# ascending; rooms with no posts (follow-only) come last, also by room name
|
||||
# ascending. Pagination metadata reports the total distinct topics and whether
|
||||
# more remain. The page is served from topicSummaries and topicRecency without
|
||||
# iterating the rooms store.
|
||||
#
|
||||
# Fixture "topic-indexes":
|
||||
# topicSummaries store:
|
||||
# memo { room: memo, postCount: 5, lastHeight: 600500 }
|
||||
# cash { room: cash, postCount: 2, lastHeight: 600400 }
|
||||
# dance { room: dance, postCount: 3, lastHeight: 600400 }
|
||||
# anime { room: anime, postCount: 1, lastHeight: 600300 }
|
||||
# lone { room: lone, postCount: 0, lastHeight: 0 }
|
||||
# quiet { room: quiet, postCount: 0, lastHeight: 0 }
|
||||
# topicRecency store:
|
||||
# memo at 600500, cash at 600400, dance at 600400,
|
||||
# anime at 600300, lone at 0, quiet at 0
|
||||
Feature: Topic Pagination
|
||||
|
||||
Background:
|
||||
Given a psf-memo-db instance with topicSummaries and topicRecency stores
|
||||
Given the fixture "topic-indexes" is loaded into the topic index stores
|
||||
|
||||
Scenario Outline: Topic Pagination - 1 GET /topics returns topics ordered by most recent post
|
||||
When the client requests /topics with limit <limit> and offset <offset>
|
||||
Then the response lists topics in order <expected_topics>
|
||||
And the response pagination shows total <total> and hasMore <hasMore>
|
||||
|
||||
Examples:
|
||||
| limit | offset | expected_topics | total | hasMore |
|
||||
| 2 | 0 | memo,cash | 6 | true |
|
||||
| 2 | 2 | dance,anime | 6 | true |
|
||||
| 2 | 4 | lone,quiet | 6 | false |
|
||||
| 6 | 0 | memo,cash,dance,anime,lone,quiet | 6 | false |
|
||||
|
||||
Scenario Outline: Topic Pagination - 2 GET /topics reports each topic's post count
|
||||
When the client requests /topics with limit <limit> and offset <offset>
|
||||
Then the response contains the topic "<topic>" with post count <postCount>
|
||||
|
||||
Examples:
|
||||
| limit | offset | topic | postCount |
|
||||
| 3 | 0 | memo | 5 |
|
||||
| 3 | 0 | cash | 2 |
|
||||
| 3 | 3 | anime | 1 |
|
||||
| 3 | 3 | lone | 0 |
|
||||
|
||||
Scenario Outline: Topic Pagination - 3 GET /topics reads the recency index without scanning rooms
|
||||
When the client requests /topics with limit <limit> and offset <offset>
|
||||
Then the topicRecency store was read exactly <read_count> records
|
||||
And the rooms store was not iterated
|
||||
|
||||
Examples:
|
||||
| limit | offset | read_count |
|
||||
| 2 | 0 | 2 |
|
||||
| 2 | 2 | 4 |
|
||||
| 2 | 4 | 6 |
|
||||
| 6 | 0 | 6 |
|
||||
@@ -22,11 +22,18 @@
|
||||
# post-200 { txid: post-200, addr: addr-b, text: bitcoin again, blockHeight: 200 }
|
||||
# post-250 { txid: post-250, addr: addr-a, text: cash rules, blockHeight: 250 }
|
||||
# post-400 { txid: post-400, addr: addr-c, text: dev stuff, blockHeight: 400 }
|
||||
# topicSummaries store (derived from rooms):
|
||||
# bitcoin { room: bitcoin, postCount: 2, lastHeight: 300 }
|
||||
# cash { room: cash, postCount: 1, lastHeight: 250 }
|
||||
# dev { room: dev, postCount: 1, lastHeight: 400 }
|
||||
# lone { room: lone, postCount: 0, lastHeight: 0 }
|
||||
# topicRecency store (derived from rooms):
|
||||
# bitcoin at 300, cash at 250, dev at 400, lone at 0
|
||||
Feature: Topic Read
|
||||
|
||||
Background:
|
||||
Given a psf-memo-db instance with a rooms store and a posts store
|
||||
Given the fixture "topics-with-posts" is loaded into the rooms and posts stores
|
||||
Given a psf-memo-db instance with rooms, posts, topicSummaries, and topicRecency stores
|
||||
Given the fixture "topics-with-posts" is loaded into the rooms, posts, topicSummaries, and topicRecency stores
|
||||
|
||||
Scenario Outline: Topic Read - 1 GET /topics lists distinct topics with their post counts
|
||||
When the client requests /topics
|
||||
|
||||
@@ -0,0 +1,65 @@
|
||||
# Scenarios: Topic Recency Indexing - 1, Topic Recency Indexing - 2, Topic Recency Indexing - 3, Topic Recency Indexing - 4, Topic Recency Indexing - 5
|
||||
#
|
||||
# The indexer maintains two topic indexes so the read side can list topics by
|
||||
# most recent post without scanning every topic post record:
|
||||
# - topicSummaries: one record per room with postCount and lastHeight.
|
||||
# - topicRecency: one record per room at that room's most recent post height;
|
||||
# a room with no posts (follow-only) is recorded at height 0.
|
||||
# Reprocessing a topic message must not double-count posts, and a follow for a
|
||||
# room that already has posts must not change the room's summary.
|
||||
Feature: Topic Recency Indexing
|
||||
|
||||
Background:
|
||||
Given a psf-memo-db instance with rooms, topicSummaries, and topicRecency stores
|
||||
Given a psf-memo-indexer configured to write to that database
|
||||
|
||||
Scenario Outline: Topic Recency Indexing - 1 a topic message records a room summary and a recency record
|
||||
When the indexer processes a Memo topic message <txid> in room "<room>" from <addr> at block height <height> with text "<text>"
|
||||
Then the topicSummaries store contains the room "<room>" with postCount 1 and lastHeight <height>
|
||||
And the topicRecency store records the room "<room>" at block height <height>
|
||||
|
||||
Examples:
|
||||
| txid | room | addr | height | text |
|
||||
| topic-a1 | bitcoin | bitcoincash:qaddr-a | 600100 | hello |
|
||||
| topic-b1 | cash | bitcoincash:qaddr-b | 600200 | cash chat |
|
||||
|
||||
Scenario Outline: Topic Recency Indexing - 2 successive topic messages accumulate postCount and keep the newest height
|
||||
When the indexer processes a Memo topic message <firstTxid> in room "<room>" from <addr> at block height <firstHeight> with text "<firstText>"
|
||||
And the indexer processes a Memo topic message <secondTxid> in room "<room>" from <addr> at block height <secondHeight> with text "<secondText>"
|
||||
Then the topicSummaries store contains the room "<room>" with postCount 2 and lastHeight <height>
|
||||
And the topicRecency store records the room "<room>" at block height <height>
|
||||
|
||||
Examples:
|
||||
| firstTxid | secondTxid | room | addr | firstHeight | secondHeight | firstText | secondText | height |
|
||||
| topic-a1 | topic-a2 | bitcoin | bitcoincash:qaddr-a | 600100 | 600200 | hello | again | 600200 |
|
||||
| topic-a3 | topic-a4 | bitcoin | bitcoincash:qaddr-a | 600200 | 600100 | later | earlier | 600200 |
|
||||
|
||||
Scenario Outline: Topic Recency Indexing - 3 reprocessing a topic message is idempotent
|
||||
When the indexer processes a Memo topic message <txid> in room "<room>" from <addr> at block height <height> with text "<text>"
|
||||
And the indexer processes the same Memo topic message <txid> again
|
||||
Then the topicSummaries store contains the room "<room>" with postCount 1 and lastHeight <height>
|
||||
And the topicRecency store records the room "<room>" at block height <height>
|
||||
|
||||
Examples:
|
||||
| txid | room | addr | height | text |
|
||||
| topic-c1 | bitcoin | bitcoincash:qaddr-c | 600300 | repeated |
|
||||
|
||||
Scenario Outline: Topic Recency Indexing - 4 a topic follow with no posts records a zero-post room
|
||||
When the indexer processes a Memo topic follow for room "<room>" from <addr>
|
||||
Then the topicSummaries store contains the room "<room>" with postCount 0 and lastHeight 0
|
||||
And the topicRecency store records the room "<room>" at block height 0
|
||||
|
||||
Examples:
|
||||
| room | addr |
|
||||
| lone | bitcoincash:qaddr-a |
|
||||
| dev | bitcoincash:qaddr-b |
|
||||
|
||||
Scenario Outline: Topic Recency Indexing - 5 a topic follow for a room with posts leaves its summary unchanged
|
||||
When the indexer processes a Memo topic message <txid> in room "<room>" from <addr> at block height <height> with text "<text>"
|
||||
And the indexer processes a Memo topic follow for room "<room>" from <addr>
|
||||
Then the topicSummaries store contains the room "<room>" with postCount 1 and lastHeight <height>
|
||||
And the topicRecency store records the room "<room>" at block height <height>
|
||||
|
||||
Examples:
|
||||
| txid | room | addr | height | text |
|
||||
| topic-d1 | bitcoin | bitcoincash:qaddr-a | 600400 | followed |
|
||||
@@ -29,6 +29,22 @@ focus is **front-end improvements** to `psf-memo-client` (the React SPA).
|
||||
- A single user-facing feature may still touch more than one component; call
|
||||
out all affected components in the task description and in the handoff.
|
||||
|
||||
## In progress
|
||||
|
||||
- **Topic recency ordering and pagination (task `topic-recency-pagination`):**
|
||||
make `GET /topics` order by most recent topic post without scanning the whole
|
||||
`rooms` store, and paginate it. The indexer maintains two new stores:
|
||||
`topicSummaries` (one record per room with `postCount` and `lastHeight`) and
|
||||
`topicRecency` (one record per room at its latest post height; follow-only
|
||||
rooms at height 0). `GET /topics` gains `limit`/`offset` and returns
|
||||
`pagination`; the client topics page loads 50 per page with Previous/Next.
|
||||
A topic backfill utility builds both indexes from the existing `rooms` store.
|
||||
Client + indexer + DB. Specs:
|
||||
`psf-memo-indexer/specs/topic-recency-indexing.feature`,
|
||||
`psf-memo-db/specs/backfill-topic-indexes.feature`,
|
||||
`psf-memo-db/specs/topic-pagination.feature`,
|
||||
`psf-memo-client/specs/topic-pagination.feature`.
|
||||
|
||||
## Recently completed
|
||||
|
||||
- **Memo multi-push encoding (2026-09-17):** fixed the payload layout for
|
||||
|
||||
Reference in New Issue
Block a user