From 69c38d5a366467ee9de170df81ef65271ae642b0 Mon Sep 17 00:00:00 2001 From: Chris Troutner Date: Wed, 26 Aug 2026 12:05:01 -0700 Subject: [PATCH] Updating Docker files for mono repo --- .dockerignore | 22 +++++++++++++ README.md | 16 +++++++++ psf-memo-db/README.md | 2 +- psf-memo-indexer/README.md | 6 ++-- psf-memo-indexer/dev-docs/architecture.md | 9 ++--- .../design-decisions-and-tradeoffs.md | 4 +-- psf-memo-indexer/dev-docs/psf-memo-db.md | 2 +- .../docker/block-indexer/Dockerfile | 25 ++++++-------- .../production/docker/docker-compose.yml | 24 +++++++++++--- .../production/docker/memo-client/Dockerfile | 33 +++++++++---------- .../production/docker/memo-db/Dockerfile | 25 ++++++-------- .../production/docker/tx-indexer/Dockerfile | 25 ++++++-------- 12 files changed, 114 insertions(+), 79 deletions(-) create mode 100644 .dockerignore diff --git a/.dockerignore b/.dockerignore new file mode 100644 index 0000000..177579a --- /dev/null +++ b/.dockerignore @@ -0,0 +1,22 @@ +# Keep Docker build contexts lean when building from the monorepo root. +.git +.gitsigners +.worktrees +swarmforge +docs +doc +test +tools +specs +**/node_modules +**/coverage +**/leveldb +**/logs +**/tmp +**/target +**/.env +!psf-memo-indexer/production/docker/**/.env +psf-memo-client/build +psf-memo-indexer/production/data +*.md +!README.md diff --git a/README.md b/README.md index 638ec13..809678a 100644 --- a/README.md +++ b/README.md @@ -40,6 +40,22 @@ npm start See each component's `README.md` and `dev-docs/` for architecture details. +### Production (Docker) + +Compose under [`psf-memo-indexer/production/docker`](psf-memo-indexer/production/docker/) builds all four services from this monorepo (no separate GitHub clones): + +```bash +cd psf-memo-indexer/production/docker +cp memo-db/.env-example memo-db/.env +cp block-indexer/.env-example block-indexer/.env +cp tx-indexer/.env-example tx-indexer/.env +cp memo-client/.env-example memo-client/.env +docker compose build +docker compose up -d +``` + +Full ops notes: [`psf-memo-indexer/README.md`](psf-memo-indexer/README.md#production-docker). + ## SwarmForge development This project is configured for SwarmForge with four pi-backed agents: diff --git a/psf-memo-db/README.md b/psf-memo-db/README.md index c2a22d5..368ba53 100644 --- a/psf-memo-db/README.md +++ b/psf-memo-db/README.md @@ -61,7 +61,7 @@ npm test ## Production (Docker) -The production Docker setup is in [psf-memo-indexer/production/docker](../psf-memo-indexer/production/docker/memo-db/) (compose builds `memo-db` from this repo). +The production Docker setup is in [psf-memo-indexer/production/docker](../psf-memo-indexer/production/docker/). Compose builds `memo-db` by copying this package from the monorepo root. ## License diff --git a/psf-memo-indexer/README.md b/psf-memo-indexer/README.md index 46662cd..e72bf0c 100644 --- a/psf-memo-indexer/README.md +++ b/psf-memo-indexer/README.md @@ -73,16 +73,16 @@ npm test ## Production (Docker) -Docker Compose under [production/docker](./production/docker) runs the full stack. Images clone their source from GitHub at build time (same pattern as [psf-slp-indexer-g2](https://github.com/Permissionless-Software-Foundation/psf-slp-indexer-g2)). +Docker Compose under [production/docker](./production/docker) runs the full stack. Build context is the [psf-memo](https://github.com/Permissionless-Software-Foundation/psf-memo) monorepo root; each image copies its package (`psf-memo-db`, `psf-memo-indexer`, or `psf-memo-client`) from the local checkout. ### Services | Service | Container | Host port | Purpose | |---------|-----------|-----------|---------| -| `memo-db` | `memo-db` | `5021` | LevelDB REST API ([psf-memo-db](https://github.com/Permissionless-Software-Foundation/psf-memo-db)) | +| `memo-db` | `memo-db` | `5021` | LevelDB REST API (`psf-memo-db/`) | | `block-indexer` | `memo-block-indexer` | — | IBD + ZMQ block indexing | | `tx-indexer` | `memo-tx-indexer` | `5455` | Mempool TX indexing (`/tx-start` control API) | -| `memo-client` | `memo-client` | `3000` | React SPA ([psf-memo-client](https://github.com/Permissionless-Software-Foundation/psf-memo-client)), nginx | +| `memo-client` | `memo-client` | `3000` | React SPA (`psf-memo-client/`), nginx | LevelDB data persists on the host at `production/data/leveldb`. diff --git a/psf-memo-indexer/dev-docs/architecture.md b/psf-memo-indexer/dev-docs/architecture.md index 42b138f..204cda0 100644 --- a/psf-memo-indexer/dev-docs/architecture.md +++ b/psf-memo-indexer/dev-docs/architecture.md @@ -168,10 +168,11 @@ Each store is a separate LevelDB with JSON values. Keys are txids, addresses, or ```text production/docker/ -├── docker-compose.yml -├── memo-db/ → build context: ../../psf-memo-db -├── block-indexer/ → build context: indexer repo root -└── tx-indexer/ +├── docker-compose.yml → build context: monorepo root (../../..) +├── memo-db/ → COPY psf-memo-db/ +├── block-indexer/ → COPY psf-memo-indexer/ +├── tx-indexer/ → COPY psf-memo-indexer/ +└── memo-client/ → COPY psf-memo-client/ ``` Volumes mount `.env` and start scripts per service, matching the SLP production pattern. LevelDB data can persist under `production/data/leveldb`. diff --git a/psf-memo-indexer/dev-docs/design-decisions-and-tradeoffs.md b/psf-memo-indexer/dev-docs/design-decisions-and-tradeoffs.md index 67348c2..2311c25 100644 --- a/psf-memo-indexer/dev-docs/design-decisions-and-tradeoffs.md +++ b/psf-memo-indexer/dev-docs/design-decisions-and-tradeoffs.md @@ -135,9 +135,9 @@ This document records **why** the Memo indexer stack looks the way it does, incl ## 13. Docker layout under `production/docker` -**Decision:** Dockerfiles live in `psf-memo-indexer/production/docker/{memo-db,block-indexer,tx-indexer}/`, not repo roots—matching psf-slp-indexer-g2. +**Decision:** Dockerfiles live in `psf-memo-indexer/production/docker/{memo-db,block-indexer,tx-indexer,memo-client}/`, not package roots—matching the psf-slp-indexer-g2 layout, adapted for the monorepo. -**Reason:** Single compose file orchestrates three services; build contexts point at `psf-memo-db` sibling repo and indexer root. +**Reason:** Single compose file orchestrates all services; build context is the monorepo root so each Dockerfile can `COPY` its package (`psf-memo-db`, `psf-memo-indexer`, `psf-memo-client`) from the local checkout. ## 14. JavaScript vs extending the Go indexer diff --git a/psf-memo-indexer/dev-docs/psf-memo-db.md b/psf-memo-indexer/dev-docs/psf-memo-db.md index 0727586..6b28126 100644 --- a/psf-memo-indexer/dev-docs/psf-memo-db.md +++ b/psf-memo-indexer/dev-docs/psf-memo-db.md @@ -215,4 +215,4 @@ Run: `npm test` in the `psf-memo-db` directory. ## Production -Docker build context is the `psf-memo-db` repo root; Dockerfile lives in [psf-memo-indexer/production/docker/memo-db/](../production/docker/memo-db/). See [architecture.md](./architecture.md#deployment-topology). +Docker build context is the monorepo root; the Dockerfile copies `psf-memo-db/` and lives in [psf-memo-indexer/production/docker/memo-db/](../production/docker/memo-db/). See [architecture.md](./architecture.md#deployment-topology). diff --git a/psf-memo-indexer/production/docker/block-indexer/Dockerfile b/psf-memo-indexer/production/docker/block-indexer/Dockerfile index ae9dd74..ed23ab0 100644 --- a/psf-memo-indexer/production/docker/block-indexer/Dockerfile +++ b/psf-memo-indexer/production/docker/block-indexer/Dockerfile @@ -1,18 +1,20 @@ # Create a Dockerized block indexer # +# Build context: psf-memo monorepo root +# docker compose build block-indexer -#IMAGE BUILD COMMANDS FROM ubuntu:24.04 MAINTAINER Chris Troutner #Update the OS and install any OS packages needed. -RUN apt-get update -RUN apt-get install -y sudo git curl nano gnupg wget zip unzip python3 +RUN apt-get update && apt-get install -y sudo curl nano gnupg wget zip unzip python3 \ + && rm -rf /var/lib/apt/lists/* #Install Node and NPM -RUN curl -sL https://deb.nodesource.com/setup_22.x -o nodesource_setup.sh -RUN bash nodesource_setup.sh -RUN apt-get install -y nodejs build-essential +RUN curl -sL https://deb.nodesource.com/setup_22.x -o nodesource_setup.sh \ + && bash nodesource_setup.sh \ + && apt-get update && apt-get install -y nodejs build-essential \ + && rm -rf /var/lib/apt/lists/* #Create the user 'safeuser' and add them to the sudo group. RUN useradd -ms /bin/bash safeuser @@ -30,17 +32,10 @@ RUN chown -R safeuser .npm-global RUN echo "export PATH=~/.npm-global/bin:$PATH" >> /home/safeuser/.profile RUN runuser -l safeuser -c "npm config set prefix '~/.npm-global'" -# Clone the psf-memo-indexer repository -WORKDIR /home/safeuser -RUN git clone https://github.com/Permissionless-Software-Foundation/psf-memo-indexer - -# Switch to the desired branch. `master` is usually stable, -# and `stage` has the most up-to-date changes. +# Copy psf-memo-indexer from the monorepo +COPY psf-memo-indexer /home/safeuser/psf-memo-indexer WORKDIR /home/safeuser/psf-memo-indexer -# For development: switch to unstable branch -#RUN git checkout - # Install dependencies RUN npm install diff --git a/psf-memo-indexer/production/docker/docker-compose.yml b/psf-memo-indexer/production/docker/docker-compose.yml index 104e7ad..a06cb0a 100644 --- a/psf-memo-indexer/production/docker/docker-compose.yml +++ b/psf-memo-indexer/production/docker/docker-compose.yml @@ -1,8 +1,16 @@ -# Start the service with: docker-compose up -d +# Build and start from this directory: +# docker compose build +# docker compose up -d +# +# Build context is the psf-memo monorepo root (../../../) so each +# image copies its package from the local checkout rather than cloning +# separate GitHub repositories. services: memo-db: - build: ./memo-db + build: + context: ../../.. + dockerfile: psf-memo-indexer/production/docker/memo-db/Dockerfile container_name: memo-db logging: driver: json-file @@ -18,7 +26,9 @@ services: restart: always tx-indexer: - build: ./tx-indexer + build: + context: ../../.. + dockerfile: psf-memo-indexer/production/docker/tx-indexer/Dockerfile image: memo-tx-indexer container_name: memo-tx-indexer logging: @@ -34,7 +44,9 @@ services: restart: always block-indexer: - build: ./block-indexer + build: + context: ../../.. + dockerfile: psf-memo-indexer/production/docker/block-indexer/Dockerfile image: memo-block-indexer container_name: memo-block-indexer logging: @@ -48,7 +60,9 @@ services: restart: always memo-client: - build: ./memo-client + build: + context: ../../.. + dockerfile: psf-memo-indexer/production/docker/memo-client/Dockerfile image: memo-client container_name: memo-client logging: diff --git a/psf-memo-indexer/production/docker/memo-client/Dockerfile b/psf-memo-indexer/production/docker/memo-client/Dockerfile index 80d31db..a333fb4 100644 --- a/psf-memo-indexer/production/docker/memo-client/Dockerfile +++ b/psf-memo-indexer/production/docker/memo-client/Dockerfile @@ -1,38 +1,35 @@ # Create a Dockerized front-end SPA served by nginx # +# Build context: psf-memo monorepo root +# docker compose build memo-client #IMAGE BUILD COMMANDS FROM ubuntu:24.04 as builder MAINTAINER Chris Troutner #Update the OS and install any OS packages needed. -RUN apt-get update -RUN apt-get install -y sudo git curl nano gnupg wget +RUN apt-get update && apt-get install -y sudo curl nano gnupg wget \ + && rm -rf /var/lib/apt/lists/* #Install Node and NPM -RUN curl -sL https://deb.nodesource.com/setup_22.x -o nodesource_setup.sh -RUN bash nodesource_setup.sh -RUN apt-get install -y nodejs build-essential +RUN curl -sL https://deb.nodesource.com/setup_22.x -o nodesource_setup.sh \ + && bash nodesource_setup.sh \ + && apt-get update && apt-get install -y nodejs build-essential \ + && rm -rf /var/lib/apt/lists/* #Set the working directory to be the home directory WORKDIR /home/safeuser -# Clone the psf-memo-client repository -RUN git clone https://github.com/Permissionless-Software-Foundation/psf-memo-client - -# Switch to the desired branch. `master` is usually stable, -# and `stage` / `unstable` have the most up-to-date changes. +# Copy psf-memo-client from the monorepo +COPY psf-memo-client /home/safeuser/psf-memo-client WORKDIR /home/safeuser/psf-memo-client -# For development: switch to unstable branch -#RUN git checkout unstable - -# Install dependencies -RUN npm install - # Create React App reads REACT_APP_* from this .env at build time. # Edit production/docker/memo-client/.env before building. -COPY .env .env +COPY psf-memo-indexer/production/docker/memo-client/.env .env + +# Install dependencies +RUN npm install # Build the site RUN npm run build @@ -45,4 +42,4 @@ EXPOSE 80 COPY --from=builder /home/safeuser/psf-memo-client/build /usr/share/nginx/html # Copy the NGINX configuration file. -COPY default.conf /etc/nginx/conf.d/default.conf +COPY psf-memo-indexer/production/docker/memo-client/default.conf /etc/nginx/conf.d/default.conf diff --git a/psf-memo-indexer/production/docker/memo-db/Dockerfile b/psf-memo-indexer/production/docker/memo-db/Dockerfile index 971c82f..dbc2d50 100644 --- a/psf-memo-indexer/production/docker/memo-db/Dockerfile +++ b/psf-memo-indexer/production/docker/memo-db/Dockerfile @@ -1,18 +1,20 @@ # Create a Dockerized API server # +# Build context: psf-memo monorepo root +# docker compose build memo-db -#IMAGE BUILD COMMANDS FROM ubuntu:24.04 MAINTAINER Chris Troutner #Update the OS and install any OS packages needed. -RUN apt-get update -RUN apt-get install -y sudo git curl nano gnupg wget zip unzip python3 +RUN apt-get update && apt-get install -y sudo curl nano gnupg wget zip unzip python3 \ + && rm -rf /var/lib/apt/lists/* #Install Node and NPM -RUN curl -sL https://deb.nodesource.com/setup_22.x -o nodesource_setup.sh -RUN bash nodesource_setup.sh -RUN apt-get install -y nodejs build-essential +RUN curl -sL https://deb.nodesource.com/setup_22.x -o nodesource_setup.sh \ + && bash nodesource_setup.sh \ + && apt-get update && apt-get install -y nodejs build-essential \ + && rm -rf /var/lib/apt/lists/* #Create the user 'safeuser' and add them to the sudo group. RUN useradd -ms /bin/bash safeuser @@ -30,17 +32,10 @@ RUN chown -R safeuser .npm-global RUN echo "export PATH=~/.npm-global/bin:$PATH" >> /home/safeuser/.profile RUN runuser -l safeuser -c "npm config set prefix '~/.npm-global'" -# Clone the psf-memo-db repository -WORKDIR /home/safeuser -RUN git clone https://github.com/Permissionless-Software-Foundation/psf-memo-db - -# Switch to the desired branch. `master` is usually stable, -# and `stage` has the most up-to-date changes. +# Copy psf-memo-db from the monorepo +COPY psf-memo-db /home/safeuser/psf-memo-db WORKDIR /home/safeuser/psf-memo-db -# For development: switch to unstable branch -#RUN git checkout - # Install dependencies RUN npm install diff --git a/psf-memo-indexer/production/docker/tx-indexer/Dockerfile b/psf-memo-indexer/production/docker/tx-indexer/Dockerfile index 0b883aa..1eb1b0e 100644 --- a/psf-memo-indexer/production/docker/tx-indexer/Dockerfile +++ b/psf-memo-indexer/production/docker/tx-indexer/Dockerfile @@ -1,18 +1,20 @@ # Create a Dockerized tx indexer # +# Build context: psf-memo monorepo root +# docker compose build tx-indexer -#IMAGE BUILD COMMANDS FROM ubuntu:24.04 MAINTAINER Chris Troutner #Update the OS and install any OS packages needed. -RUN apt-get update -RUN apt-get install -y sudo git curl nano gnupg wget zip unzip python3 +RUN apt-get update && apt-get install -y sudo curl nano gnupg wget zip unzip python3 \ + && rm -rf /var/lib/apt/lists/* #Install Node and NPM -RUN curl -sL https://deb.nodesource.com/setup_22.x -o nodesource_setup.sh -RUN bash nodesource_setup.sh -RUN apt-get install -y nodejs build-essential +RUN curl -sL https://deb.nodesource.com/setup_22.x -o nodesource_setup.sh \ + && bash nodesource_setup.sh \ + && apt-get update && apt-get install -y nodejs build-essential \ + && rm -rf /var/lib/apt/lists/* #Create the user 'safeuser' and add them to the sudo group. RUN useradd -ms /bin/bash safeuser @@ -30,17 +32,10 @@ RUN chown -R safeuser .npm-global RUN echo "export PATH=~/.npm-global/bin:$PATH" >> /home/safeuser/.profile RUN runuser -l safeuser -c "npm config set prefix '~/.npm-global'" -# Clone the psf-memo-indexer repository -WORKDIR /home/safeuser -RUN git clone https://github.com/Permissionless-Software-Foundation/psf-memo-indexer - -# Switch to the desired branch. `master` is usually stable, -# and `stage` has the most up-to-date changes. +# Copy psf-memo-indexer from the monorepo +COPY psf-memo-indexer /home/safeuser/psf-memo-indexer WORKDIR /home/safeuser/psf-memo-indexer -# For development: switch to unstable branch -#RUN git checkout - # Install dependencies RUN npm install