DemozPay — System Topology
Audience: a junior engineer (or a senior who hasn't read this codebase yet) who wants to know exactly what runs, what talks to what, and where every piece of code lives. Scope: every process, every port, every database, every HTTP/gRPC API surface, and the end-to-end traces for the three flows that matter most (EWA disburse, settlement webhook, daily reconciliation). Status: reflects Phase A + B + C + D as shipped. Last updated 2026-05-29.
§1. The one-paragraph thesis
When DemozPay is "up", 13 processes are running side-by-side on one machine (or one cluster in production). Some are Node.js, some are Go. They talk to each other via gRPC (typed network calls) and HTTP (for webhooks). The TypeScript code in packages/ is library code that gets compiled INTO apps/api — it doesn't run on its own. The Go code in services/ runs as separate processes with their own databases. The frontends in apps/*-web are separate Next.js processes that call apps/api over HTTP. Three Postgres databases, one Redis, one Redpanda (Kafka), and one stand-in bank (bank-sandbox) round out the picture.
If you understand that paragraph, the rest of this doc is detail.
§2. The three top-level folders — what each one IS
| Folder | What it is | Analogy |
|---|---|---|
packages/ | Library code. Does not run on its own. Other code imports it. | Ingredients in a pantry. |
apps/ | Things that run as Node.js processes (NestJS monolith + Next.js frontends). | Cooked meals on the plates. |
services/ | Things that run as independent Go processes with their own databases. | Specialised stations in the kitchen — the grill, the bar. |
The big distinction: packages/ gets compiled into other things. apps/ and services/ are processes you can start with one command and see in docker ps.
§3. The full picture — every process, every port, every database
YOUR MACHINE (or one prod cluster)
┌─────────────────────────────────────────────────────────────────────────────────────────┐
│ │
│ ┌─────────────────────────────────────────────────────────────────────────────────┐ │
│ │ FRONTENDS │ │
│ │ Next.js processes — separate from the backend, talk to apps/api over HTTP │ │
│ │ │ │
│ │ ┌─────────────────┐ ┌─────────────────┐ ┌─────────────────┐ │ │
│ │ │ apps/admin-web │ │ apps/employer- │ │ apps/employee- │ ... + fi-web, │ │
│ │ │ :4200 │ │ web :4201 │ │ web :4202 │ merchant-web, │ │
│ │ │ platform admin │ │ employer admin │ │ employee app │ docs-web │ │
│ │ └────────┬────────┘ └────────┬────────┘ └────────┬────────┘ │ │
│ └───────────┼───────────────────┼───────────────────┼─────────────────────────────┘ │
│ │ │ │ │
│ └───────────────────┼───────────────────┘ │
│ │ HTTPS (fetch / axios) │
│ ▼ │
│ ┌─────────────────────────────────────────────────────────────────────────────────┐ │
│ │ apps/api (Node.js, port 3000) │ │
│ │ THE NESTJS MONOLITH │ │
│ │ │ │
│ │ This ONE process contains the compiled code of every TypeScript package: │ │
│ │ - packages/ewa/backend/ (EWA bounded context) │ │
│ │ - packages/lending/backend/ (lending bounded context) │ │
│ │ - packages/shared/money/ (Money type) │ │
│ │ - packages/shared/idempotency/ (replay protection) │ │
│ │ - packages/shared/audit/ (audit emitter) │ │
│ │ - packages/shared/events/ (transactional outbox) │ │
│ │ - packages/shared/tenant-context/ (AsyncLocalStorage) │ │
│ │ - packages/shared/auth/ (RBAC decorators, Phase A) │ │
│ │ - packages/shared/database/ (transaction runner) │ │
│ │ - packages/shared/logging/ (Pino + PII redact, Phase A) │ │
│ │ - packages/shared/validation/ (Zod helpers) │ │
│ │ - packages/shared/config/ (env loader) │ │
│ │ - packages/contracts/grpc/*.proto (loaded at runtime by gRPC clients) │ │
│ │ │ │
│ │ HTTP routes it exposes (all under prefix /api): │ │
│ │ /healthz, /readyz, /metrics, / (public — for ops + monitoring) │ │
│ │ /auth/* (better-auth: signup, signin, sessions, org) │ │
│ │ /business/* (employer-org CRUD — admin/owner-gated, Phase A) │ │
│ │ /employees/* (employee CRUD — admin/owner-gated, Phase A) │ │
│ │ /ewa/eligibility (read accrued earnings) │ │
│ │ /ewa/requests (create EWA request) │ │
│ │ /ewa/requests/:id/disburse (Phase B1 + Phase C lookup gate) │ │
│ │ /ewa/requests/:id/record-repayment (Phase B1 admin endpoint) │ │
│ │ /loans/quote (compute repayment schedule) │ │
│ │ /loans (create loan request) │ │
│ │ /loans/:id/disburse (Phase B1 + Phase C lookup gate) │ │
│ │ /loans/:id/installments/:idx/record-repayment (Phase B1) │ │
│ │ /loans/:id/installments/:idx/remit-to-fi (Phase B2) │ │
│ │ /integration/bank-callback/:partner (HMAC-signed webhook receiver) │ │
│ └────────┬────────────────────────────┬─────────────────────┬────────────────────┘ │
│ │ │ │ │
│ │ Prisma (TCP+SQL) │ gRPC │ gRPC │
│ ▼ ▼ ▼ │
│ ┌─────────────────┐ ┌────────────────────────────┐ ┌──────────────────────────────┐ │
│ │ postgres :5432 │ │ services/ledger │ │ services/integration-gateway │ │
│ │ apps/api's DB │ │ Go process │ │ Go process │ │
│ │ │ │ - :50051 gRPC │ │ - :50052 gRPC │ │
│ │ tables: │ │ - :50054 /metrics + /healthz│ │ - :50053 HTTP (webhooks + │ │
│ │ EwaRequest │ │ (Phase D) │ │ /metrics) │ │
│ │ Loan │ │ │ │ │ │
│ │ LoanRepayment │ │ RPCs: │ │ RPCs: │ │
│ │ Business │ │ PostTransaction │ │ InitiateDisbursement │ │
│ │ Employee │ │ ConfirmSettlement │ │ LookupAccount (Phase C) │ │
│ │ User │ │ MarkSettlementFailed │ │ GetDisbursementStatus │ │
│ │ Session │ │ Reverse │ │ GetAdapterStatus │ │
│ │ Member │ │ GetBalance │ │ │ │
│ │ Organization │ │ GetEntries │ │ Adapters loaded: │ │
│ │ outbox_event │ │ ReconcileAccount │ │ - mock │ │
│ │ audit_entry │ │ ReconcileWithBank │ │ - dashen (HTTP+HMAC) │ │
│ │ idempotency_* │ │ │ │ │ │
│ └─────────────────┘ └────────────┬───────────────┘ └────────────┬─────────────────┘ │
│ │ │ │
│ ▼ ▼ │
│ ┌────────────────────┐ ┌────────────────────┐ │
│ │ pg-ledger :5441 │ │ pg-gateway :5442 │ │
│ │ ledger's OWN DB │ │ gateway's OWN DB │ │
│ │ │ │ │ │
│ │ tables: │ │ tables: │ │
│ │ ledger_account │ │ disbursement │ │
│ │ ledger_transaction│ │ bank_event │ │
│ │ ledger_entry │ │ bank_statement_ │ │
│ │ │ │ line │ │
│ │ FORCE RLS on all │ │ FORCE RLS on all │ │
│ └────────────────────┘ └─────────┬──────────┘ │
│ │ │
│ │ HTTPS+HMAC (outbound) │
│ ▼ │
│ ┌──────────────────────────────────────────────────────────────────────────────────┐ │
│ │ services/bank-sandbox (Go, port 8088) │ │
│ │ Pretends to be a real Ethiopian bank (Dashen wire shape). │ │
│ │ When real Dashen onboards: env var swap, same code. │ │
│ │ │ │
│ │ HTTP routes: │ │
│ │ POST /api/v1/transfers (initiate; HMAC required) │ │
│ │ GET /api/v1/transfers/:ref (status; HMAC) │ │
│ │ GET /api/v1/accounts/:account (account lookup, Phase C) │ │
│ │ POST /admin/force-settle/:ref (test helper, no auth) │ │
│ │ POST /admin/force-fail/:ref (test helper) │ │
│ │ GET /healthz │ │
│ │ │ │
│ │ It SENDS WEBHOOKS to integration-gateway's :50053 when transfers settle. │ │
│ └──────────────────────────────────────────────────────────────────────────────────┘ │
│ │
│ ┌──────────────────────────────────────────────────────────────────────────────────┐ │
│ │ services/notifications (Go, port 3003) │ │
│ │ STUB — only /healthz responds today. Planned: SMS/email consumer of │ │
│ │ outbox events (ewa.bank_transfer_settled.v1, etc.). │ │
│ └──────────────────────────────────────────────────────────────────────────────────┘ │
│ │
│ ┌──────────────────────────────────────────────────────────────────────────────────┐ │
│ │ apps/api ALSO talks to: │ │
│ │ - redis :6379 (EWA eligibility cache; optional) │ │
│ │ - redpanda :9092 (Kafka — outbox events get published here) │ │
│ │ - redpanda-console :8080 (web UI for browsing topics, dev-only) │ │
│ └──────────────────────────────────────────────────────────────────────────────────┘ │
│ │
│ ┌──────────────────────────────────────────────────────────────────────────────────┐ │
│ │ services/integration-gateway/cmd/recon-runner (Phase D) │ │
│ │ Standalone Go binary. Run on a schedule (cron / k8s CronJob). │ │
│ │ Reads pg-gateway, calls Matcher + Runner, emits JSON summary on stdout. │ │
│ │ NOT a long-running process — runs and exits. │ │
│ └──────────────────────────────────────────────────────────────────────────────────┘ │
│ │
└─────────────────────────────────────────────────────────────────────────────────────────┘
§4. The ports table — everything that listens
| Process | Port | Protocol | Purpose | Auth |
|---|---|---|---|---|
apps/api | 3000 | HTTP | Main API + /api/metrics for Prometheus | Per-route (see Phase A permission-matrix.md) |
apps/admin-web | 4200 | HTTP | Platform admin frontend | better-auth session |
apps/employer-web | 4201 | HTTP | Employer admin frontend | better-auth session |
apps/employee-web | 4202 | HTTP | Employee app | better-auth session |
apps/fi-web | 4203 | HTTP | FI partner frontend | better-auth session |
apps/merchant-web | 4204 | HTTP | BNPL merchant frontend | better-auth session |
apps/docs-web | 4205 | HTTP | Public docs site | none (public) |
services/ledger | 50051 | gRPC | Money-truth RPCs | None today; mTLS is PLANNED |
services/ledger | 50054 | HTTP | /metrics, /healthz (Phase D) | none |
services/integration-gateway | 50052 | gRPC | Disbursement + LookupAccount RPCs | None today |
services/integration-gateway | 50053 | HTTP | Inbound partner webhooks + /metrics + /healthz | HMAC per partner |
services/bank-sandbox | 8088 | HTTP | Test partner bank API | HMAC |
services/notifications | 3003 | HTTP | /healthz (stub) | none |
postgres (apps/api DB) | 5432 | SQL | API's transactional DB | Prisma role + RLS |
pg-ledger | 5441 | SQL | Ledger's DB | ledger role + RLS (no BYPASSRLS) |
pg-gateway | 5442 | SQL | Gateway's DB | gateway role + RLS |
redis | 6379 | TCP | EWA eligibility cache | none (private network) |
redpanda | 9092 | Kafka | Outbox event publishing | none (private network) |
redpanda-console | 8080 | HTTP | Kafka UI (dev only) | none |
§5. The directory map — what's where
apps/
| Path | What's there |
|---|---|
apps/api/ | NestJS monolith (the backend). Composition root: src/app/app.module.ts. |
apps/api/prisma/ | The API's Prisma schema + 12 migrations. |
apps/api/src/identity/auth/ | better-auth wiring + AuthGuard + OrgRoleGuard + permission decorators (Phase A). |
apps/api/src/business/ | Employer-org CRUD service + controller. |
apps/api/src/workforce/employee/ | Employee CRUD service + controller. |
apps/api/src/products/ewa/ | apps/api side of EWA — Prisma repo + gRPC clients + ledger-accounts adapter. |
apps/api/src/products/lending/ | apps/api side of lending — same shape. |
apps/api/src/money/integration/ | Bank webhook controller + settlement poller + applier (Phase S3). |
apps/api/src/_infra/observability/ | Pino logger + Prometheus registry + OTel tracing. |
apps/api/src/_infra/shared-infra/ | Prisma-backed transaction runner / outbox / audit / idempotency. |
apps/api/src/identity/tenant/ | TenantContextMiddleware (Phase A — strict, no header fallback). |
apps/{admin,employer,employee,fi,merchant,docs}-web/ | Next.js frontends. |
apps/*-e2e/ | Playwright end-to-end tests per frontend. |
services/
| Path | What's there |
|---|---|
services/ledger/ | Go gRPC service. The money source of truth. Phase D: /metrics + interceptor. |
services/ledger/migrations/ | 4 SQL migrations. Append-only triggers; FORCE RLS; deferred balance check. |
services/integration-gateway/ | Go gRPC service. Partner-bank adapter framework. |
services/integration-gateway/internal/adapters/ | Per-partner adapters (mock, dashen — Phase C). |
services/integration-gateway/internal/reconciliation/ | Statement ingester + matcher + runner (Phase S4). |
services/integration-gateway/cmd/recon-runner/ | Standalone scheduled binary (Phase D). |
services/bank-sandbox/ | Go HTTP service that fakes a real partner bank (Dashen wire shape). |
services/notifications/ | Go stub — /healthz only. Planned: SMS/email outbox consumer. |
packages/
| Path | What's there |
|---|---|
packages/ewa/backend/ | EWA bounded context (domain + application + presentation). |
packages/lending/backend/ | Lending bounded context. |
packages/shared/money/ | Money type (NUMERIC(20,0) santim, never floats). |
packages/shared/idempotency/ | Idempotency-Key store contract. |
packages/shared/audit/ | Audit emitter contract. |
packages/shared/events/ | Outbox repository contract. |
packages/shared/tenant-context/ | AsyncLocalStorage tenant + actor (Phase A1 added actor). |
packages/shared/auth/ | RBAC decorators (Phase A) — @RequireOrgRole, @RequirePlatformAdmin. |
packages/shared/database/ | Transaction runner abstraction. |
packages/shared/logging/ | Logger contract + PII mask helpers (Phase A3). |
packages/shared/validation/ | Zod helpers. |
packages/shared/config/ | Env loader. |
packages/contracts/grpc/ | .proto files — ledger.proto, integration_gateway.proto. |
packages/contracts/gen/go/ | Generated Go stubs for both protos. |
infra/
| Path | What's there |
|---|---|
infra/docker-compose.test.yml | Spins up the full Go stack for testing (3 postgres + ledger + gateway + bank-sandbox + api). |
infra/sql/ | Helper SQL (BYPASSRLS role provisioning). |
infra/prometheus/alerts.yml | Phase D — 15 alert rules. |
infra/prometheus/prometheus.yml | Phase D — scrape config template. |
§6. How packages/ get into apps/api — the "in-process" wiring
Key fact: importing a package is NOT a network call. It is a normal TypeScript import. The package code becomes part of the apps/api binary.
How it works:
pnpm-workspace.yamllistspackages/**as workspace members.pnpm installcreates symlinks innode_modules/@demoz-pay/*pointing to the actualpackages/<name>/folders.tsconfig.base.jsonhas path aliases:"@demoz-pay/ewa": ["packages/ewa/backend/src/index.ts"].- When apps/api imports
from '@demoz-pay/ewa', TypeScript follows the alias. - At build time, webpack bundles the imported code into
apps/api/dist/main.js. - At runtime, when apps/api starts, every package's code is already inside the Node process — no inter-process communication.
Concrete example. Open apps/api/src/app/app.module.ts:
import { EwaController, EwaModule } from '@demoz-pay/ewa';
import { LendingController, LendingModule } from '@demoz-pay/lending';
// ...
@Module({
imports: [
// ... shared infra modules ...
EwaApiModule, // binds EWA's ports to Prisma adapters
EwaModule.register(), // mounts EWA's controller + use cases
LendingApiModule,
LendingModule.register(),
IntegrationModule,
],
})
EwaModule.register() lives in packages/ewa/backend/presentation/ewa.module.ts. NestJS treats it as if it were defined in apps/api itself.
§7. How apps/api talks to services/ — the "network" wiring
This IS a network call. apps/api and services/ledger are separate processes.
How it works:
- The contract lives in
packages/contracts/grpc/ledger.proto. services/ledger(Go) generates Go server stubs from the proto viabuf generateand IMPLEMENTS them.apps/apiloads the same.protofile at runtime using@grpc/proto-loaderand creates a CLIENT.- The client opens a TCP connection to
localhost:50051(dev) orledger:50051(docker). - Calls like
client.PostTransaction(request, callback)serialise the request, send it over TCP, the ledger handles it, response comes back, callback fires.
Concrete example. apps/api/src/products/ewa/ledger.grpc-client.ts:
this.client = new grpcObject.demozpay.ledger.v1.Ledger(
process.env.LEDGER_GRPC_ADDR ?? 'localhost:50051',
credentials.createInsecure(),
);
// Later, inside a use case:
this.client.PostTransaction(request, (err, resp) => { ... });
If services/ledger isn't running, this call hangs and eventually times out. That's why verify-s3.sh brings up all 3 services before running tests.
§8. End-to-end trace #1 — EWA disburse (the happy path)
This is the canonical money-moving flow. Phases A + B + C + D all touch it. Following the trace makes the architecture click.
USER (employer admin)
│
│ 1. Clicks "Disburse" in employer-web
│ HTTPS: POST :4201/.../disburse
▼
apps/employer-web (Next.js, :4201)
│
│ 2. Forwards to backend:
│ HTTPS: POST :3000/api/ewa/requests/ewa_001/disburse
│ Headers: cookie=better-auth-session; Idempotency-Key=ik_xyz
▼
apps/api (NestJS, :3000)
│
│ 3. SessionMiddleware reads cookie → req.user = { id, businessId, ... }
│ TenantContextMiddleware:
│ - rejects if 'x-actor-id' header present (Phase A1)
│ - sets AsyncLocalStorage: { tenantId, actorId }
│ AuthGuard verifies req.user.id exists
│ OrgRoleGuard runs (Phase A2):
│ - reads @RequireOrgRole('admin','owner') metadata
│ - looks up Member(userId, tenantId, role)
│ - 403 if not admin/owner
│
│ 4. NestJS routes to packages/ewa/backend/presentation/ewa.controller.ts:disburse()
│ This code RUNS INSIDE apps/api (in-process — see §6).
│
│ 5. EwaController calls DisburseEwaUseCase
│ (packages/ewa/backend/application/disburse-ewa.usecase.ts)
│
│ 6. Use case loads EwaRequest from Prisma (apps/api's DB :5432)
│
│ 7. PHASE C / GL-04 gate — lookupAccount FIRST:
│ ─────────────────────────────────────────
│ IntegrationGatewayClient.lookupAccount({ destination, partner })
│ │
│ │ gRPC call to gateway
│ ▼
│ services/integration-gateway (Go, :50052)
│ │ Routes to dashen adapter (or mock per env)
│ ▼
│ adapter HTTP+HMAC GET bank-sandbox :8088/api/v1/accounts/:account
│ │
│ ▼ 200 {exists:true, resolved_name:"..."} OR 404 + typed reason
│ │
│ ◄── exists:false → throw EwaDestinationAccountInvalidError
│ → audit row written, NO ledger entry, NO disburse
│ → 409 to caller
│
│ 8. If lookup OK: post PENDING ledger entry
│ LedgerClient.postTransaction(...)
│ │
│ │ gRPC PostTransaction
│ ▼
│ services/ledger (Go, :50051)
│ │ writes to pg-ledger :5441
│ │ ledger_transaction (status=PENDING)
│ │ ledger_entry (DR receivable, CR payable, CR fee)
│ ▼ deferred trigger checks Σdebits = Σcredits
│ ◄── returns tx_id, status=PENDING
│
│ 9. Initiate the actual transfer
│ IntegrationGatewayClient.disburse(...)
│ │
│ │ gRPC InitiateDisbursement
│ ▼
│ services/integration-gateway (Go, :50052)
│ │ writes disbursement row to pg-gateway :5442 (status=SUBMITTED)
│ │ Calls dashen adapter
│ │ adapter HTTP+HMAC POST bank-sandbox :8088/api/v1/transfers
│ │ bank-sandbox stores transfer row, returns ACCEPTED
│ │ bank-sandbox schedules an async webhook
│ ▼
│ ◄── returns partner_reference, bankStatus=PENDING
│
│ 10. Use case persists EwaRequest as SUBMITTED_TO_BANK
│ Emits outbox event ewa.bank_transfer_submitted.v1
│ Audit row in same DB transaction (ADR-008)
│ All in apps/api's DB :5432
│
◄── 200 OK to client: { id, status: 'SUBMITTED_TO_BANK', providerRef, ... }
... time passes ...
bank-sandbox (Go, :8088)
│ Async settlement: marks transfer COMPLETED
│ Dispatches signed webhook → POST gateway :50053/webhooks/dashen
▼
services/integration-gateway (:50053)
│ Verifies HMAC signature
│ Forwards normalised event to apps/api
▼ POST apps/api :3000/api/integration/bank-callback/dashen
apps/api (BankWebhookController)
│ Verifies HMAC again (defense in depth)
│ BankSettlementApplier.apply():
│ - finds EwaRequest by providerRef (BYPASSRLS cross-tenant lookup)
│ - runWithTenant({ tenantId: ewa.tenantId, actorId: 'system:bank-settlement-applier' })
│ - LedgerClient.confirmSettlement(tx_id) → PENDING → POSTED in pg-ledger
│ - EwaRequest → DISBURSED
│ - outbox event ewa.bank_transfer_settled.v1
│ - audit row, all in apps/api DB
◄── 200 OK { matched:true, alreadyApplied:false, domain:'EWA' }
That trace touches every layer. Read it once and the whole topology clicks.
§9. End-to-end trace #2 — daily reconciliation (Phase D)
SCHEDULER (cron / k8s CronJob / operator)
│
│ Once per day at e.g. 07:30 UTC:
▼
$ /recon-runner --tenants biz_acme,biz_beta --partners dashen
│
│ The runner is services/integration-gateway/cmd/recon-runner/main.go
│ Standalone Go binary. Reads GATEWAY_DATABASE_URL.
│
▼ For each (tenant, partner):
services/integration-gateway/internal/reconciliation/Runner.Run()
│
│ 1. Store.ListUnmatched(tenant, partner) — page from pg-gateway :5442
│ → bank_statement_line rows where matched_disbursement_id IS NULL
│
│ 2. For each line, Matcher.Match():
│ │ Store.FindByPartnerReference() — same pg-gateway query
│ │ reads disbursement table
│ │ Compares amount + currency + value_date ± 24h
│ │
│ ├─ matched → Store.MarkMatched(line, disbursement_id)
│ └─ flagged → Store.MarkFlagged(line, reason)
│
▼
recon-runner emits JSON summary on stdout:
{
"runs": [
{"tenant_id":"biz_acme", "partner":"dashen",
"scanned": 12, "matched": 12, "flagged": 0, ...}
],
"total_flagged": 0,
"exit_code": 0,
"next_action_hint": "all clear"
}
OPERATOR SIDE
│
│ Pipe JSON to a Slack webhook or Loki / log shipper
│ Read it in the morning. If exit_code != 0 OR total_flagged > 0:
│ → consult docs/runbooks/drift-detected.md
Per infra/prometheus/alerts.yml, when the runner emits Prometheus counters (Phase D continuation item), DemozpayReconciliationRunnerMissedDay fires if no successful run for 24h.
§10. End-to-end trace #3 — what happens when you pnpm dev:api
Useful to see boot order.
$ pnpm dev:api
│
▼
nx serve api
│ Compiles all packages/* + apps/api into one bundle
▼
node dist/apps/api/main.js
│
│ main.ts boot sequence:
│ 1. NestFactory.create(AppModule, { rawBody: true })
│ 2. ConfigModule loads env, validates with Zod
│ 3. PrismaModule connects to postgres :5432
│ 4. AuthModule constructs better-auth singleton
│ 5. EwaModule.register() binds EWA use cases to ports
│ EwaApiModule binds those ports to:
│ - PrismaEwaRequestRepository (apps/api/src/products/ewa/...)
│ - IntegrationGatewayClient (loads .proto, opens TCP to gateway)
│ - LedgerGrpcClient (loads .proto, opens TCP to ledger)
│ 6. LendingModule.register() — same shape
│ 7. IntegrationModule wires bank-webhook controller + settlement poller
│ 8. HealthModule starts startup checks (postgres, redis, kafka, ledger)
│ ◄── if any fails, the API still starts but /readyz reports degraded
│ 9. MetricsModule mounts /api/metrics
│ 10. Express adapter binds to PORT (default 3000)
│
◄── log line: "Nest application successfully started" :3000
§11. How to build everything
A. Local dev (fastest)
# Once after cloning:
pnpm install # installs all Node deps + symlinks packages/*
pnpm docker:up # boots postgres :5432 + redis + redpanda
pnpm prisma:migrate # apply API's migrations
# Bring up Go services (in a separate terminal):
docker compose -p demoz -f infra/docker-compose.test.yml up -d \
pg-ledger pg-gateway bank-sandbox ledger gateway
# Then in another terminal:
pnpm dev:api # apps/api with hot reload
pnpm dev # all 5 frontends in parallel
You now have everything talking. Open http://localhost:3000/api/healthz and you should see {"status":"ok"}.
B. Verify the stack works end-to-end
# S3 — the canonical bank-webhook + applier smoke test
./services/bank-sandbox/test/verify-s3.sh
# S4 — reconciliation primitive
./services/ledger/test/verify-s4-recon.sh
./services/integration-gateway/test/verify-s4-recon.sh
# Phase C — LookupAccount + safe disburse gate
./services/bank-sandbox/test/verify-c-lookup.sh
Each script brings up its own docker-compose stack, runs the assertions, and tears down. If they all pass, your local install is healthy.
C. Build all Docker images at once
Each service has its own Dockerfile:
| Image | Built from |
|---|---|
demoz-api | apps/api/Dockerfile |
demoz-ledger | services/ledger/Dockerfile |
demoz-gateway | services/integration-gateway/Dockerfile |
demoz-bank-sandbox | services/bank-sandbox/Dockerfile |
demoz-notifications | services/notifications/Dockerfile |
demoz-*-web | NOT YET — frontends have no Dockerfile. Use pnpm build + serve static today. |
# Regenerate Go stubs from proto (only when proto changes)
cd packages/contracts && ../../.tools/bin/buf generate
cd -
# Build all images in one shot
docker compose -p demoz -f infra/docker-compose.test.yml build
§12. How to deploy via Docker (today)
There are two working compose files depending on what you want:
| File | What's in it | When to use |
|---|---|---|
infra/docker-compose.full.yml (recommended) | The WHOLE platform: backend + 5 frontends + Prometheus monitoring stub + 3 databases + Redis + Redpanda. One command. | "I want to see the platform running end-to-end on my machine." |
infra/docker-compose.test.yml | Backend only (no frontends, no Prometheus). Used by the verify-*.sh scripts. | "I'm running the canonical regression tests." |
Full stack (one command)
# Boot everything
docker compose -p demoz -f infra/docker-compose.full.yml up -d --build
# Apply the API DB migrations (one-shot profile)
docker compose -p demoz -f infra/docker-compose.full.yml --profile migrate \
run --rm api-migrate
# Apply ledger + gateway migrations (raw SQL, no Prisma)
psql postgresql://ledger:ledger@localhost:5441/ledger \
-f services/ledger/migrations/0001_init.up.sql \
-f services/ledger/migrations/0002_reverses_unique.up.sql \
-f services/ledger/migrations/0003_pending_posted.up.sql \
-f services/ledger/migrations/0004_allow_metadata_mutation.up.sql
psql postgresql://gateway:gateway@localhost:5442/gateway \
-f services/integration-gateway/migrations/0001_init.up.sql \
-f services/integration-gateway/migrations/0002_bank_statement_line.up.sql
# Verify it's healthy
curl http://localhost:3030/api/healthz # apps/api (NOT :3000 — see port table)
curl http://localhost:50053/healthz # gateway
curl http://localhost:50054/healthz # ledger (Phase D)
curl http://localhost:50054/metrics | head # Phase D metrics surface
curl http://localhost:8088/healthz # bank-sandbox
# Browse the frontends
open http://localhost:4200 # admin
open http://localhost:4201 # employer
open http://localhost:4202 # employee
open http://localhost:4203 # fi
open http://localhost:4204 # merchant
# Browse Prometheus + Kafka UI
open http://localhost:9090 # Prometheus — see metrics + alert rules
open http://localhost:9080 # Redpanda Console — browse outbox topics
# Run the Phase D recon-runner (one-shot)
docker compose -p demoz -f infra/docker-compose.full.yml --profile recon \
run --rm recon-runner --tenants biz_pilot_acme --partners dashen
# Tear down (volumes too)
docker compose -p demoz -f infra/docker-compose.full.yml down -v
Port plan — deliberately offset from common defaults
The full compose uses non-default host ports so you can boot it alongside other dev work on the same machine (including telemedhin, which holds host ports 3000, 3001-3005, 5432, 5433). Container-internal ports are unchanged.
| Service | Host port | Container port | Why offset |
|---|---|---|---|
api | 3030 | 3000 | telemedhin-landing-dev holds 3000 |
notifications | 3033 | 3003 | telemedhn-prod-client-1 holds 3003 |
postgres (API DB) | 5440 | 5432 | telemedhin-db-dev holds 5432 |
pg-ledger | 5441 | 5432 | |
pg-gateway | 5442 | 5432 | |
ledger gRPC | 50051 | 50051 | |
ledger HTTP (Phase D /metrics) | 50054 | 50054 | |
gateway gRPC | 50052 | 50052 | |
gateway HTTP webhooks + /metrics | 50053 | 50053 | |
bank-sandbox | 8088 | 8088 | |
redis | 6379 | 6379 | |
redpanda Kafka | 9092 | 9092 | |
redpanda-console | 9080 | 8080 | common :8080 clash |
prometheus | 9090 | 9090 | |
admin/employer/employee/fi/merchant-web | 4200/4201/4202/4203/4204 | same |
Backend-only stack (for verify scripts)
docker compose -p demoz-test -f infra/docker-compose.test.yml up -d --build
# The verify-*.sh scripts use this stack.
Older compose at repo root
docker-compose.yml (at the repo root) is a dev-mode source-mount variant — uses nx serve with the workspace mounted. It binds host ports 3000 + 5432 which clash with telemedhin. Don't use it on this machine; the full compose above supersedes it. Kept for compat with repos cloned elsewhere.
§13. What's NOT yet shipped for production (honest list)
| Item | Status | Why it matters |
|---|---|---|
Unified docker-compose.prod.yml for the FULL stack incl. frontends + reverse proxy + TLS | PLANNED | Today you boot test stack + run pnpm dev separately. |
| Kubernetes manifests (Deployments, Services, Ingress) | PLANNED | Needed for any real production cluster. |
| Terraform / IaC for cloud infra | PLANNED | Need Postgres, Redis, Kafka as managed services + load balancer + DNS. |
| TLS termination at ingress | PLANNED — GL-17 | Today every port is plain HTTP/gRPC. |
| Real Postgres backups + PITR | PLANNED — GL-18 | Today no backup runs anywhere. |
| Prometheus + Alertmanager + Grafana deployed | PLANNED — GL-08 | Phase D has rules + scrape config as code; nothing loads them yet. |
| Frontend Dockerfiles | PLANNED | apps/*-web have no Dockerfile today; build with pnpm build and serve static output. |
| Real Dashen sandbox swap | OPERATIONAL — credentials needed | Adapter code is ready; needs GATEWAY_DASHEN_BASE_URL + GATEWAY_DASHEN_SIGNING_KEY env. |
§14. Quick mental model — for the back of your head
- One
apps/apiprocess holds every bounded-context's code (EWA, lending, etc.). They run together because the monolith pattern (ADR-001) is cheaper to operate than 10 microservices we don't need yet. - Three Go services because each has a distinct security or operational boundary: ledger holds money-truth (isolated DB, no BYPASSRLS), gateway holds partner credentials (isolated DB, network egress to partners), notifications has outbound-only credentials. (Notifications is currently a stub.)
- Three Postgres databases for blast-radius isolation. If one corrupts, the other two are unaffected.
- One Redpanda (Kafka) for outbox event publishing. Consumers don't exist yet — events fall into a void.
- One Redis for the EWA eligibility cache. Optional — falls back to in-memory.
- One bank-sandbox for end-to-end testing without a real partner.
Everything else is either documentation, build infrastructure, or future-planned.
§15. Cross-references
- High-level project context →
CLAUDE.md(terse) +README.md(verbose). - The bank-orchestration model these flows operate under →
BANK_ORCHESTRATION.md. - The reconciliation lifecycle that catches when these flows lie →
RECONCILIATION_ARCHITECTURE.md. - Per-flow status with verify-script evidence →
MONEY_FLOWS.md. - What's LIVE vs STUB across every domain →
REAL_SYSTEM_STATE.md+DOMAIN_COMPLETENESS_MATRIX.md. - Phase-by-phase changes (chronological) →
PHASE_C_LOOKUP_ACCOUNT.md,SLOS_AND_ALERTING.md. - Junior-friendly onboarding pack →
docs/onboarding/(starts atREADME.md). - Operational runbooks →
docs/runbooks/.
§16. If you remember only one thing
packages/is code.apps/andservices/are processes.apps/apitalks topackages/by importing (in-memory).apps/apitalks toservices/over gRPC (network). Three Postgres databases. One fake bank that pretends to be Dashen. That's the whole platform.