Self-hosting the scheduler

The scheduler is a Nest, Fastify, and BullMQ app. It owns the cron jobs (subscription sweeper, invoice-overdue marker, subscription-lapsed transitions) and the webhook delivery queue. At boot it also warms the webhook-secret cache so the API can verify outgoing signatures without a Postgres round-trip.

The relayer's signing key does not live in the scheduler. It lives in apps/api, where it is accessed through a KMS provider. The scheduler enqueues work and the API signs and broadcasts.

Build

cd apps/scheduler
pnpm build
pnpm start:prod

Configuration

NODE_ENV=production
PORT=4200
DATABASE_URL=postgresql://...
REDIS_URL=redis://...

ARC_ENVIRONMENT=mainnet
ARC_RPC_URL=https://your-arc-rpc.com

STRIMZ_API_URL=http://api:4000      # the relayer endpoint the scheduler calls
STRIMZ_API_INTERNAL_TOKEN=...       # shared secret for internal API calls

STRIMZ_WEBHOOK_SIGNING_SECRET=...   # 32+ chars
WEBHOOK_DELIVERY_TIMEOUT_MS=10000
WEBHOOK_MAX_ATTEMPTS=5

SUBSCRIPTION_SWEEPER_CRON=0 */15 * * * *
INVOICE_OVERDUE_CRON=0 0 * * * *
SUBSCRIPTION_LAPSED_CRON=0 0 * * * *

RESEND_API_KEY=...
RESEND_FROM_EMAIL=noreply@your-domain.com

Talking to the relayer

On every tick that needs a transaction broadcast (subscription charge batches, on-chain cancels, agent job operations, CCTP settlements), the scheduler posts an internal request to apps/api. The API loads its KMS-backed key, signs, broadcasts, and writes the resulting TxRequest row. The scheduler never sees the key material.

Operations the relayer signs on behalf of the scheduler:

Operation	Contract / Function
Cancel a subscription	`StrimzSubscriptions.cancel(subId)`
Charge subscriptions	`StrimzSubscriptions.batchCharge(ids[], attemptIds[])`
Create an agent job	`StrimzAgentEscrow.createJob(...)`
Release a job	`StrimzAgentEscrow.approveAndRelease(jobId)`
Dispute a job	`StrimzAgentEscrow.dispute(jobId, reason)`
Cancel a job	`StrimzAgentEscrow.cancelJob(jobId, reason)`
Settle CCTP routing	`MessageTransmitter.receiveMessage(...)`

Multi-instance safety

Two scheduler instances behind a load balancer are safe to run side by side:

The subscription sweeper uses Postgres UPDATE … WHERE chargeLock=false to claim work atomically.
BullMQ guarantees each job is delivered to exactly one worker.
The webhook delivery worker no-ops on any WebhookDelivery.status that's already terminal.

If you're processing over 100k webhook deliveries a day, scale horizontally. Throughput grows roughly linearly with replicas.

Observability

/healthz, /readyz
BullMQ dashboard at /admin/queues (gated by an admin password)
Prometheus metrics for job counts, latency, failure rates

Failure modes

API outage. The scheduler's call to the relayer fails; BullMQ retries with exponential backoff. No transaction is broadcast until the API is back.
DB outage. Workers fail their current job. BullMQ retries. The job stays in the queue.
Redis outage. The scheduler can't pick up new jobs. Existing in-flight work completes.
API broadcasts but the scheduler crashes before recording. The on-chain side succeeded. The indexer projects the resulting event. The DB row catches up on the next sweep.

Source

github.com/StrimzLab/strimz/tree/main/apps/scheduler