Skip to content

RBS Ecosystem Infrastructure

Scheduled jobs, health checks, dead-letter queues, and background coordination across all 19 apps. Last Audit: April 9, 2026 — 70+ infrastructure endpoints across 3 scheduler types.


Scheduling Architecture

The ecosystem uses three distinct scheduling layers, chosen by constraint:

LayerTechnologyWho Uses ItWhen to Use
App cronVercel Cron → HTTP endpointCamino, Constanza, Ecosystem APIPrimary jobs that need Vercel's serverless timeout and env vars
Queue cronUpstash QStashColectiva, CaracolFirebase apps (no Supabase pg_cron), need signed delivery and retry
DB cronSupabase pg_cronCamino, Agora, Baúl, Patadas, La Hoja, Plenura, Ecosystem APIDatabase-internal jobs, keep-alives; run inside the DB network

All HTTP cron endpoints use timingSafeEqual comparison on a CRON_SECRET or equivalent Bearer token. QStash endpoints additionally verify Upstash HMAC signatures via @upstash/qstash Receiver.


Camino — The Ecosystem's Primary Dispatcher

Camino runs the most critical daily batch via a single Vercel Cron trigger that fans out to 18 sub-jobs sequentially:

Schedule: 0 14 * * * (08:00 CDT) — POST /api/cron/daily-dispatcher

#Sub-jobPurpose
1expire-trialsMark overdue trial subscriptions as expired
2trial-expiry-remindersEmail at 7d / 3d / 1d / expired
3nurture-emailsDrip email sequences for CRM leads
4publish-scheduled-postsRelease scheduled social media posts
5refresh-tokensRotate OAuth + API tokens before expiry
6optimize-landingsAI A/B test landing pages; promote winners
7optimize-pricesGeo + demand pricing updates
8campaigns/enrollAuto-enroll leads matching campaign rules
9sync-commentsFetch social comments + AI auto-response
10social-listening-scanBrand mention scanning across platforms
11usage-alertsFire at 80% and 100% of tier limits
12upsell-alertsCross-sell: missing Constanza, idle leads
13webhook-retryRe-send failed ecosystem webhooks (exponential backoff)
14reconcile-orgsRetry Colectiva registration for orphaned orgs
15sync-metricsAggregate and push platform metrics
16cleanup-audit-logsDelete logs older than retention policy
17cleanup-oauth-codesExpire OAuth codes, revoked tokens, idempotency keys
18process-scheduled-messagesDispatch queued SMS/Email messages

Camino also has two pg_cron jobs that run inside its Supabase DB (via pg_net HTTP calls):

JobScheduleEndpoint
sync-constanza-invoicesEvery 6 hoursPOST /api/cron/sync-constanza-invoices
voice-outboundEvery 15 min, 09:00–19:00 Mon–Sat MXPOST /api/cron/voice-outbound

Colectiva — QStash Scheduler (15 Jobs)

Colectiva uses Upstash QStash exclusively (Firebase app — no pg_cron). Jobs are registered via scripts/cron/register-qstash-schedules.mjs. All endpoints verify QStash HMAC signatures.

Revenue & Billing (Daily)

EndpointSchedulePurpose
/api/cron/subscription-renewals0 8 * * *Renew subscriptions, retry failed payments
/api/cron/earnings-reconciliation0 12 * * *Reconcile earnings, fire threshold alerts
/api/cron/sync-polizas0 16 * * *Sync queued polizas to Constanza
/api/cron/offering-expiration0 14 * * *Auto-close sold-out/expired CPI offerings
/api/cron/deal-billingDailyDeal-specific billing operations
/api/cron/ai-invoicesDailyAI usage invoice batch processing

AI & Intelligence (Daily)

EndpointSchedulePurpose
/api/cron/daily-brainDailyBrain briefing generation
/api/cron/daily-intelligenceDailyIntelligence report generation
/api/cron/process-approved-actionsDailyExecute Brain-approved actions

Ecosystem Coordination

EndpointSchedulePurpose
/api/cron/ecosystem-heartbeat*/30 * * * *Every 30 min — health + fan-out to Agora, La Hoja, Servilleta
/api/cron/ecosystem-sync0 10 * * 0Weekly Sunday — ecosystem data snapshot for AI

Monthly

EndpointSchedulePurpose
/api/cron/monthly-first0 9 1 * *Revenue recognition + earnings poliza + distributions
/api/cron/monthly-fifth0 9 5 * *5th-day monthly operations
/api/cron/revenue-reconciliationMonthlyRevenue reconciliation
/api/cron/earnings-polizaMonthly 1stEarnings poliza generation

The Ecosystem Heartbeat

/api/cron/ecosystem-heartbeat is the most important infrastructure job nobody talks about. Running every 30 minutes, it:

  1. Pings Colectiva's own health
  2. Fan-out to 3 apps that lack their own Vercel Cron: calls Agora keep-alive, La Hoja dispatcher, and Servilleta expire-tasks
  3. Logs results to health_checks table

This means Agora, La Hoja, and Servilleta's background work is triggered by Colectiva — not by their own vercel.json.


Constanza — Certificate & Billing Crons

vercel.json crons:

ScheduleEndpointPurpose
0 15 * * * (09:00 CDT)/api/cron/check-certificate-expiryMonitor X.509 CSD certificate expiry, warn tenants
0 16 1 * * (10:00 CDT, 1st)/api/cron/partner-billingMonthly partner invoice generation

Additional manual-trigger endpoints:

  • /api/cron/ecosystem-billing — Ecosystem-wide billing aggregation
  • /api/cron/ecosystem-health — Health aggregation across ecosystem

Caracol — QStash Failed Stamp Queue

Caracol uses QStash for one critical job: retrying failed CFDI stamps.

EndpointScheduleAuth
/api/stamps/process-failed-queue*/15 * * * * (every 15 min)QStash HMAC signature

Retry logic:

  • Checks invoice status before stamping (dedup guard: skips stamped, defers stamping)
  • Supports ecosystem stamp retry (stored XML + credentials)
  • PAC retry NOT supported in queue — user must retry manually
  • 500ms delay between items to avoid overwhelming the PAC
  • Dead letter cleanup: /api/cron/cleanup-dead-letter (30-day retention)
  • Mancha sync retry: /api/cron/mancha-retry

La Hoja — Next.js Dispatcher

La Hoja (Next.js 15 App Router) uses Vercel Cron via app/api/cron/:

  • /api/cron/dispatcher — Main job dispatcher
  • /api/cron/reset-allocations — Reset inventory allocations
  • /api/cron/mancha-retry — Retry Mancha sync failures
  • /api/cron/cleanup-dead-letter — Clean Firestore dead letter entries

Note: La Hoja's dispatcher is also called by Colectiva's ecosystem heartbeat every 30 minutes.


Servilleta

  • /api/cron/check-subscriptions — Subscription status verification
  • /api/cron/expire-tasks — Expire stale tasks, suspend low-rating taskers

Note: Also triggered by Colectiva ecosystem heartbeat.


Baúl

EndpointSchedule (vercel.json)Purpose
/api/cron/generate-routes0 12 * * 1-5 (06:00 CDT, weekdays)Generate recurring delivery orders from schedules
/api/cron/referral-shares0 8 1 * * (02:00 CDT, 1st)Pay 3% monthly referral revenue shares

pg_cron keep-alive: every 3 days (migration 20260410000004_keep_alive.sql)


Ecosystem Control Plane API

URL: ecosystem.redbroomsoftware.comRepo: ecosystem-sdk/packages/api

ScheduleEndpointPurpose
0 6 * * * (06:00 UTC)/api/cron/health-checkHEAD all active apps, log latency, cleanup old data

What health-check does:

  1. Queries apps table for all active apps + their base_url
  2. Fires parallel HEAD requests (10s timeout each)
  3. Inserts results into health_checks (30-day retention)
  4. Logs down apps to console/Sentry
  5. Cleanup: rate_limit_events (7d), webhook_stats (90d)
  6. Aggregates per-app webhook availability stats

pg_cron keep-alive: every 3 days (migration 004_keep_alive.sql)


Supabase Keep-Alive Pattern

7 projects use the identical pg_cron keep-alive pattern to prevent free-tier suspension (7-day inactivity limit):

ProjectMigration File
Agora003_keep_alive.sql
Baúl20260410000004_keep_alive.sql
Patadas004_keep_alive.sql
La Hoja004_cron_jobs.sql
Plenura004_cron_jobs.sql
Ecosystem API004_keep_alive.sql
Camino20260102_setup_pg_cron.sql (includes keep-alive)

Pattern (identical across all):

sql
CREATE EXTENSION IF NOT EXISTS pg_cron;
GRANT USAGE ON SCHEMA cron TO postgres;

CREATE OR REPLACE FUNCTION keep_alive() RETURNS void LANGUAGE sql AS $$ SELECT 1; $$;
SELECT cron.schedule('keep-alive-ping', '0 12 */3 * *', 'SELECT keep_alive()');

-- Cleanup cron history weekly (prevents table bloat)
CREATE OR REPLACE FUNCTION cleanup_cron_history() RETURNS void LANGUAGE sql AS $$
    DELETE FROM cron.job_run_details WHERE end_time < NOW() - INTERVAL '7 days';
$$;
SELECT cron.schedule('cleanup-cron-history', '0 3 * * 0', 'SELECT cleanup_cron_history()');

When adding a new Supabase project: copy this migration. Run it after unpausing the project.


Dead Letter & Retry Infrastructure

Camino

  • webhook-retry cron job (in daily dispatcher) re-sends failed ecosystem webhooks with exponential backoff

Caracol

  • Failed stamp queue (QStash, every 15 min): retries failed CFDI stamps
  • Dead letter collection (failed_webhooks_dead): webhooks that exceeded max retries
  • Cleanup (/api/cron/cleanup-dead-letter): removes entries >30 days

La Hoja

  • Mancha sync failures → dead letter collection → /api/cron/cleanup-dead-letter

General Pattern

  1. First attempt: fire-and-forget with HMAC signature
  2. On failure: enqueue to retry collection/queue
  3. Retry with exponential backoff (max N attempts)
  4. On max retries: move to dead letter
  5. Periodic cleanup of dead letter entries

Authentication Patterns for Cron Endpoints

PatternUsed ByCode
CRON_SECRET timing-safe compareBaúl, Camino, Constanza, Caracol (non-QStash)timingSafeEqual(a, b)
QStash HMAC verificationColectiva, Caracol (stamp queue)receiver.verify({ signature, body })
Vercel auto-inject (no explicit check)Some internal endpointsVercel adds Authorization: Bearer $CRON_SECRET automatically

Rule: All cron endpoints must deny-by-default (return 401/500 if secret not configured). Never use == for secret comparison.


Infrastructure Map

DAILY BACKBONE
──────────────
08:00 CDT  Camino daily-dispatcher (18 jobs)
           ├── Marketing: emails, social, campaigns
           ├── Billing: webhook-retry, reconcile-orgs
           ├── AI: optimize-landings, optimize-prices
           └── Cleanup: audit-logs, oauth-codes, messages

EVERY 30 MIN
────────────
Colectiva ecosystem-heartbeat
           ├── → Agora keep-alive
           ├── → La Hoja dispatcher
           └── → Servilleta expire-tasks

EVERY 15 MIN
────────────
Caracol QStash → process-failed-stamp-queue

EVERY 6H (DB)
─────────────
Camino pg_cron → sync-constanza-invoices

06:00 UTC DAILY
────────────────
Ecosystem API → health-check (all apps HEAD)

MONTHLY (1st)
──────────────
Colectiva: revenue-recognition + distributions
Constanza: partner billing
Baúl: referral-shares
Camino: subscription expirations

EVERY 3 DAYS (DB)
──────────────────
7× pg_cron keep-alive (Agora, Baúl, Patadas, La Hoja, Plenura, Ecosystem API, Camino)

Known Gaps

GapImpactSuggested Fix
No centralized cron monitoring dashboardSilent failures invisibleAdd Sentry alerts on cron endpoint 5xx
Colectiva QStash registration is manual (register-qstash-schedules.mjs)Easy to driftAdd to deployment runbook
Camino pg_cron Bearer token is hardcoded in migrationRotation requires re-migrationMove to pg_net with env-injected secret or Vault
No circuit breaker on webhook retriesBad endpoint retried indefinitelyAdd max-age cutoff in webhook-retry cron
La Hoja / Servilleta / Agora depend on Colectiva heartbeatColectiva outage → 3 apps lose background jobsEach should have fallback vercel.json cron
Plenura has no vercel.jsonreminders and weekly-reports pg_cron jobs are DB-onlyCheck if pg_net is configured; add vercel.json if not

Red Broom Software Ecosystem