RBS Ecosystem Infrastructure
Scheduled jobs, health checks, dead-letter queues, and background coordination across all 19 apps. Last Audit: April 9, 2026 — 70+ infrastructure endpoints across 3 scheduler types.
Scheduling Architecture
The ecosystem uses three distinct scheduling layers, chosen by constraint:
| Layer | Technology | Who Uses It | When to Use |
|---|---|---|---|
| App cron | Vercel Cron → HTTP endpoint | Camino, Constanza, Ecosystem API | Primary jobs that need Vercel's serverless timeout and env vars |
| Queue cron | Upstash QStash | Colectiva, Caracol | Firebase apps (no Supabase pg_cron), need signed delivery and retry |
| DB cron | Supabase pg_cron | Camino, Agora, Baúl, Patadas, La Hoja, Plenura, Ecosystem API | Database-internal jobs, keep-alives; run inside the DB network |
All HTTP cron endpoints use timingSafeEqual comparison on a CRON_SECRET or equivalent Bearer token. QStash endpoints additionally verify Upstash HMAC signatures via @upstash/qstash Receiver.
Camino — The Ecosystem's Primary Dispatcher
Camino runs the most critical daily batch via a single Vercel Cron trigger that fans out to 18 sub-jobs sequentially:
Schedule: 0 14 * * * (08:00 CDT) — POST /api/cron/daily-dispatcher
| # | Sub-job | Purpose |
|---|---|---|
| 1 | expire-trials | Mark overdue trial subscriptions as expired |
| 2 | trial-expiry-reminders | Email at 7d / 3d / 1d / expired |
| 3 | nurture-emails | Drip email sequences for CRM leads |
| 4 | publish-scheduled-posts | Release scheduled social media posts |
| 5 | refresh-tokens | Rotate OAuth + API tokens before expiry |
| 6 | optimize-landings | AI A/B test landing pages; promote winners |
| 7 | optimize-prices | Geo + demand pricing updates |
| 8 | campaigns/enroll | Auto-enroll leads matching campaign rules |
| 9 | sync-comments | Fetch social comments + AI auto-response |
| 10 | social-listening-scan | Brand mention scanning across platforms |
| 11 | usage-alerts | Fire at 80% and 100% of tier limits |
| 12 | upsell-alerts | Cross-sell: missing Constanza, idle leads |
| 13 | webhook-retry | Re-send failed ecosystem webhooks (exponential backoff) |
| 14 | reconcile-orgs | Retry Colectiva registration for orphaned orgs |
| 15 | sync-metrics | Aggregate and push platform metrics |
| 16 | cleanup-audit-logs | Delete logs older than retention policy |
| 17 | cleanup-oauth-codes | Expire OAuth codes, revoked tokens, idempotency keys |
| 18 | process-scheduled-messages | Dispatch queued SMS/Email messages |
Camino also has two pg_cron jobs that run inside its Supabase DB (via pg_net HTTP calls):
| Job | Schedule | Endpoint |
|---|---|---|
sync-constanza-invoices | Every 6 hours | POST /api/cron/sync-constanza-invoices |
voice-outbound | Every 15 min, 09:00–19:00 Mon–Sat MX | POST /api/cron/voice-outbound |
Colectiva — QStash Scheduler (15 Jobs)
Colectiva uses Upstash QStash exclusively (Firebase app — no pg_cron). Jobs are registered via scripts/cron/register-qstash-schedules.mjs. All endpoints verify QStash HMAC signatures.
Revenue & Billing (Daily)
| Endpoint | Schedule | Purpose |
|---|---|---|
/api/cron/subscription-renewals | 0 8 * * * | Renew subscriptions, retry failed payments |
/api/cron/earnings-reconciliation | 0 12 * * * | Reconcile earnings, fire threshold alerts |
/api/cron/sync-polizas | 0 16 * * * | Sync queued polizas to Constanza |
/api/cron/offering-expiration | 0 14 * * * | Auto-close sold-out/expired CPI offerings |
/api/cron/deal-billing | Daily | Deal-specific billing operations |
/api/cron/ai-invoices | Daily | AI usage invoice batch processing |
AI & Intelligence (Daily)
| Endpoint | Schedule | Purpose |
|---|---|---|
/api/cron/daily-brain | Daily | Brain briefing generation |
/api/cron/daily-intelligence | Daily | Intelligence report generation |
/api/cron/process-approved-actions | Daily | Execute Brain-approved actions |
Ecosystem Coordination
| Endpoint | Schedule | Purpose |
|---|---|---|
/api/cron/ecosystem-heartbeat | */30 * * * * | Every 30 min — health + fan-out to Agora, La Hoja, Servilleta |
/api/cron/ecosystem-sync | 0 10 * * 0 | Weekly Sunday — ecosystem data snapshot for AI |
Monthly
| Endpoint | Schedule | Purpose |
|---|---|---|
/api/cron/monthly-first | 0 9 1 * * | Revenue recognition + earnings poliza + distributions |
/api/cron/monthly-fifth | 0 9 5 * * | 5th-day monthly operations |
/api/cron/revenue-reconciliation | Monthly | Revenue reconciliation |
/api/cron/earnings-poliza | Monthly 1st | Earnings poliza generation |
The Ecosystem Heartbeat
/api/cron/ecosystem-heartbeat is the most important infrastructure job nobody talks about. Running every 30 minutes, it:
- Pings Colectiva's own health
- Fan-out to 3 apps that lack their own Vercel Cron: calls Agora keep-alive, La Hoja dispatcher, and Servilleta expire-tasks
- Logs results to
health_checkstable
This means Agora, La Hoja, and Servilleta's background work is triggered by Colectiva — not by their own vercel.json.
Constanza — Certificate & Billing Crons
vercel.json crons:
| Schedule | Endpoint | Purpose |
|---|---|---|
0 15 * * * (09:00 CDT) | /api/cron/check-certificate-expiry | Monitor X.509 CSD certificate expiry, warn tenants |
0 16 1 * * (10:00 CDT, 1st) | /api/cron/partner-billing | Monthly partner invoice generation |
Additional manual-trigger endpoints:
/api/cron/ecosystem-billing— Ecosystem-wide billing aggregation/api/cron/ecosystem-health— Health aggregation across ecosystem
Caracol — QStash Failed Stamp Queue
Caracol uses QStash for one critical job: retrying failed CFDI stamps.
| Endpoint | Schedule | Auth |
|---|---|---|
/api/stamps/process-failed-queue | */15 * * * * (every 15 min) | QStash HMAC signature |
Retry logic:
- Checks invoice status before stamping (dedup guard: skips
stamped, defersstamping) - Supports ecosystem stamp retry (stored XML + credentials)
- PAC retry NOT supported in queue — user must retry manually
- 500ms delay between items to avoid overwhelming the PAC
- Dead letter cleanup:
/api/cron/cleanup-dead-letter(30-day retention) - Mancha sync retry:
/api/cron/mancha-retry
La Hoja — Next.js Dispatcher
La Hoja (Next.js 15 App Router) uses Vercel Cron via app/api/cron/:
/api/cron/dispatcher— Main job dispatcher/api/cron/reset-allocations— Reset inventory allocations/api/cron/mancha-retry— Retry Mancha sync failures/api/cron/cleanup-dead-letter— Clean Firestore dead letter entries
Note: La Hoja's dispatcher is also called by Colectiva's ecosystem heartbeat every 30 minutes.
Servilleta
/api/cron/check-subscriptions— Subscription status verification/api/cron/expire-tasks— Expire stale tasks, suspend low-rating taskers
Note: Also triggered by Colectiva ecosystem heartbeat.
Baúl
| Endpoint | Schedule (vercel.json) | Purpose |
|---|---|---|
/api/cron/generate-routes | 0 12 * * 1-5 (06:00 CDT, weekdays) | Generate recurring delivery orders from schedules |
/api/cron/referral-shares | 0 8 1 * * (02:00 CDT, 1st) | Pay 3% monthly referral revenue shares |
pg_cron keep-alive: every 3 days (migration 20260410000004_keep_alive.sql)
Ecosystem Control Plane API
URL: ecosystem.redbroomsoftware.comRepo: ecosystem-sdk/packages/api
| Schedule | Endpoint | Purpose |
|---|---|---|
0 6 * * * (06:00 UTC) | /api/cron/health-check | HEAD all active apps, log latency, cleanup old data |
What health-check does:
- Queries
appstable for all active apps + theirbase_url - Fires parallel HEAD requests (10s timeout each)
- Inserts results into
health_checks(30-day retention) - Logs down apps to console/Sentry
- Cleanup: rate_limit_events (7d), webhook_stats (90d)
- Aggregates per-app webhook availability stats
pg_cron keep-alive: every 3 days (migration 004_keep_alive.sql)
Supabase Keep-Alive Pattern
7 projects use the identical pg_cron keep-alive pattern to prevent free-tier suspension (7-day inactivity limit):
| Project | Migration File |
|---|---|
| Agora | 003_keep_alive.sql |
| Baúl | 20260410000004_keep_alive.sql |
| Patadas | 004_keep_alive.sql |
| La Hoja | 004_cron_jobs.sql |
| Plenura | 004_cron_jobs.sql |
| Ecosystem API | 004_keep_alive.sql |
| Camino | 20260102_setup_pg_cron.sql (includes keep-alive) |
Pattern (identical across all):
CREATE EXTENSION IF NOT EXISTS pg_cron;
GRANT USAGE ON SCHEMA cron TO postgres;
CREATE OR REPLACE FUNCTION keep_alive() RETURNS void LANGUAGE sql AS $$ SELECT 1; $$;
SELECT cron.schedule('keep-alive-ping', '0 12 */3 * *', 'SELECT keep_alive()');
-- Cleanup cron history weekly (prevents table bloat)
CREATE OR REPLACE FUNCTION cleanup_cron_history() RETURNS void LANGUAGE sql AS $$
DELETE FROM cron.job_run_details WHERE end_time < NOW() - INTERVAL '7 days';
$$;
SELECT cron.schedule('cleanup-cron-history', '0 3 * * 0', 'SELECT cleanup_cron_history()');When adding a new Supabase project: copy this migration. Run it after unpausing the project.
Dead Letter & Retry Infrastructure
Camino
webhook-retrycron job (in daily dispatcher) re-sends failed ecosystem webhooks with exponential backoff
Caracol
- Failed stamp queue (QStash, every 15 min): retries failed CFDI stamps
- Dead letter collection (
failed_webhooks_dead): webhooks that exceeded max retries - Cleanup (
/api/cron/cleanup-dead-letter): removes entries >30 days
La Hoja
- Mancha sync failures → dead letter collection →
/api/cron/cleanup-dead-letter
General Pattern
- First attempt: fire-and-forget with HMAC signature
- On failure: enqueue to retry collection/queue
- Retry with exponential backoff (max N attempts)
- On max retries: move to dead letter
- Periodic cleanup of dead letter entries
Authentication Patterns for Cron Endpoints
| Pattern | Used By | Code |
|---|---|---|
CRON_SECRET timing-safe compare | Baúl, Camino, Constanza, Caracol (non-QStash) | timingSafeEqual(a, b) |
| QStash HMAC verification | Colectiva, Caracol (stamp queue) | receiver.verify({ signature, body }) |
| Vercel auto-inject (no explicit check) | Some internal endpoints | Vercel adds Authorization: Bearer $CRON_SECRET automatically |
Rule: All cron endpoints must deny-by-default (return 401/500 if secret not configured). Never use == for secret comparison.
Infrastructure Map
DAILY BACKBONE
──────────────
08:00 CDT Camino daily-dispatcher (18 jobs)
├── Marketing: emails, social, campaigns
├── Billing: webhook-retry, reconcile-orgs
├── AI: optimize-landings, optimize-prices
└── Cleanup: audit-logs, oauth-codes, messages
EVERY 30 MIN
────────────
Colectiva ecosystem-heartbeat
├── → Agora keep-alive
├── → La Hoja dispatcher
└── → Servilleta expire-tasks
EVERY 15 MIN
────────────
Caracol QStash → process-failed-stamp-queue
EVERY 6H (DB)
─────────────
Camino pg_cron → sync-constanza-invoices
06:00 UTC DAILY
────────────────
Ecosystem API → health-check (all apps HEAD)
MONTHLY (1st)
──────────────
Colectiva: revenue-recognition + distributions
Constanza: partner billing
Baúl: referral-shares
Camino: subscription expirations
EVERY 3 DAYS (DB)
──────────────────
7× pg_cron keep-alive (Agora, Baúl, Patadas, La Hoja, Plenura, Ecosystem API, Camino)Known Gaps
| Gap | Impact | Suggested Fix |
|---|---|---|
| No centralized cron monitoring dashboard | Silent failures invisible | Add Sentry alerts on cron endpoint 5xx |
Colectiva QStash registration is manual (register-qstash-schedules.mjs) | Easy to drift | Add to deployment runbook |
| Camino pg_cron Bearer token is hardcoded in migration | Rotation requires re-migration | Move to pg_net with env-injected secret or Vault |
| No circuit breaker on webhook retries | Bad endpoint retried indefinitely | Add max-age cutoff in webhook-retry cron |
| La Hoja / Servilleta / Agora depend on Colectiva heartbeat | Colectiva outage → 3 apps lose background jobs | Each should have fallback vercel.json cron |
| Plenura has no vercel.json | reminders and weekly-reports pg_cron jobs are DB-only | Check if pg_net is configured; add vercel.json if not |