# Research Brief: Engage Platform

### 1. FUNCTIONALITY

The Engage platform processes operator webhooks, runs durable multi-step journeys, dispatches omnichannel communications, and integrates gamification and AI assistance into a unified engagement system [1-3].

*   **End-to-End Ingestion & Event Normalization**:
    *   **Edge Ingress**: Operator backends emit HTTP webhooks (deposits, bets, registrations, KYC updates) to per-tenant Cloudflare Workers for Platforms endpoints (`/t/{tenantId}/...`) as documented in `deployment-and-hosting.md` and `README.md` [1, 2, 4].
    *   **Signature Verification & Normalization**: Edge Workers verify incoming request signatures (HMAC SHA-256 or SHA3-256) and normalize raw payloads into the canonical `IGamingEvent` shape defined in `packages/shared/src/events.ts` (`AGENTS.md`, `README.md`, `igaming-engagement-platform-plan.md`) [2, 5-7].
    *   **Ingest API**: Normalized JSON is posted to `POST /ingest` on the Fastify API (`apps/api`) with bearer token authentication (`README.md`) [2, 8]. The API validates payloads with AJV, performs recipient upserts using a `COALESCE` no-null-overwrite strategy, dedupes events, and enqueues jobs into `pg-boss` and the analytics pipeline (`cio-feature-parity-analysis.md`, `README.md`) [2, 9, 10].
*   **Journeys & Workflow Engine**:
    *   Workflows run asynchronously via `pg-boss` queue consumers operating in worker processes (`apps/workers`) as specified in `AGENTS.md` and `README.md` [2, 5].
    *   Journeys are triggered by operator events, segment entries/exits, or gamification milestones (`README.md`, `engage-ai-concierge-spec.md`) [2, 11].
    *   Supported workflow nodes include delays (`time_delay`, `time_window`, `wait_until`), flow control (`true_false_branch`, `multi_split`, `random_cohort`, `exit`), data transformations (`create_update_person`, `set_journey_attributes`, `batch_update`, `send_event`), external integrations (`send_slack`, `run_llm`, `query_collection`), and channel dispatchers (`cio-workflow-editor-visual-spec-2026-07-06.md`, `NEXT_STEPS.md`) [12, 13].
*   **Omnichannel Delivery**:
    *   Unified channel adapters in `packages/channel-adapters` handle multi-channel dispatch (`AGENTS.md`, `README.md`) [5, 8]:
        *   **Email**: Native HTML block editor and rich-text rendering with multi-locale fallback (`AGENTS.md`, `cio-workflow-editor-visual-spec-2026-07-06.md`, `PROJECT_STATUS_2026-07-07.md`) [14-16].
        *   **SMS & WhatsApp**: Twilio integration supporting E.164 formatting, GSM/UCS-2 segment counting, approved WhatsApp `template_sid` variables, and 24-hour freeform messaging sessions (`README.md`) [8].
        *   **Push Notifications**: FCM and APNs dispatch matched by device token prefixes (`fcm:` / `apns:`) with automatic invalidation pruning (`README.md`) [8].
        *   **In-App Messages**: Real-time toasts, modals, overlays, and persistent message feeds via embedded client SDKs (`deployment-and-hosting.md`, `gamification-spec.md`) [17, 18].
        *   **Preferences & Webhooks**: Public subscriber preference endpoints (`GET/PUT /preferences/public`) and inbound Twilio webhook processing (`POST /webhooks/twilio/inbound`) for STOP/HELP keyword opt-outs (`README.md`) [8].
*   **Segments**:
    *   **Dynamic Segments**: Rule-based audiences evaluated via a custom Filter DSL over recipient traits and behavioral aggregates (`cio-feature-parity-analysis.md`, `NEXT_STEPS.md`) [19, 20]. Provides live SQL count estimation (`compileToSQL COUNT`) prior to saving, materializes membership, and recomputes on schedule or event triggers (`cio-feature-parity-analysis.md`) [19]. Segment entries and exits emit `segment_entered` and `segment_exited` events (`cio-feature-parity-analysis.md`) [19].
    *   **Static Segments**: Manual recipient lists or bulk CSV list imports (`cio-feature-parity-analysis.md`) [19].
*   **Gamification Module**:
    *   Gamification elements act directly as workflow triggers (e.g., `gam.level_up`, `gam.mission_complete`, `gam.streak_milestone`) and treatments inside the journey engine (`engage-ai-concierge-spec.md`, `gamification-spec.md`) [11, 18].
    *   **Currencies & Ledger**: Points/XP for progression and spendable Coins backed by an append-only audited ledger (`gam_ledger` and `gam_balances`) (`PROJECT_STATUS_2026-07-07.md`, `gamification-spec.md`) [21, 22].
    *   **Mechanics**: Levels/tiers (`gam_levels`), Missions/quests with Filter DSL criteria, return Streaks (`gam_streaks`), Leaderboards/tournaments (`gam_leaderboards`), Badges (`gam_recipient_badges`), Loot boxes, Gift shop, and Exclusive coin game access controls (`PROJECT_STATUS_2026-07-07.md`, `gamification-spec.md`) [22, 23].
    *   **Bonus Engine & RG Gates**: Abstract `RewardGrantAdapter` interface executes bonus grants to casino APIs, while every reward and send passes through strict Responsible Gambling (RG) suppression filters (cool-off, deposit limits, self-exclusion) (`PROJECT_STATUS_2026-07-07.md`, `gamification-spec.md`) [23-26].
*   **AI Concierge ("Engage")**:
    *   An in-product assistant that automates operator onboarding and program generation (`engage-ai-concierge-spec.md`, `PROJECT_STATUS_2026-07-07.md`) [27, 28].
    *   Features include Web Speech voice dictation/replies and text interaction, SSRF-safe operator website crawling (`POST /dashboard/ai/analyze-site`) to detect brand tokens and product mix, budget-aware journey graph generation (`POST /dashboard/ai/generate-journey` and `/propose-program`), auto-integration assistance, and an ongoing copilot mode with per-operator AI autonomy controls (`engage-ai-concierge-spec.md`, `PROJECT_STATUS_2026-07-07.md`) [27-29].

---

### 2. UI/UX DESIGN

The UI is built specifically for data-dense iGaming CRM operations using a dedicated, token-driven design system (`cursor-design-system-manual.md`) [30, 31].

*   **Design System (`packages/design-system` & `.cursor/rules/design-system.mdc`)**:
    *   **Tokens & Palette**: Configured in `tokens/tokens.css` and mapped to Tailwind utilities (`tokens/tailwind-preset.js`) (`cursor-design-system-manual.md`) [30, 31].
    *   **Color Palette**: Background (`--ds-bg: #0b0e14`), Surface (`--ds-surface: #141925`), Surface-2 (`--ds-surface-2: #1c2333`), Primary Text (`--ds-text: #e7ecf3`), Muted Text (`--ds-text-muted: #93a0b4`), Brand Accent (`--ds-accent: #e1062c`), Contrast (`#ffffff`), Success (`#14b87a`), Warning (`#f5a524`), Danger (`#ef4444`) (`cursor-design-system-manual.md`) [30].
    *   **Typography**: Base interface font `"Inter", system-ui, sans-serif` paired with display font `"Space Grotesk"` for headings (`cursor-design-system-manual.md`) [30].
    *   **Shape & Motion**: Border radius `--ds-radius: 10px`, elevation shadow `--ds-shadow`, and smooth transition curves `--ds-ease: cubic-bezier(.2,.8,.2,1)` (`cursor-design-system-manual.md`) [30].
    *   **Component Set**: Standardized primitives (`Button`, `Card`, `Input`, `Badge`, `Tabs`, `Drawer`, `NodeCard`) built strictly on tokens without inline hex codes or ad-hoc styles (`cursor-design-system-manual.md`) [31].
*   **Information Architecture & Dashboard Structure**:
    *   Next.js 14 admin dashboard (`apps/dashboard`) mapped to WorkOS Organization IDs for tenant context (`AGENTS.md`, `README.md`) [5, 32].
    *   Core navigation spans **Workflows**, **Customers (Recipients)**, **Segments**, **Snippets**, **Assets**, **Message Library**, **Activity Log**, **Collections**, **Concierge Onboarding / Program Review**, **Gamification Dashboard**, and **Settings** (`README.md`, `PROJECT_STATUS_2026-07-07.md`, `NEXT_STEPS.md`) [8, 20, 23, 28, 33].
*   **Key Screen Specifications**:
    *   **Workflow Builder**: Single vertical spine canvas over a dark dotted-grid background (`cio-workflow-editor-visual-spec-2026-07-06.md`) [34, 35]. Features a right-side "Add" palette categorized into Messages, Delays, Flow Control, and Data (`cio-workflow-editor-visual-spec-2026-07-06.md`) [12]. Hover edge teal "+" drop-zone bubbles, deep-linkable node URLs (`?nodeId=`), node status indicators, inline config drawer, and zoom toolbar (`cio-workflow-editor-visual-spec-2026-07-06.md`, `NEXT_STEPS.md`) [12, 35-37]. Includes a 3-mode content composer starting place chooser (Drag-and-drop Blocks, Rich Text, Code) with "Start from existing email" search and template grid (`cio-workflow-editor-visual-spec-2026-07-06.md`, `PROJECT_STATUS_2026-07-07.md`) [16, 38-40].
    *   **Customers (Recipients)**: Filterable and searchable table listing recipient attributes, email, segments, and status (`cio-feature-parity-analysis.md`, `PROJECT_STATUS_2026-07-07.md`) [9, 21]. Detail views show custom traits, segment memberships, and delivery history (`cio-feature-parity-analysis.md`) [9]. Includes manual creation form and a CSV importer with column mapping and downloadable sample template (`sample-customers.csv`) (`cio-feature-parity-analysis.md`, `PROJECT_STATUS_2026-07-07.md`) [9, 21].
    *   **Segments**: List view displaying dynamic vs. static types, live member counts, and updated timestamps (`cio-feature-parity-analysis.md`, `PROJECT_STATUS_2026-07-07.md`) [19, 21]. Dynamic segment creation uses the visual Filter DSL condition builder with live `compileToSQL COUNT` headcount estimation (`cio-feature-parity-analysis.md`) [19]. Static segments support manual member editing or CSV file upload (`cio-feature-parity-analysis.md`) [19].
    *   **Snippets**: Syntax-highlighted code editors for creating reusable Liquid/HTML/text content fragments (NOT simple key-value merge fields) (`cio-feature-parity-analysis.md`, `PROJECT_STATUS_2026-07-07.md`) [21, 41, 42]. Referenced as `{{ snippets.<name> }}` or `{{ snippets['<name>'] }}` with internal Liquid locale branching (`{% case recipient.locale %}`) and usage tracking (`cio-feature-parity-analysis.md`) [42].
    *   **Assets**: Image grid and folder hierarchy backed by Cloudflare R2 object storage with pre-signed URL uploads (`cio-feature-parity-analysis.md`, `PROJECT_STATUS_2026-07-07.md`) [21, 43]. Folder operations (create/rename/delete/move), file search, and an integrated asset picker component for message content editors (`cio-feature-parity-analysis.md`, `PROJECT_STATUS_2026-07-07.md`) [21, 43].
    *   **Message Library**: Unified catalog listing all reusable templates across email, in-app, push, SMS, and WhatsApp (`cio-feature-parity-analysis.md`, `PROJECT_STATUS_2026-07-07.md`) [21, 44]. Searchable and filterable by tag or channel, selectable directly inside workflow message nodes (`cio-feature-parity-analysis.md`) [44].
    *   **Activity Log**: Chronological feed unifying `ingested_events`, `delivery_records`, `journey_events`, segment entry/exits, and reward issuances (`cio-feature-parity-analysis.md`, `PROJECT_STATUS_2026-07-07.md`) [21, 45]. Filterable by recipient ID, channel, event type, and date range (`cio-feature-parity-analysis.md`) [45].
*   **UX Principles**:
    *   **Reuse Existing Primitives**: Avoid duplicating recipient stores, condition builders, or render pipelines (`AGENTS.md`, `NEXT_STEPS.md`) [46, 47].
    *   **Client-Side Draft Safety**: Unsaved canvas changes exist in client-side draft state prior to explicit persistence (`cio-workflow-editor-visual-spec-2026-07-06.md`) [12].
    *   **High Information Density**: Dark, compact aesthetic tailored for CRM operators (`cursor-design-system-manual.md`) [30, 31].

---

### 3. SYSTEM ARCHITECTURE

Engage is designed as an API-first monorepo using database-enforced multi-tenancy and edge ingestion (`AGENTS.md`, `architecture-and-scalability.md`, `README.md`) [5, 48, 49].

*   **Tech Stack & Monorepo Layout**:
    *   Turborepo monorepo managed with `pnpm 9`, Node.js 20+, and strict TypeScript (`AGENTS.md`, `README.md`) [5, 50, 51].
    *   **Repository Structure** (`AGENTS.md`, `README.md`) [5, 49]:
        *   `apps/api`: Fastify HTTP REST API (`:3001`) handling ingestion, tenants, health, and CRUD.
        *   `apps/workers`: `pg-boss` queue consumers running the journey engine.
        *   `apps/dashboard`: Next.js 14 admin UI (`:8233`) using Tailwind CSS and `@engage/design-system`.
        *   `apps/edge`: Cloudflare Workers for Platforms (dispatch worker and per-tenant adapters).
        *   `packages/shared`: Canonical `IGamingEvent` types and JSON Schemas.
        *   `packages/channel-adapters`: Unified send interfaces for email, SMS, WhatsApp, push, and in-app.
        *   `db/`: Postgres schema (`schema.sql`), migration scripts (`migrate.mjs`), and role grants.
*   **Data Flow**:
    1. **Ingress**: Operator platform posts webhooks to per-tenant Cloudflare edge URL (`/t/{tenantId}/...`) (`architecture-and-scalability.md`, `README.md`) [1, 2].
    2. **Edge Normalization**: Per-tenant Worker for Platforms worker verifies HMAC SHA-256 / SHA3-256 signatures and normalizes payload into `IGamingEvent` (`architecture-and-scalability.md`, `deployment-and-hosting.md`, `README.md`) [1, 2, 4].
    3. **Ingest API**: Edge Worker forwards normalized JSON to Fastify API `POST /ingest` with bearer token (`architecture-and-scalability.md`, `README.md`) [1, 2, 8]. API validates via AJV, COALESCE upserts recipient, dedupes, and writes to `ingested_events` (`cio-feature-parity-analysis.md`, `README.md`) [2, 9].
    4. **Execution Queue**: Ingested events trigger workflow jobs written to Postgres-backed `pg-boss` queue (`architecture-and-scalability.md`, `README.md`) [1, 2].
    5. **Worker Execution**: Worker processes pick up `pg-boss` jobs, evaluate journey graph nodes and Filter DSL logic, and invoke `channel-adapters` (`architecture-and-scalability.md`, `AGENTS.md`) [1, 5].
    6. **Delivery & Audit**: Channel adapters dispatch messages via external APIs and log status to `delivery_records` and `journey_events` (`architecture-and-scalability.md`, `cio-feature-parity-analysis.md`) [1, 45].
*   **Multi-Tenancy & Row-Level Security (RLS)**:
    *   Enforced strictly at the Postgres database level using Row-Level Security (RLS) as a non-negotiable invariant (`AGENTS.md`, `architecture-and-scalability.md`) [6, 48].
    *   **Database Roles**: `engage` (owner/migrations role with `BYPASSRLS`) and `engage_app` (restricted runtime role with `NOBYPASSRLS`) (`AGENTS.md`, `architecture-and-scalability.md`, `README.md`) [32, 48, 52]. The API runtime connects exclusively as `engage_app` (`architecture-and-scalability.md`, `README.md`) [32, 48].
    *   **Isolation Policy**: Every tenant table enables and forces RLS (`ENABLE` and `FORCE ROW LEVEL SECURITY`) with policy `USING (tenant_id = current_tenant_id())` (`architecture-and-scalability.md`) [48, 53].
    *   **Session Context**: `current_tenant_id()` reads `app.tenant_id`, set transaction-locally via `set_config('app.tenant_id', $1, true)` in `withTenant()` (`architecture-and-scalability.md`, `README.md`) [32, 48]. Client-supplied tenant IDs are never trusted (`AGENTS.md`, `architecture-and-scalability.md`) [6, 48].
*   **Deployment Topology**:
    *   **Edge Layer**: Cloudflare Workers for Platforms (stateless dispatch and per-tenant adapter Workers) (`architecture-and-scalability.md`, `deployment-and-hosting.md`) [1, 54].
    *   **Core API**: Fastify application running in containerized Fly.io deployment (`deployment-and-hosting.md`, `PROJECT_STATUS_2026-07-07.md`) [54, 55].
    *   **Workflow Engine**: Node.js process running `pg-boss` hosted on Railway or Fly.io (`deployment-and-hosting.md`, `PROJECT_STATUS_2026-07-07.md`) [54, 55].
    *   **Database**: Neon serverless Postgres (`eu-central-1`) utilizing process-specific endpoint routing (`deployment-and-hosting.md`, `PROJECT_STATUS_2026-07-07.md`) [55, 56]:
        *   `DATABASE_URL`: Neon **Transaction Pooler** endpoint (`...-pooler....neon.tech`) for short API request/response cycles (`deployment-and-hosting.md`) [56].
        *   `WORKER_DATABASE_URL`: Neon **Direct/Session** endpoint (`ep-....neon.tech`) for `pg-boss` workers (`LISTEN/NOTIFY` and advisory locks) (`deployment-and-hosting.md`) [56].
        *   `DATABASE_MIGRATION_URL`: Direct owner connection for schema migrations (`deployment-and-hosting.md`) [56].
    *   **Dashboard**: Next.js deployed on Vercel via Git auto-deploy integration (`deployment-and-hosting.md`, `PROJECT_STATUS_2026-07-07.md`) [54, 55, 57].
    *   **Object Storage**: Cloudflare R2 (S3-compatible API with zero egress fees) for asset media (`deployment-and-hosting.md`, `PROJECT_STATUS_2026-07-07.md`) [54, 58, 59].
    *   **SDK Delivery**: Single versioned bundle (`engage.v1.js`) served via Cloudflare CDN (`deployment-and-hosting.md`, `PROJECT_STATUS_2026-07-07.md`) [17, 54, 59].
    *   **Analytics**: Managed ClickHouse Cloud for high-volume event telemetry storage (`architecture-and-scalability.md`, `deployment-and-hosting.md`) [1, 54, 60].

---

### 4. IMPROVEMENT OPPORTUNITIES & MODERNIZATION

Documented technical gap analyses (`cio-workflow-editor-visual-spec-2026-07-06.md`, `cio-feature-parity-analysis.md`), project status tracking (`PROJECT_STATUS_2026-07-07.md`), and runbook roadmaps (`NEXT_STEPS.md`) highlight key UI/UX and architectural modernization targets:

*   **SDK Packaging & CDN Delivery**:
    *   *Current Gap*: Deprecated split scripts (`engage-inapp.js`, `engage-push.js`) are served separately (`deployment-and-hosting.md`) [60].
    *   *Modernization*: Finalize the build pipeline (`pnpm prepare:platform-sdk`) to bundle in-app feeds, push handlers, and gamification listeners into a single versioned `engage.v1.js` script served via Cloudflare CDN (`deployment-and-hosting.md`, `PROJECT_STATUS_2026-07-07.md`) [17, 59].
*   **Sample Event Data Inspector for Composer**:
    *   *Current Gap*: The workflow message composer currently relies on static preview context (`cio-workflow-editor-visual-spec-2026-07-06.md`) [36].
    *   *Modernization*: Replicate Customer.io's left-side sample data panel in the composer (`cio-workflow-editor-visual-spec-2026-07-06.md`) [36, 61]. This allows CRM managers to search real recipient profiles and select actual trigger event payloads (e.g., `player_balance` events with currency and amount) to evaluate Liquid syntax and conditional blocks dynamically against real player data (`cio-workflow-editor-visual-spec-2026-07-06.md`) [61].
*   **Graphical Player Journey Lineage & Debugger**:
    *   *Current Gap*: The Activity Log presents execution data as flat tabular delivery records and journey events without visual graph pathing (`cio-feature-parity-analysis.md`, `PROJECT_STATUS_2026-07-07.md`) [45, 62].
    *   *Modernization*: Build a graphical player journey lineage view that overlays individual player traversals directly onto the visual workflow canvas, highlighting executed node paths, evaluation branch outcomes, and delay timers (`PROJECT_STATUS_2026-07-07.md`) [62].
*   **Localization & Translation Management UI**:
    *   *Current Gap*: Multi-locale messaging currently relies on manual inline Liquid logic (`{% case recipient.locale %}`) or raw JSON `content_by_locale` blocks (`cio-feature-parity-analysis.md`, `PROJECT_STATUS_2026-07-07.md`) [42, 62].
    *   *Modernization*: Create a dedicated translation management dashboard UI exposing side-by-side locale editing tabs (`+ Add Language`), string extraction, and translation completion indicators across all channels (`cio-workflow-editor-visual-spec-2026-07-06.md`, `PROJECT_STATUS_2026-07-07.md`) [61, 62].
*   **AI Autonomy Dashboard Controls**:
    *   *Current Gap*: AI Concierge journey generation backend is fully built (`G1–G6`), but the autonomy toggle UI remains deferred (`engage-ai-concierge-spec.md`, `PROJECT_STATUS_2026-07-07.md`) [28, 29, 62].
    *   *Modernization*: Implement the deferred dashboard control panel enabling operators to switch per-journey AI autonomy settings between "propose for human approval" (default) and "auto-activate within guardrails" (bounded by budget and RG caps) (`engage-ai-concierge-spec.md`, `PROJECT_STATUS_2026-07-07.md`) [29, 62].
*   **Incremental Real-Time Segmentation**:
    *   *Current Gap*: Dynamic segments recompute on filter updates or cron schedules using batch `compileToSQL` queries (`cio-feature-parity-analysis.md`, `PROJECT_STATUS_2026-07-07.md`) [19, 63].
    *   *Modernization*: Modernize segmentation to incremental real-time membership updating driven by event stream processing (e.g., ClickHouse materialized view rollups), eliminating full-table re-evaluation overhead (`PROJECT_STATUS_2026-07-07.md`, `igaming-engagement-platform-plan.md`) [63, 64].
*   **Leaderboard & Histogram Performance Rollups**:
    *   *Current Gap*: Leaderboards (`F6`) and engagement send-time histograms (`F10`) execute on-demand queries across raw event tables without background refresh schedules (`PROJECT_STATUS_2026-07-07.md`) [63].
    *   *Modernization*: Introduce scheduled background rollup cron jobs and materialized view caching for leaderboards and send-time distribution histograms to maintain sub-second UI response times under high operator event volume (`PROJECT_STATUS_2026-07-07.md`) [55, 63].
*   **Operator CRM Manager Mobile Application**:
    *   *Current Gap*: Administration is currently desktop-only (`AGENTS.md`, `README.md`) [5, 49].
    *   *Modernization*: Execute planned Phase H to build a native mobile app for operator CRM managers, enabling campaign health monitoring, real-time alert notifications, and voice-assisted copilot interactions on mobile devices (`engage-ai-concierge-spec.md`, `PROJECT_STATUS_2026-07-07.md`, `NEXT_STEPS.md`) [29, 65, 66].

---

💡 **Next Steps**: Would you like me to turn this research brief into an interactive slide deck or generate a tailored executive report document in your Studio panel?