Webhooks and realtime

Phase 2 delivers realtime triage via Gmail Pub/Sub push → Corsair → /api/webhooks → AI classify/embed → Pusher UI updates.

Prerequisites

1. Gmail API enabled in your Google Cloud project (same project as OAuth)
2. At least one AI provider key (OPENAI_API_KEY and/or GOOGLE_GENERATIVE_AI_API_KEY)
3. Pusher Channels (optional but recommended) — dashboard.pusher.com
4. Migrations applied: bun run db:migrate

1. Create Pub/Sub topic (GCP)

gcloud config set project YOUR_PROJECT_ID
gcloud services enable pubsub.googleapis.com
gcloud pubsub topics create gmail-inbox
gcloud pubsub topics add-iam-policy-binding gmail-inbox \
  --member=serviceAccount:gmail-api-push@system.gserviceaccount.com \
  --role=roles/pubsub.publisher

Set in env:

GMAIL_PUBSUB_TOPIC=projects/YOUR_PROJECT_ID/topics/gmail-inbox

Run bun run corsair:setup to persist the topic on the Gmail integration.

2. Webhook URL

Corsair delivers verified Gmail events to:

{APP_URL}/api/webhooks?tenantId={userId}

Local ngrok example:

ngrok http 3000
# Set APP_URL=https://abc123.ngrok-free.app in .env.local
# Restart bun dev

Your tenant ID is your Better Auth user ID (see users table after sign-in).

3. Pub/Sub push subscription

Create a push subscription pointing at your webhook:

gcloud pubsub subscriptions create gmail-inbox-push \
  --project=YOUR_PROJECT_ID \
  --topic=gmail-inbox \
  --push-endpoint="https://YOUR_HOST/api/webhooks?tenantId=YOUR_USER_ID"

Replace YOUR_HOST and YOUR_USER_ID appropriately. For production, use your Vercel domain and each user's ID requires their own subscription — or use a shared routing pattern if your deployment supports it.

4. Gmail users.watch

After OAuth connect, register the mailbox for push notifications:

bun run gmail:watch

Or via API:

curl -X POST "https://gmail.googleapis.com/gmail/v1/users/me/watch" \
  -H "Authorization: Bearer ACCESS_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{"topicName":"projects/YOUR_PROJECT/topics/gmail-inbox","labelIds":["INBOX"]}'

Gmail watches expire after ~7 days — renew periodically.

5. Pusher configuration

Server env:

PUSHER_APP_ID=
PUSHER_KEY=
PUSHER_SECRET=
PUSHER_CLUSTER=us2

Client env (must match):

NEXT_PUBLIC_PUSHER_KEY=
NEXT_PUBLIC_PUSHER_CLUSTER=us2

Without Pusher, the inbox polls every 5 seconds — functional but not instant.

Processing pipeline

POST /api/webhooks?tenantId=...
  → Verify x-corsair-signature
  → Corsair processWebhook
  → If gmail.messageChanged:
      → handleGmailMessageChanged (async)
      → Fetch thread from cache
      → AI classify + embed
      → Upsert classifications
      → Pusher trigger → browser
  → Return 200 (Pub/Sub ack)

Failures during async classification are logged; verified webhooks still return 200 to prevent Pub/Sub retry storms.

Verify end-to-end

1. Sign in and complete Connect Google
2. Open /inbox — immediate index classifies latest threads; full INBOX index continues in background
3. Send yourself a test email — lane should update within ~30s
4. Press / for semantic search after backfill completes
5. Check webhook_logs table for delivery debugging

Environment reference

Related

• Deploy to production — production Pub/Sub setup
• API routes — webhooks