feat: multi-step cron — check-research + ingest refactor + pg_cron

[codercatdev]

Summary

Splits the blocking 10+ minute NotebookLM research step into a fire-and-forget + polling architecture, matching the existing check-renders pattern. All cron schedules defined in Supabase pg_cron migration.

Replaces #603 (which had stale files from other branches in the diff).

New Status Flow

ingest (daily 10:00 UTC) → creates doc as "researching" (returns in ~15s)
check-research (every 5min) → polls NotebookLM → enriches script → "script_ready"
check-renders (every 5min) → audio_gen → rendering → video_gen → uploading → published

When ENABLE_NOTEBOOKLM_RESEARCH is NOT set, ingest goes straight to "script_ready" — no change to existing behavior.

Files Changed (4 files only — clean diff)

NEW: app/api/cron/check-research/route.ts (690 lines)

Polls NotebookLM research status for docs in "researching" state:

1. Imports research sources into notebook
2. Generates 5 infographics (architecture, comparison, workflow, timeline, pros/cons)
3. Waits for infographic completion
4. Gets notebook summary + infographic URLs
5. Re-generates enriched Gemini script with research data
6. Runs Claude critic pass
7. Updates Sanity doc to "script_ready"

Patterns from check-renders:

• ✅ CRON_SECRET bearer auth
• ✅ fetchCache = 'force-no-store' + maxDuration = 60
• ✅ Stuck detection (>30min in "researching" → flagged)
• ✅ Idempotent — safe to run on overlap

MODIFIED: app/api/cron/ingest/route.ts

• Removed blocking conductResearch() call (was 10+ minutes)
• Creates notebook + starts research in ~15s (fire-and-forget)
• Stores researchNotebookId and researchTaskId on Sanity doc
• Status: "researching" when NotebookLM enabled, "script_ready" when not

MODIFIED: sanity/schemas/documents/automatedVideo.ts

• Added "researching" to pipeline status options
• Added researchNotebookId, researchTaskId fields (hidden)
• Added trendScore, trendSources fields

NEW: supabase/migrations/002_cron_schedules.sql

Job	Schedule	Route
ingest-daily	0 10 * * *	/api/cron/ingest
check-research	*/5 * * * *	/api/cron/check-research
check-renders	*/5 * * * *	/api/cron/check-renders
sponsor-outreach	0 9 * * 1,4	/api/cron/sponsor-outreach

Prerequisites: Set app.site_url and app.cron_secret as Supabase config vars.

[vercel]

The latest updates on your projects. Learn more about Vercel for GitHub.

Project	Deployment	Actions	Updated (UTC)
codingcat-dev	Canceled		Mar 5, 2026 4:09am

[codercatdev]

✅ Re-review: All 4 blockers fixed

Verified commit ebef51c:

1. researchData field — added to schema as type: 'text', hidden: true ✅
2. after() pattern — Phase 3 uses after(async () => processResearchingDoc(...)) with error handling that flags failed docs. maxDuration = 300. Returns 200 before processing. ✅
3. CRON_SECRET fail-closed — Both check-research AND ingest now check !cronSecret → 503 before comparison. ✅
4. Stuck detection independent — Phase 1 queries, Phase 2 flags stuck docs (no NotebookLM needed), Phase 3 only calls initAuth() if active docs exist. ✅

Minor fixes also confirmed: _updatedAt for stuck detection, schema description cleaned.

Architecture is solid. Merging to dev.