crawler/plan.md

Dashboard Separation Plan

Separating the dashboard from the monolithic app/ into standalone dashboard/ service.

Overview

The dashboard is currently tightly coupled to the Crawler struct in app/. All handlers are methods on *Crawler. We need to:

1. Create a Dashboard struct in dashboard/ with *shared.DB
2. Move each file, converting methods from (c *Crawler) to (d *Dashboard)
3. Commit after each file move
4. Update imports and dependencies as needed

File Migration Checklist

Phase 1: Core Dashboard

• dashboard.go (266 lines) - Stats types and calculation
• templates.go (553 lines) - HTML templates and handlers
• static/dashboard.css - Stylesheet
• static/dashboard.js - JavaScript

Phase 2: OAuth

• oauth.go (287 lines) - OAuth config and manager
• oauth_session.go (352 lines) - Session management
• oauth_middleware.go (126 lines) - Auth middleware
• oauth_handlers.go (521 lines) - OAuth HTTP handlers

Phase 3: API Handlers

• api_search.go (311 lines) - Search API
• api_feeds.go (481 lines) - Feed API handlers
• api_publish.go (1031 lines) - Publish API handlers
• api_domains.go (2068 lines) - Domain API handlers

Phase 4: Routes & Main

• routes.go (216 lines) - Route registration
• Create main.go - Entry point for dashboard service

Phase 5: Cleanup

• Remove migrated code from app/
• Update app/main.go to remove dashboard startup
• Test both services independently

Progress Log

---

Last updated: 2026-02-02