- Feed details now expand inline instead of navigating to new page
- Add TLD section headers with domains sorted by TLD then name
- Add TLD filter button to show/hide domain sections by TLD
- Feed status behavior: pass creates account, hold crawls only, skip stops, drop cleans up
- Auto-follow new accounts from directory account (1440.news)
- Fix handle derivation (removed duplicate .1440.news suffix)
- Increase domain import batch size to 100k
- Various bug fixes for account creation and profile updates
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
When setting publish_status to deny, now performs complete cleanup:
- Deletes all posts from the feed account on PDS
- Deletes the PDS account itself
- Clears published_at timestamps on all items
- Clears the publish_account field on the feed
Added Publisher methods:
- DeleteAllPosts: lists and deletes all posts from an account
- DeleteRecord: deletes a single AT Protocol record
- DeleteAccount: deletes account via PDS admin API
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Added ability to toggle between alphabetical and feed count sorting when
viewing domains under a TLD. Includes UI toggle in breadcrumb area and
backend support for the sort parameter.
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
- Add /api/tldStats endpoint for domain/feed counts per TLD
- Show stats bar at top when browsing a TLD
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
- Use labeled links (Article · Audio) instead of raw URLs in posts
- Add language filter dropdown to dashboard with toggle selection
- Auto-deny feeds with no language on discovery
- Add deny/undeny buttons for domains to block crawling
- Denied domains set feeds to dead status, preventing future checks
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
- Changed short URL format from app.1440.news/r/{code} to url.1440.news/{code}
- Added Traefik routing for url.1440.news domain
- Root handler checks Host header to route url.1440.news requests
- Legacy /r/ path still supported for backwards compatibility
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
- New short_urls and clicks tables for URL mapping and analytics
- /r/{code} redirect endpoint with click tracking
- Short URLs use 6-char base64 hash codes (26 chars total)
- Publish loop now shortens article links and enclosure URLs
- Enables podcast audio URLs to fit in posts (139 → 26 chars)
- Tracks: timestamp, referrer, user agent, anonymized IP
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Both createdAt and rkey now use the original publication date,
so posts sort consistently by their original publication time.
Falls back to DiscoveredAt if PubDate is not available.
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Publishing:
- Add publisher.go for posting feed items to AT Protocol PDS
- Support deterministic rkeys from SHA256(guid + discoveredAt)
- Handle multiple URLs in posts with facets for each link
- Image embed support (app.bsky.embed.images) for up to 4 images
- External embed with thumbnail fallback
- Podcast/audio enclosure URLs included in post text
Media extraction:
- Parse RSS enclosures (audio, video, images)
- Extract Media RSS content and thumbnails
- Extract images from HTML content in descriptions
- Store enclosure and imageUrls in items table
SQLite stability improvements:
- Add synchronous=NORMAL and wal_autocheckpoint pragmas
- Connection pool tuning (idle conns, max lifetime)
- Periodic WAL checkpoint every 5 minutes
- Hourly integrity checks with PRAGMA quick_check
- Daily hot backup via VACUUM INTO
- Docker stop_grace_period: 30s for graceful shutdown
Dashboard:
- Feed publishing UI and API endpoints
- Account creation with invite codes
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
- Split main.go into separate files for better organization:
crawler.go, domain.go, feed.go, parser.go, html.go, util.go
- Add PebbleDB for persistent storage of feeds and domains
- Store feeds with metadata: title, TTL, update frequency, ETag, etc.
- Track domains with crawl status (uncrawled/crawled/error)
- Normalize URLs by stripping scheme and www. prefix
- Add web dashboard on port 4321 with real-time stats:
- Crawl progress with completion percentage
- Feed counts by type (RSS/Atom)
- Top TLDs and domains by feed count
- Recent feeds table
- Filter out comment feeds from results
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
