crawler

primal a5af4e14c3 Simplify: auto-deny all domains starting with digits

Replaced complex checks (digit-dash, all-digit hostname) with simple rule:
deny any domain starting with a digit, except 1440.news.

Database: denied 2,870,090 domains and 24,885 feeds.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

2026-01-29 13:32:42 -05:00

static

Add sort toggle for domain list (A-Z vs feed count)

2026-01-29 13:09:31 -05:00

.dockerignore

Add Docker support and refactor data layer

2026-01-26 16:02:05 -05:00

.gitignore

Add PDS credentials env file for service auth

2026-01-26 19:38:36 -05:00

CLAUDE.md

Migrate from SQLite to PostgreSQL

2026-01-28 20:38:13 -05:00

crawler.go

Optimize stats and TLD queries for performance

2026-01-29 13:00:13 -05:00

dashboard.go

Add sort toggle for domain list (A-Z vs feed count)

2026-01-29 13:09:31 -05:00

db.go

Add URL shortener for link tracking and shorter posts

2026-01-28 22:43:42 -05:00

docker-compose.yml

Use url.1440.news for shorter URLs (28 chars vs 32)

2026-01-28 22:52:45 -05:00

Dockerfile

Add Docker support and refactor data layer

2026-01-26 16:02:05 -05:00

domain.go

Simplify: auto-deny all domains starting with digits

2026-01-29 13:32:42 -05:00

feed.go

Auto-deny all-digit domains, whitelist 1440.news

2026-01-29 13:27:48 -05:00

html.go

Add JSON Feed support

2026-01-29 13:16:50 -05:00

main.go

Enable .com domain import from vertices.txt.gz

2026-01-28 21:59:14 -05:00

parser.go

Add JSON Feed support

2026-01-29 13:16:50 -05:00

pds.env.example

Add PDS credentials env file for service auth

2026-01-26 19:38:36 -05:00

publisher.go

Add rich text links, language filter, and domain deny feature

2026-01-29 12:36:58 -05:00

shortener.go

Use url.1440.news for shorter URLs (28 chars vs 32)

2026-01-28 22:52:45 -05:00

util.go

Add PebbleDB storage, domain tracking, and web dashboard

2026-01-22 16:29:00 -05:00