crawler

primal 037b453a68 Remove oldest_item_date and newest_item_date columns

- Drop columns from schema, add migration
- Remove date stats calculation from parsers
- Update all feed queries to exclude these columns

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

2026-02-04 19:28:20 -05:00

cmd

Remove discovered_at column from feeds and items tables

2026-02-04 19:07:20 -05:00

scripts

Restore working codebase with all methods

2026-02-01 19:08:53 -05:00

.dockerignore

Restore working codebase with all methods

2026-02-01 19:08:53 -05:00

.gitignore

Add AT Protocol OAuth 2.0 authentication for dashboard

2026-01-30 15:16:51 -05:00

.launch.sh

Use Unix timestamp for cache busting, remove version display

2026-02-01 18:01:04 -05:00

cdx.go

Remove discovered_at column from feeds and items tables

2026-02-04 19:07:20 -05:00

CLAUDE.md

Remove publishing code - now handled by publish service

2026-02-02 15:40:49 -05:00

crawler.go

Simplify crawler: use CDX for feed discovery, remove unused loops

2026-02-03 22:53:23 -05:00

db.go

Remove oldest_item_date and newest_item_date columns

2026-02-04 19:28:20 -05:00

docker-compose.yml

Simplify crawler: use CDX for feed discovery, remove unused loops

2026-02-03 22:53:23 -05:00

Dockerfile

Update shared references to commons

2026-02-02 15:19:48 -05:00

domain.go

Simplify crawler: use CDX for feed discovery, remove unused loops

2026-02-03 22:53:23 -05:00

feed_check.go

Remove discovered_at column from feeds and items tables

2026-02-04 19:07:20 -05:00

feed.go

Remove oldest_item_date and newest_item_date columns

2026-02-04 19:28:20 -05:00

html.go

Simplify crawler: use CDX for feed discovery, remove unused loops

2026-02-03 22:53:23 -05:00

item.go

Remove discovered_at column from feeds and items tables

2026-02-04 19:07:20 -05:00

main.go

Add bulk import for CDX feeds

2026-02-04 10:42:52 -05:00

oauth.env.example

Add AT Protocol OAuth 2.0 authentication for dashboard

2026-01-30 15:16:51 -05:00

parser.go

Remove oldest_item_date and newest_item_date columns

2026-02-04 19:28:20 -05:00

pds.env.example

Add PDS credentials env file for service auth

2026-01-26 19:38:36 -05:00

shortener.go

Trim shortener.go - keep only URL creation, remove click tracking

2026-02-02 13:28:10 -05:00

test-app

Restore working codebase with all methods

2026-02-01 19:08:53 -05:00

tld.go

Remove next_check_at column from feeds table

2026-02-04 16:07:04 -05:00

tlds.txt

Restore working codebase with all methods

2026-02-01 19:08:53 -05:00

util.go

Simplify crawler: use CDX for feed discovery, remove unused loops

2026-02-03 22:53:23 -05:00