Week 51 Changelog

28 Dec, 2025

Added new scripts for content and community workflows:
- scripts/weekly_changelog_llm.py for generating weekly changelogs using an LLM
- scripts/gen_newsletter.py for newsletter generation
- scripts/generate_social_posts.py for creating social media posts
Added new utility code in the comment crawler, including book_link_utils.py and an updated implementation of check_recent_book_links.py.
Added a new version of the bulk indexer script as scripts/bulk_index_new.py.new for future indexing improvements.
Added scripts/latest_social_posts.json to support social post generation.

Moved existing maintenance scripts (backfill_authors_books.py and bulk_index_new.py) into a dedicated scripts/ directory to improve project organization.
Updated crawler-related files, including hnb-comment-crawler/hn_crawler.py and the crawler crontab configuration (hnb-book-crawler/crontab.crawler).
Tweaked .gitignore to reflect the current project structure and remove obsolete entries.

Improved comment crawler behavior by updating hn_crawler.py and replacing the old check_recent_book_links.py with a cleaner implementation.
Adjusted crawler scheduling and automation via changes to hnb-book-crawler/crontab.crawler so recent book links stay fresh.

Updated cron jobs and Docker-related automation for the book crawler (hnb-book-crawler/crontab.crawler and related files).
Removed legacy deployment and maintenance scripts (various old .py and .sh utilities) to streamline infrastructure.
Cleaned up large, outdated data dumps (all_books.txt, all_urls_before_update.txt, have_ids.txt, test_books.txt) and other obsolete files (link_update_log.jsonl, etc.) to reduce clutter and keep the repo lean.

Next up: Fix Bluesky logo