← Logs

social_media_monitor_2026-07-04.log

2026-07-04 19:30:00,653 - __main__ - INFO - === Social Media Monitor v2.0 (Async) Started ===
2026-07-04 19:30:00,683 - social_media_monitor.config.settings - INFO - Configuration loaded from /home/openclaw/Social_Media_Monitor/config.yaml
2026-07-04 19:30:00,683 - __main__ - INFO - Configuration loaded and validated successfully
2026-07-04 19:30:00,685 - social_media_monitor.database.connection - INFO - Database connection pool initialized with 5 connections
2026-07-04 19:30:00,685 - social_media_monitor.database.connection - INFO - add_speaker_filters column already exists
2026-07-04 19:30:00,702 - social_media_monitor.database.connection - INFO - Database setup completed successfully
2026-07-04 19:30:00,702 - __main__ - INFO - Database initialized at: /mnt/data/openclaw-shared/home/shared/Social_Media_Monitor/social_media_monitor.db
2026-07-04 19:30:00,704 - social_media_monitor.scrapers.reddit_scraper - INFO - Reddit API initialized successfully
2026-07-04 19:30:00,705 - __main__ - INFO - GitHub scraper initialized and enabled
2026-07-04 19:30:00,705 - __main__ - INFO - GitHub feed scraper initialized and enabled
2026-07-04 19:30:00,705 - RSSFeedScraper - INFO - RSS scraper initialized: feeds=2, html_urls=0 (0 with custom selectors), include_terms=0, exclude_terms=0
2026-07-04 19:30:00,705 - __main__ - INFO - LinkedIn Feed scraper initialized and enabled
2026-07-04 19:30:00,708 - Yiddish24Scraper - INFO - File downloader initialized with 55 terms, prefix 'Y2', and file type organization enabled
2026-07-04 19:30:00,708 - Yiddish24Scraper - INFO - Yiddish24 scraper initialized: data_id='latest', page_limit=100, total_pages=10, debug_html=True, include_terms=55, exclude_terms=0
2026-07-04 19:30:00,710 - AhBlickLiveScraper - INFO - AhBlickLive scraper initialized: urls=1, max_items=100, include_terms=49, exclude_terms=0
2026-07-04 19:30:00,715 - TorahAnytimeScraper - INFO - File downloader initialized with 53 terms, prefix 'TA', and file type organization enabled
2026-07-04 19:30:00,715 - TorahAnytimeScraper - INFO - TorahAnytime scraper initialized: limit=400, offset=0, project_id=1, include_terms=53, exclude_terms=10
2026-07-04 19:30:00,717 - BatorahScraper - INFO - File downloader initialized with 19 terms, prefix 'BT', and file type organization enabled
2026-07-04 19:30:00,717 - BatorahScraper - INFO - Batorah scraper initialized: api_urls=2, page_size=100, include_terms=19, exclude_terms=1
2026-07-04 19:30:00,717 - KolHalashonScraper - INFO - Kol Halashon scraper initialized: base_url=https://www2.kolhalashon.com, max_shiurim=50, fetch_all_speakers=True, headless=False, exclude_filters=1, include_filters=0
2026-07-04 19:30:00,717 - __main__ - INFO - Running 3 scrapers concurrently...
2026-07-04 19:30:00,718 - GitHubScraper - INFO - Searching GitHub for terms: ['paperclip', '"Hermes Agent"', 'multica']
2026-07-04 19:30:00,815 - LinkedInFeedScraper - INFO - Fetching LinkedIn feed (25 posts)
2026-07-04 19:30:05,401 - LinkedInFeedScraper - ERROR - LinkedIn feed fetch failed: Unauthorized: LinkedIn session cookies are expired or invalid.
2026-07-04 19:30:05,410 - LinkedInFeedScraper - INFO - LinkedIn Feed scrape complete. Found 0 new posts.
2026-07-04 19:30:06,091 - httpx - INFO - HTTP Request: GET https://api.github.com/users/abeperl/received_events?per_page=100&page=1 "HTTP/1.1 200 OK"
2026-07-04 19:30:06,783 - httpx - INFO - HTTP Request: GET https://api.github.com/search/repositories?q=paperclip+in%3Aname%2Cdescription%2Creadme+stars%3A%3E150&sort=updated&order=desc&per_page=20 "HTTP/1.1 200 OK"
2026-07-04 19:30:06,788 - GitHubScraper - INFO - Found 242 GitHub repositories for term: paperclip
2026-07-04 19:30:07,478 - httpx - INFO - HTTP Request: GET https://api.github.com/users/abeperl/received_events?per_page=100&page=2 "HTTP/1.1 200 OK"
2026-07-04 19:30:08,279 - httpx - INFO - HTTP Request: GET https://api.github.com/search/repositories?q=%22Hermes+Agent%22+in%3Aname%2Cdescription%2Creadme+stars%3A%3E150&sort=updated&order=desc&per_page=20 "HTTP/1.1 200 OK"
2026-07-04 19:30:08,283 - GitHubScraper - INFO - Found 420 GitHub repositories for term: "Hermes Agent"
2026-07-04 19:30:09,429 - httpx - INFO - HTTP Request: GET https://api.github.com/search/repositories?q=multica+in%3Aname%2Cdescription%2Creadme+stars%3A%3E150&sort=updated&order=desc&per_page=20 "HTTP/1.1 200 OK"
2026-07-04 19:30:09,521 - GitHubScraper - INFO - Found 57 GitHub repositories for term: multica
2026-07-04 19:30:09,685 - GitHubScraper - INFO - Total GitHub results found: 0
2026-07-04 19:30:09,725 - httpx - INFO - HTTP Request: GET https://api.github.com/users/abeperl/received_events?per_page=100&page=3 "HTTP/1.1 200 OK"
2026-07-04 19:30:10,447 - GitHubScraper - INFO - GitHub feed: 300 fetched across pages, 6 new events for abeperl
2026-07-04 19:30:10,447 - __main__ - INFO - Scraper github completed: 0 results
2026-07-04 19:30:10,447 - __main__ - INFO - Scraper github_feed completed: 6 results
2026-07-04 19:30:10,447 - __main__ - INFO - Scraper linkedin_feed completed: 0 results
2026-07-04 19:30:10,448 - __main__ - WARNING - LinkedIn Feed: 46 consecutive failures (0 results)
2026-07-04 19:30:11,318 - social_media_monitor.email.sender - INFO - Email sent successfully at 2026-07-04 19:30:11.318108
2026-07-04 19:30:11,318 - __main__ - INFO - Email sent successfully
2026-07-04 19:30:11,318 - __main__ - INFO - Cache stats: 0 items in memory, 0 files on disk
2026-07-04 19:30:11,318 - __main__ - INFO - Monitor complete. Found 6 total results: 6 new github feed results.
2026-07-04 19:30:11,318 - __main__ - INFO - === Social Media Monitor v2.0 Complete ===
2026-07-04 19:30:11,319 - social_media_monitor.database.connection - INFO - Closing all database connections
2026-07-04 19:30:11,394 - social_media_monitor.database.connection - INFO - Closed 5 database connections
2026-07-04 19:30:11,394 - __main__ - INFO - Closed all database connections
2026-07-04 19:30:11,396 - social_media_monitor.database.connection - INFO - Closing all database connections
2026-07-04 19:30:11,396 - social_media_monitor.database.connection - INFO - Closed 0 database connections
2026-07-04 20:04:39,270 - social_media_monitor.utils.logging_config - INFO - Posts log file: /mnt/data/openclaw-shared/home/shared/Social_Media_Monitor/logs/social_media_monitor_posts_2026-07-04.log
2026-07-04 20:04:39,270 - social_media_monitor.utils.logging_config - INFO - Downloads log file: /mnt/data/openclaw-shared/home/shared/Social_Media_Monitor/logs/social_media_monitor_downloads_2026-07-04.log
2026-07-04 20:04:39,270 - social_media_monitor.utils.logging_config - INFO - Logging configured with level INFO, log file: /mnt/data/openclaw-shared/home/shared/Social_Media_Monitor/logs/social_media_monitor_2026-07-04.log
2026-07-04 20:04:39,270 - __main__ - INFO - === Social Media Monitor v2.0 (Async) Started ===
2026-07-04 20:04:39,300 - social_media_monitor.config.settings - INFO - Configuration loaded from /home/openclaw/Social_Media_Monitor/config.yaml
2026-07-04 20:04:39,300 - __main__ - INFO - Configuration loaded and validated successfully
2026-07-04 20:04:39,303 - social_media_monitor.database.connection - INFO - Database connection pool initialized with 5 connections
2026-07-04 20:04:39,303 - social_media_monitor.database.connection - INFO - add_speaker_filters column already exists
2026-07-04 20:04:39,318 - social_media_monitor.database.connection - INFO - Database setup completed successfully
2026-07-04 20:04:39,318 - __main__ - INFO - Database initialized at: /mnt/data/openclaw-shared/home/shared/Social_Media_Monitor/social_media_monitor.db
2026-07-04 20:04:39,321 - social_media_monitor.scrapers.reddit_scraper - INFO - Reddit API initialized successfully
2026-07-04 20:04:39,321 - RSSFeedScraper - INFO - RSS scraper initialized: feeds=2, html_urls=0 (0 with custom selectors), include_terms=0, exclude_terms=0
2026-07-04 20:04:39,321 - __main__ - INFO - RSS scraper initialized and enabled
2026-07-04 20:04:39,324 - Yiddish24Scraper - INFO - File downloader initialized with 55 terms, prefix 'Y2', and file type organization enabled
2026-07-04 20:04:39,324 - Yiddish24Scraper - INFO - Yiddish24 scraper initialized: data_id='latest', page_limit=100, total_pages=10, debug_html=True, include_terms=55, exclude_terms=0
2026-07-04 20:04:39,324 - __main__ - INFO - Yiddish24 scraper initialized and enabled
2026-07-04 20:04:39,327 - AhBlickLiveScraper - INFO - AhBlickLive scraper initialized: urls=1, max_items=100, include_terms=49, exclude_terms=0
2026-07-04 20:04:39,331 - TorahAnytimeScraper - INFO - File downloader initialized with 53 terms, prefix 'TA', and file type organization enabled
2026-07-04 20:04:39,331 - TorahAnytimeScraper - INFO - TorahAnytime scraper initialized: limit=400, offset=0, project_id=1, include_terms=53, exclude_terms=10
2026-07-04 20:04:39,333 - BatorahScraper - INFO - File downloader initialized with 19 terms, prefix 'BT', and file type organization enabled
2026-07-04 20:04:39,333 - BatorahScraper - INFO - Batorah scraper initialized: api_urls=2, page_size=100, include_terms=19, exclude_terms=1
2026-07-04 20:04:39,333 - KolHalashonScraper - INFO - Kol Halashon scraper initialized: base_url=https://www2.kolhalashon.com, max_shiurim=50, fetch_all_speakers=True, headless=False, exclude_filters=1, include_filters=0
2026-07-04 20:04:39,333 - __main__ - INFO - Ivelt scraper initialized and enabled
2026-07-04 20:04:39,334 - __main__ - INFO - Running 3 scrapers concurrently...
2026-07-04 20:04:39,334 - Yiddish24Scraper - INFO - Starting Yiddish24 scraping (max 10 pages, limit 100 items)
2026-07-04 20:04:39,428 - Yiddish24Scraper - INFO - Fetching Yiddish24 page 1 with data_id='latest', page_limit=100
2026-07-04 20:04:39,433 - IveltScraper - INFO - Loaded 181 include, 0 exclude filters from DB topic_filters
2026-07-04 20:04:39,433 - IveltScraper - INFO - Fetching ivelt active topics from https://www.ivelt.com/forum/search.php?search_id=active_topics...
2026-07-04 20:04:39,435 - RSSFeedScraper - INFO - Fetching RSS feed: https://www.theyeshivaworld.com/feed
2026-07-04 20:04:39,699 - RSSFeedScraper - INFO - Fetching RSS feed: https://www.thegatewaypundit.com/feed/
2026-07-04 20:04:40,130 - RSSFeedScraper - INFO - Total RSS/HTML results: 3
2026-07-04 20:04:40,532 - httpx - INFO - HTTP Request: POST https://www.yiddish24.com/ajax/cat_pagination.php "HTTP/1.1 200 OK"
2026-07-04 20:04:40,761 - Yiddish24Scraper - INFO - Received response with 454297 characters
2026-07-04 20:04:40,764 - Yiddish24Scraper - INFO - Extracted HTML from JSON result field (293757 chars)
2026-07-04 20:04:40,767 - Yiddish24Scraper - INFO - Saved HTML debug file to: /mnt/data/openclaw-shared/home/shared/Social_Media_Monitor/debug/yiddish24_page1_response.html
2026-07-04 20:04:40,849 - Yiddish24Scraper - INFO - Found 100 potential article elements on page 1
2026-07-04 20:04:40,889 - Yiddish24Scraper - INFO - Page 1 extraction: 100 successful, 0 failed out of 100 elements
2026-07-04 20:04:40,889 - Yiddish24Scraper - INFO - Parsed 100 articles from HTML on page 1
2026-07-04 20:04:41,560 - Yiddish24Scraper - INFO - Page 1 summary: 0 new out of 100 total articles
2026-07-04 20:04:41,561 - Yiddish24Scraper - INFO - Page 1: 0/100 new items
2026-07-04 20:04:41,561 - Yiddish24Scraper - INFO - Low new item ratio (0.0%), stopping pagination
2026-07-04 20:04:41,561 - Yiddish24Scraper - INFO - Total Yiddish24 results found: 0
2026-07-04 20:04:51,608 - IveltScraper - INFO - Found 50 active topics on ivelt
2026-07-04 20:05:06,562 - IveltScraper - INFO - ivelt scrape complete. Checked 4 topics.
2026-07-04 20:05:06,562 - IveltScraper - INFO - All forums scrape complete. Found 0 new posts.
2026-07-04 20:05:06,562 - __main__ - INFO - Scraper rss completed: 3 results
2026-07-04 20:05:06,563 - __main__ - INFO - Scraper yiddish24 completed: 0 results
2026-07-04 20:05:06,563 - __main__ - INFO - Scraper ivelt completed: 0 results
2026-07-04 20:05:07,380 - social_media_monitor.email.sender - INFO - Email sent successfully at 2026-07-04 20:05:07.380115
2026-07-04 20:05:07,380 - __main__ - INFO - Email sent successfully
2026-07-04 20:05:07,380 - __main__ - INFO - Cache stats: 0 items in memory, 0 files on disk
2026-07-04 20:05:07,381 - __main__ - INFO - Monitor complete. Found 3 total results: 3 new rss results.
2026-07-04 20:05:07,381 - __main__ - INFO - === Social Media Monitor v2.0 Complete ===
2026-07-04 20:05:07,381 - social_media_monitor.database.connection - INFO - Closing all database connections
2026-07-04 20:05:07,501 - social_media_monitor.database.connection - INFO - Closed 5 database connections
2026-07-04 20:05:07,502 - __main__ - INFO - Closed all database connections
2026-07-04 20:05:07,503 - social_media_monitor.database.connection - INFO - Closing all database connections
2026-07-04 20:05:07,503 - social_media_monitor.database.connection - INFO - Closed 0 database connections
2026-07-04 20:30:00,653 - social_media_monitor.utils.logging_config - INFO - Posts log file: /mnt/data/openclaw-shared/home/shared/Social_Media_Monitor/logs/social_media_monitor_posts_2026-07-04.log
2026-07-04 20:30:00,654 - social_media_monitor.utils.logging_config - INFO - Downloads log file: /mnt/data/openclaw-shared/home/shared/Social_Media_Monitor/logs/social_media_monitor_downloads_2026-07-04.log
2026-07-04 20:30:00,654 - social_media_monitor.utils.logging_config - INFO - Logging configured with level INFO, log file: /mnt/data/openclaw-shared/home/shared/Social_Media_Monitor/logs/social_media_monitor_2026-07-04.log
2026-07-04 20:30:00,654 - __main__ - INFO - === Social Media Monitor v2.0 (Async) Started ===
2026-07-04 20:30:00,685 - social_media_monitor.config.settings - INFO - Configuration loaded from /home/openclaw/Social_Media_Monitor/config.yaml
2026-07-04 20:30:00,685 - __main__ - INFO - Configuration loaded and validated successfully
2026-07-04 20:30:00,687 - social_media_monitor.database.connection - INFO - Database connection pool initialized with 5 connections
2026-07-04 20:30:00,688 - social_media_monitor.database.connection - INFO - add_speaker_filters column already exists
2026-07-04 20:30:00,703 - social_media_monitor.database.connection - INFO - Database setup completed successfully
2026-07-04 20:30:00,703 - __main__ - INFO - Database initialized at: /mnt/data/openclaw-shared/home/shared/Social_Media_Monitor/social_media_monitor.db
2026-07-04 20:30:00,752 - social_media_monitor.scrapers.reddit_scraper - INFO - Reddit API initialized successfully
2026-07-04 20:30:00,752 - __main__ - INFO - GitHub scraper initialized and enabled
2026-07-04 20:30:00,752 - __main__ - INFO - GitHub feed scraper initialized and enabled
2026-07-04 20:30:00,754 - RSSFeedScraper - INFO - RSS scraper initialized: feeds=2, html_urls=0 (0 with custom selectors), include_terms=0, exclude_terms=0
2026-07-04 20:30:00,754 - __main__ - INFO - LinkedIn Feed scraper initialized and enabled
2026-07-04 20:30:00,762 - Yiddish24Scraper - INFO - File downloader initialized with 55 terms, prefix 'Y2', and file type organization enabled
2026-07-04 20:30:00,762 - Yiddish24Scraper - INFO - Yiddish24 scraper initialized: data_id='latest', page_limit=100, total_pages=10, debug_html=True, include_terms=55, exclude_terms=0
2026-07-04 20:30:00,769 - AhBlickLiveScraper - INFO - AhBlickLive scraper initialized: urls=1, max_items=100, include_terms=49, exclude_terms=0
2026-07-04 20:30:00,781 - TorahAnytimeScraper - INFO - File downloader initialized with 53 terms, prefix 'TA', and file type organization enabled
2026-07-04 20:30:00,781 - TorahAnytimeScraper - INFO - TorahAnytime scraper initialized: limit=400, offset=0, project_id=1, include_terms=53, exclude_terms=10
2026-07-04 20:30:00,788 - BatorahScraper - INFO - File downloader initialized with 19 terms, prefix 'BT', and file type organization enabled
2026-07-04 20:30:00,788 - BatorahScraper - INFO - Batorah scraper initialized: api_urls=2, page_size=100, include_terms=19, exclude_terms=1
2026-07-04 20:30:00,789 - KolHalashonScraper - INFO - Kol Halashon scraper initialized: base_url=https://www2.kolhalashon.com, max_shiurim=50, fetch_all_speakers=True, headless=False, exclude_filters=1, include_filters=0
2026-07-04 20:30:00,789 - __main__ - INFO - Running 3 scrapers concurrently...
2026-07-04 20:30:00,789 - GitHubScraper - INFO - Searching GitHub for terms: ['paperclip', '"Hermes Agent"', 'multica']
2026-07-04 20:30:00,887 - LinkedInFeedScraper - INFO - Fetching LinkedIn feed (25 posts)
2026-07-04 20:30:06,553 - LinkedInFeedScraper - ERROR - LinkedIn feed fetch failed: Unauthorized: LinkedIn session cookies are expired or invalid.
2026-07-04 20:30:06,561 - LinkedInFeedScraper - INFO - LinkedIn Feed scrape complete. Found 0 new posts.
2026-07-04 20:30:07,232 - httpx - INFO - HTTP Request: GET https://api.github.com/users/abeperl/received_events?per_page=100&page=1 "HTTP/1.1 200 OK"
2026-07-04 20:30:07,990 - httpx - INFO - HTTP Request: GET https://api.github.com/search/repositories?q=paperclip+in%3Aname%2Cdescription%2Creadme+stars%3A%3E150&sort=updated&order=desc&per_page=20 "HTTP/1.1 200 OK"
2026-07-04 20:30:07,995 - GitHubScraper - INFO - Found 242 GitHub repositories for term: paperclip
2026-07-04 20:30:08,700 - httpx - INFO - HTTP Request: GET https://api.github.com/users/abeperl/received_events?per_page=100&page=2 "HTTP/1.1 200 OK"
2026-07-04 20:30:09,399 - httpx - INFO - HTTP Request: GET https://api.github.com/search/repositories?q=%22Hermes+Agent%22+in%3Aname%2Cdescription%2Creadme+stars%3A%3E150&sort=updated&order=desc&per_page=20 "HTTP/1.1 200 OK"
2026-07-04 20:30:09,403 - GitHubScraper - INFO - Found 420 GitHub repositories for term: "Hermes Agent"
2026-07-04 20:30:10,598 - httpx - INFO - HTTP Request: GET https://api.github.com/search/repositories?q=multica+in%3Aname%2Cdescription%2Creadme+stars%3A%3E150&sort=updated&order=desc&per_page=20 "HTTP/1.1 200 OK"
2026-07-04 20:30:10,602 - GitHubScraper - INFO - Found 57 GitHub repositories for term: multica
2026-07-04 20:30:10,741 - GitHubScraper - INFO - Total GitHub results found: 0
2026-07-04 20:30:10,742 - httpx - INFO - HTTP Request: GET https://api.github.com/users/abeperl/received_events?per_page=100&page=3 "HTTP/1.1 200 OK"
2026-07-04 20:30:11,526 - GitHubScraper - INFO - GitHub feed: 300 fetched across pages, 6 new events for abeperl
2026-07-04 20:30:11,527 - __main__ - INFO - Scraper github completed: 0 results
2026-07-04 20:30:11,528 - __main__ - INFO - Scraper github_feed completed: 6 results
2026-07-04 20:30:11,528 - __main__ - INFO - Scraper linkedin_feed completed: 0 results
2026-07-04 20:30:11,528 - __main__ - WARNING - LinkedIn Feed: 47 consecutive failures (0 results)
2026-07-04 20:30:12,286 - social_media_monitor.email.sender - INFO - Email sent successfully at 2026-07-04 20:30:12.285926
2026-07-04 20:30:12,286 - __main__ - INFO - Email sent successfully
2026-07-04 20:30:12,286 - __main__ - INFO - Cache stats: 0 items in memory, 0 files on disk
2026-07-04 20:30:12,286 - __main__ - INFO - Monitor complete. Found 6 total results: 6 new github feed results.
2026-07-04 20:30:12,286 - __main__ - INFO - === Social Media Monitor v2.0 Complete ===
2026-07-04 20:30:12,287 - social_media_monitor.database.connection - INFO - Closing all database connections
2026-07-04 20:30:12,366 - social_media_monitor.database.connection - INFO - Closed 5 database connections
2026-07-04 20:30:12,366 - __main__ - INFO - Closed all database connections
2026-07-04 20:30:12,368 - social_media_monitor.database.connection - INFO - Closing all database connections
2026-07-04 20:30:12,368 - social_media_monitor.database.connection - INFO - Closed 0 database connections
2026-07-04 21:04:39,277 - social_media_monitor.utils.logging_config - INFO - Posts log file: /mnt/data/openclaw-shared/home/shared/Social_Media_Monitor/logs/social_media_monitor_posts_2026-07-04.log
2026-07-04 21:04:39,277 - social_media_monitor.utils.logging_config - INFO - Downloads log file: /mnt/data/openclaw-shared/home/shared/Social_Media_Monitor/logs/social_media_monitor_downloads_2026-07-04.log
2026-07-04 21:04:39,277 - social_media_monitor.utils.logging_config - INFO - Logging configured with level INFO, log file: /mnt/data/openclaw-shared/home/shared/Social_Media_Monitor/logs/social_media_monitor_2026-07-04.log
2026-07-04 21:04:39,278 - __main__ - INFO - === Social Media Monitor v2.0 (Async) Started ===
2026-07-04 21:04:39,308 - social_media_monitor.config.settings - INFO - Configuration loaded from /home/openclaw/Social_Media_Monitor/config.yaml
2026-07-04 21:04:39,308 - __main__ - INFO - Configuration loaded and validated successfully
2026-07-04 21:04:39,310 - social_media_monitor.database.connection - INFO - Database connection pool initialized with 5 connections
2026-07-04 21:04:39,310 - social_media_monitor.database.connection - INFO - add_speaker_filters column already exists
2026-07-04 21:04:39,323 - social_media_monitor.database.connection - INFO - Database setup completed successfully
2026-07-04 21:04:39,323 - __main__ - INFO - Database initialized at: /mnt/data/openclaw-shared/home/shared/Social_Media_Monitor/social_media_monitor.db
2026-07-04 21:04:39,326 - social_media_monitor.scrapers.reddit_scraper - INFO - Reddit API initialized successfully
2026-07-04 21:04:39,326 - RSSFeedScraper - INFO - RSS scraper initialized: feeds=2, html_urls=0 (0 with custom selectors), include_terms=0, exclude_terms=0
2026-07-04 21:04:39,326 - __main__ - INFO - RSS scraper initialized and enabled
2026-07-04 21:04:39,329 - Yiddish24Scraper - INFO - File downloader initialized with 55 terms, prefix 'Y2', and file type organization enabled
2026-07-04 21:04:39,329 - Yiddish24Scraper - INFO - Yiddish24 scraper initialized: data_id='latest', page_limit=100, total_pages=10, debug_html=True, include_terms=55, exclude_terms=0
2026-07-04 21:04:39,329 - __main__ - INFO - Yiddish24 scraper initialized and enabled
2026-07-04 21:04:39,331 - AhBlickLiveScraper - INFO - AhBlickLive scraper initialized: urls=1, max_items=100, include_terms=49, exclude_terms=0
2026-07-04 21:04:39,336 - TorahAnytimeScraper - INFO - File downloader initialized with 53 terms, prefix 'TA', and file type organization enabled
2026-07-04 21:04:39,336 - TorahAnytimeScraper - INFO - TorahAnytime scraper initialized: limit=400, offset=0, project_id=1, include_terms=53, exclude_terms=10
2026-07-04 21:04:39,338 - BatorahScraper - INFO - File downloader initialized with 19 terms, prefix 'BT', and file type organization enabled
2026-07-04 21:04:39,338 - BatorahScraper - INFO - Batorah scraper initialized: api_urls=2, page_size=100, include_terms=19, exclude_terms=1
2026-07-04 21:04:39,338 - KolHalashonScraper - INFO - Kol Halashon scraper initialized: base_url=https://www2.kolhalashon.com, max_shiurim=50, fetch_all_speakers=True, headless=False, exclude_filters=1, include_filters=0
2026-07-04 21:04:39,338 - __main__ - INFO - Ivelt scraper initialized and enabled
2026-07-04 21:04:39,338 - __main__ - INFO - Running 3 scrapers concurrently...
2026-07-04 21:04:39,338 - Yiddish24Scraper - INFO - Starting Yiddish24 scraping (max 10 pages, limit 100 items)
2026-07-04 21:04:39,430 - Yiddish24Scraper - INFO - Fetching Yiddish24 page 1 with data_id='latest', page_limit=100
2026-07-04 21:04:39,435 - IveltScraper - INFO - Loaded 181 include, 0 exclude filters from DB topic_filters
2026-07-04 21:04:39,435 - IveltScraper - INFO - Fetching ivelt active topics from https://www.ivelt.com/forum/search.php?search_id=active_topics...
2026-07-04 21:04:39,436 - RSSFeedScraper - INFO - Fetching RSS feed: https://www.theyeshivaworld.com/feed
2026-07-04 21:04:39,704 - RSSFeedScraper - INFO - Fetching RSS feed: https://www.thegatewaypundit.com/feed/
2026-07-04 21:04:40,160 - RSSFeedScraper - INFO - Total RSS/HTML results: 3
2026-07-04 21:04:40,544 - httpx - INFO - HTTP Request: POST https://www.yiddish24.com/ajax/cat_pagination.php "HTTP/1.1 200 OK"
2026-07-04 21:04:40,778 - Yiddish24Scraper - INFO - Received response with 454297 characters
2026-07-04 21:04:40,781 - Yiddish24Scraper - INFO - Extracted HTML from JSON result field (293757 chars)
2026-07-04 21:04:40,784 - Yiddish24Scraper - INFO - Saved HTML debug file to: /mnt/data/openclaw-shared/home/shared/Social_Media_Monitor/debug/yiddish24_page1_response.html
2026-07-04 21:04:40,861 - Yiddish24Scraper - INFO - Found 100 potential article elements on page 1
2026-07-04 21:04:40,900 - Yiddish24Scraper - INFO - Page 1 extraction: 100 successful, 0 failed out of 100 elements
2026-07-04 21:04:40,900 - Yiddish24Scraper - INFO - Parsed 100 articles from HTML on page 1
2026-07-04 21:04:41,543 - Yiddish24Scraper - INFO - Page 1 summary: 0 new out of 100 total articles
2026-07-04 21:04:41,543 - Yiddish24Scraper - INFO - Page 1: 0/100 new items
2026-07-04 21:04:41,544 - Yiddish24Scraper - INFO - Low new item ratio (0.0%), stopping pagination
2026-07-04 21:04:41,544 - Yiddish24Scraper - INFO - Total Yiddish24 results found: 0
2026-07-04 21:04:51,603 - IveltScraper - INFO - Found 50 active topics on ivelt
2026-07-04 21:05:06,103 - IveltScraper - INFO - ivelt scrape complete. Checked 4 topics.
2026-07-04 21:05:06,104 - IveltScraper - INFO - All forums scrape complete. Found 0 new posts.
2026-07-04 21:05:06,104 - __main__ - INFO - Scraper rss completed: 3 results
2026-07-04 21:05:06,104 - __main__ - INFO - Scraper yiddish24 completed: 0 results
2026-07-04 21:05:06,104 - __main__ - INFO - Scraper ivelt completed: 0 results
2026-07-04 21:05:06,910 - social_media_monitor.email.sender - INFO - Email sent successfully at 2026-07-04 21:05:06.910824
2026-07-04 21:05:06,911 - __main__ - INFO - Email sent successfully
2026-07-04 21:05:06,911 - __main__ - INFO - Cache stats: 0 items in memory, 0 files on disk
2026-07-04 21:05:06,911 - __main__ - INFO - Monitor complete. Found 3 total results: 3 new rss results.
2026-07-04 21:05:06,911 - __main__ - INFO - === Social Media Monitor v2.0 Complete ===
2026-07-04 21:05:06,911 - social_media_monitor.database.connection - INFO - Closing all database connections
2026-07-04 21:05:06,946 - social_media_monitor.database.connection - INFO - Closed 5 database connections
2026-07-04 21:05:06,946 - __main__ - INFO - Closed all database connections
2026-07-04 21:05:06,948 - social_media_monitor.database.connection - INFO - Closing all database connections
2026-07-04 21:05:06,948 - social_media_monitor.database.connection - INFO - Closed 0 database connections
2026-07-04 21:30:00,634 - social_media_monitor.utils.logging_config - INFO - Posts log file: /mnt/data/openclaw-shared/home/shared/Social_Media_Monitor/logs/social_media_monitor_posts_2026-07-04.log
2026-07-04 21:30:00,634 - social_media_monitor.utils.logging_config - INFO - Downloads log file: /mnt/data/openclaw-shared/home/shared/Social_Media_Monitor/logs/social_media_monitor_downloads_2026-07-04.log
2026-07-04 21:30:00,634 - social_media_monitor.utils.logging_config - INFO - Logging configured with level INFO, log file: /mnt/data/openclaw-shared/home/shared/Social_Media_Monitor/logs/social_media_monitor_2026-07-04.log
2026-07-04 21:30:00,634 - __main__ - INFO - === Social Media Monitor v2.0 (Async) Started ===
2026-07-04 21:30:00,664 - social_media_monitor.config.settings - INFO - Configuration loaded from /home/openclaw/Social_Media_Monitor/config.yaml
2026-07-04 21:30:00,664 - __main__ - INFO - Configuration loaded and validated successfully
2026-07-04 21:30:00,666 - social_media_monitor.database.connection - INFO - Database connection pool initialized with 5 connections
2026-07-04 21:30:00,666 - social_media_monitor.database.connection - INFO - add_speaker_filters column already exists
2026-07-04 21:30:00,681 - social_media_monitor.database.connection - INFO - Database setup completed successfully
2026-07-04 21:30:00,681 - __main__ - INFO - Database initialized at: /mnt/data/openclaw-shared/home/shared/Social_Media_Monitor/social_media_monitor.db
2026-07-04 21:30:00,683 - social_media_monitor.scrapers.reddit_scraper - INFO - Reddit API initialized successfully
2026-07-04 21:30:00,683 - __main__ - INFO - GitHub scraper initialized and enabled
2026-07-04 21:30:00,683 - __main__ - INFO - GitHub feed scraper initialized and enabled
2026-07-04 21:30:00,684 - RSSFeedScraper - INFO - RSS scraper initialized: feeds=2, html_urls=0 (0 with custom selectors), include_terms=0, exclude_terms=0
2026-07-04 21:30:00,684 - __main__ - INFO - LinkedIn Feed scraper initialized and enabled
2026-07-04 21:30:00,686 - Yiddish24Scraper - INFO - File downloader initialized with 55 terms, prefix 'Y2', and file type organization enabled
2026-07-04 21:30:00,686 - Yiddish24Scraper - INFO - Yiddish24 scraper initialized: data_id='latest', page_limit=100, total_pages=10, debug_html=True, include_terms=55, exclude_terms=0
2026-07-04 21:30:00,689 - AhBlickLiveScraper - INFO - AhBlickLive scraper initialized: urls=1, max_items=100, include_terms=49, exclude_terms=0
2026-07-04 21:30:00,693 - TorahAnytimeScraper - INFO - File downloader initialized with 53 terms, prefix 'TA', and file type organization enabled
2026-07-04 21:30:00,693 - TorahAnytimeScraper - INFO - TorahAnytime scraper initialized: limit=400, offset=0, project_id=1, include_terms=53, exclude_terms=10
2026-07-04 21:30:00,695 - BatorahScraper - INFO - File downloader initialized with 19 terms, prefix 'BT', and file type organization enabled
2026-07-04 21:30:00,695 - BatorahScraper - INFO - Batorah scraper initialized: api_urls=2, page_size=100, include_terms=19, exclude_terms=1
2026-07-04 21:30:00,696 - KolHalashonScraper - INFO - Kol Halashon scraper initialized: base_url=https://www2.kolhalashon.com, max_shiurim=50, fetch_all_speakers=True, headless=False, exclude_filters=1, include_filters=0
2026-07-04 21:30:00,696 - __main__ - INFO - Running 3 scrapers concurrently...
2026-07-04 21:30:00,696 - GitHubScraper - INFO - Searching GitHub for terms: ['paperclip', '"Hermes Agent"', 'multica']
2026-07-04 21:30:00,789 - LinkedInFeedScraper - INFO - Fetching LinkedIn feed (25 posts)
2026-07-04 21:30:05,394 - LinkedInFeedScraper - ERROR - LinkedIn feed fetch failed: Unauthorized: LinkedIn session cookies are expired or invalid.
2026-07-04 21:30:05,405 - LinkedInFeedScraper - INFO - LinkedIn Feed scrape complete. Found 0 new posts.
2026-07-04 21:30:06,131 - httpx - INFO - HTTP Request: GET https://api.github.com/users/abeperl/received_events?per_page=100&page=1 "HTTP/1.1 200 OK"
2026-07-04 21:30:06,791 - httpx - INFO - HTTP Request: GET https://api.github.com/search/repositories?q=paperclip+in%3Aname%2Cdescription%2Creadme+stars%3A%3E150&sort=updated&order=desc&per_page=20 "HTTP/1.1 200 OK"
2026-07-04 21:30:06,796 - GitHubScraper - INFO - Found 242 GitHub repositories for term: paperclip
2026-07-04 21:30:07,667 - httpx - INFO - HTTP Request: GET https://api.github.com/users/abeperl/received_events?per_page=100&page=2 "HTTP/1.1 200 OK"
2026-07-04 21:30:08,323 - httpx - INFO - HTTP Request: GET https://api.github.com/search/repositories?q=%22Hermes+Agent%22+in%3Aname%2Cdescription%2Creadme+stars%3A%3E150&sort=updated&order=desc&per_page=20 "HTTP/1.1 200 OK"
2026-07-04 21:30:08,327 - GitHubScraper - INFO - Found 420 GitHub repositories for term: "Hermes Agent"
2026-07-04 21:30:09,115 - httpx - INFO - HTTP Request: GET https://api.github.com/search/repositories?q=multica+in%3Aname%2Cdescription%2Creadme+stars%3A%3E150&sort=updated&order=desc&per_page=20 "HTTP/1.1 200 OK"
2026-07-04 21:30:09,119 - GitHubScraper - INFO - Found 57 GitHub repositories for term: multica
2026-07-04 21:30:09,247 - GitHubScraper - INFO - Total GitHub results found: 0
2026-07-04 21:30:09,618 - httpx - INFO - HTTP Request: GET https://api.github.com/users/abeperl/received_events?per_page=100&page=3 "HTTP/1.1 200 OK"
2026-07-04 21:30:10,343 - GitHubScraper - INFO - GitHub feed: 300 fetched across pages, 2 new events for abeperl
2026-07-04 21:30:10,344 - __main__ - INFO - Scraper github completed: 0 results
2026-07-04 21:30:10,344 - __main__ - INFO - Scraper github_feed completed: 2 results
2026-07-04 21:30:10,344 - __main__ - INFO - Scraper linkedin_feed completed: 0 results
2026-07-04 21:30:10,345 - __main__ - WARNING - LinkedIn Feed: 48 consecutive failures (0 results)
2026-07-04 21:30:11,133 - __main__ - INFO - Sent LinkedIn Feed failure alert email (48 failures)
2026-07-04 21:30:12,107 - social_media_monitor.email.sender - INFO - Email sent successfully at 2026-07-04 21:30:12.107703
2026-07-04 21:30:12,108 - __main__ - INFO - Email sent successfully
2026-07-04 21:30:12,108 - __main__ - INFO - Cache stats: 0 items in memory, 0 files on disk
2026-07-04 21:30:12,108 - __main__ - INFO - Monitor complete. Found 2 total results: 2 new github feed results.
2026-07-04 21:30:12,108 - __main__ - INFO - === Social Media Monitor v2.0 Complete ===
2026-07-04 21:30:12,108 - social_media_monitor.database.connection - INFO - Closing all database connections
2026-07-04 21:30:12,256 - social_media_monitor.database.connection - INFO - Closed 5 database connections
2026-07-04 21:30:12,257 - __main__ - INFO - Closed all database connections
2026-07-04 21:30:12,259 - social_media_monitor.database.connection - INFO - Closing all database connections
2026-07-04 21:30:12,259 - social_media_monitor.database.connection - INFO - Closed 0 database connections
2026-07-04 22:04:39,309 - social_media_monitor.utils.logging_config - INFO - Posts log file: /mnt/data/openclaw-shared/home/shared/Social_Media_Monitor/logs/social_media_monitor_posts_2026-07-04.log
2026-07-04 22:04:39,309 - social_media_monitor.utils.logging_config - INFO - Downloads log file: /mnt/data/openclaw-shared/home/shared/Social_Media_Monitor/logs/social_media_monitor_downloads_2026-07-04.log
2026-07-04 22:04:39,309 - social_media_monitor.utils.logging_config - INFO - Logging configured with level INFO, log file: /mnt/data/openclaw-shared/home/shared/Social_Media_Monitor/logs/social_media_monitor_2026-07-04.log
2026-07-04 22:04:39,309 - __main__ - INFO - === Social Media Monitor v2.0 (Async) Started ===
2026-07-04 22:04:39,346 - social_media_monitor.config.settings - INFO - Configuration loaded from /home/openclaw/Social_Media_Monitor/config.yaml
2026-07-04 22:04:39,346 - __main__ - INFO - Configuration loaded and validated successfully
2026-07-04 22:04:39,349 - social_media_monitor.database.connection - INFO - Database connection pool initialized with 5 connections
2026-07-04 22:04:39,349 - social_media_monitor.database.connection - INFO - add_speaker_filters column already exists
2026-07-04 22:04:39,364 - social_media_monitor.database.connection - INFO - Database setup completed successfully
2026-07-04 22:04:39,364 - __main__ - INFO - Database initialized at: /mnt/data/openclaw-shared/home/shared/Social_Media_Monitor/social_media_monitor.db
2026-07-04 22:04:39,416 - social_media_monitor.scrapers.reddit_scraper - INFO - Reddit API initialized successfully
2026-07-04 22:04:39,417 - RSSFeedScraper - INFO - RSS scraper initialized: feeds=2, html_urls=0 (0 with custom selectors), include_terms=0, exclude_terms=0
2026-07-04 22:04:39,417 - __main__ - INFO - RSS scraper initialized and enabled
2026-07-04 22:04:39,421 - Yiddish24Scraper - INFO - File downloader initialized with 55 terms, prefix 'Y2', and file type organization enabled
2026-07-04 22:04:39,421 - Yiddish24Scraper - INFO - Yiddish24 scraper initialized: data_id='latest', page_limit=100, total_pages=10, debug_html=True, include_terms=55, exclude_terms=0
2026-07-04 22:04:39,421 - __main__ - INFO - Yiddish24 scraper initialized and enabled
2026-07-04 22:04:39,424 - AhBlickLiveScraper - INFO - AhBlickLive scraper initialized: urls=1, max_items=100, include_terms=49, exclude_terms=0
2026-07-04 22:04:39,429 - TorahAnytimeScraper - INFO - File downloader initialized with 53 terms, prefix 'TA', and file type organization enabled
2026-07-04 22:04:39,429 - TorahAnytimeScraper - INFO - TorahAnytime scraper initialized: limit=400, offset=0, project_id=1, include_terms=53, exclude_terms=10
2026-07-04 22:04:39,431 - BatorahScraper - INFO - File downloader initialized with 19 terms, prefix 'BT', and file type organization enabled
2026-07-04 22:04:39,431 - BatorahScraper - INFO - Batorah scraper initialized: api_urls=2, page_size=100, include_terms=19, exclude_terms=1
2026-07-04 22:04:39,431 - KolHalashonScraper - INFO - Kol Halashon scraper initialized: base_url=https://www2.kolhalashon.com, max_shiurim=50, fetch_all_speakers=True, headless=False, exclude_filters=1, include_filters=0
2026-07-04 22:04:39,431 - __main__ - INFO - Ivelt scraper initialized and enabled
2026-07-04 22:04:39,431 - __main__ - INFO - Running 3 scrapers concurrently...
2026-07-04 22:04:39,432 - Yiddish24Scraper - INFO - Starting Yiddish24 scraping (max 10 pages, limit 100 items)
2026-07-04 22:04:39,529 - Yiddish24Scraper - INFO - Fetching Yiddish24 page 1 with data_id='latest', page_limit=100
2026-07-04 22:04:39,534 - IveltScraper - INFO - Loaded 181 include, 0 exclude filters from DB topic_filters
2026-07-04 22:04:39,534 - IveltScraper - INFO - Fetching ivelt active topics from https://www.ivelt.com/forum/search.php?search_id=active_topics...
2026-07-04 22:04:39,535 - RSSFeedScraper - INFO - Fetching RSS feed: https://www.theyeshivaworld.com/feed
2026-07-04 22:04:39,794 - RSSFeedScraper - INFO - Fetching RSS feed: https://www.thegatewaypundit.com/feed/
2026-07-04 22:04:40,232 - RSSFeedScraper - INFO - Total RSS/HTML results: 3
2026-07-04 22:04:40,604 - httpx - INFO - HTTP Request: POST https://www.yiddish24.com/ajax/cat_pagination.php "HTTP/1.1 200 OK"
2026-07-04 22:04:40,821 - Yiddish24Scraper - INFO - Received response with 454297 characters
2026-07-04 22:04:40,824 - Yiddish24Scraper - INFO - Extracted HTML from JSON result field (293757 chars)
2026-07-04 22:04:40,827 - Yiddish24Scraper - INFO - Saved HTML debug file to: /mnt/data/openclaw-shared/home/shared/Social_Media_Monitor/debug/yiddish24_page1_response.html
2026-07-04 22:04:40,895 - Yiddish24Scraper - INFO - Found 100 potential article elements on page 1
2026-07-04 22:04:40,937 - Yiddish24Scraper - INFO - Page 1 extraction: 100 successful, 0 failed out of 100 elements
2026-07-04 22:04:40,937 - Yiddish24Scraper - INFO - Parsed 100 articles from HTML on page 1
2026-07-04 22:04:41,672 - Yiddish24Scraper - INFO - Page 1 summary: 0 new out of 100 total articles
2026-07-04 22:04:41,672 - Yiddish24Scraper - INFO - Page 1: 0/100 new items
2026-07-04 22:04:41,673 - Yiddish24Scraper - INFO - Low new item ratio (0.0%), stopping pagination
2026-07-04 22:04:41,673 - Yiddish24Scraper - INFO - Total Yiddish24 results found: 0
2026-07-04 22:04:51,804 - IveltScraper - INFO - Found 50 active topics on ivelt
2026-07-04 22:05:06,377 - IveltScraper - INFO - ivelt scrape complete. Checked 4 topics.
2026-07-04 22:05:06,377 - IveltScraper - INFO - All forums scrape complete. Found 0 new posts.
2026-07-04 22:05:06,378 - __main__ - INFO - Scraper rss completed: 3 results
2026-07-04 22:05:06,378 - __main__ - INFO - Scraper yiddish24 completed: 0 results
2026-07-04 22:05:06,378 - __main__ - INFO - Scraper ivelt completed: 0 results
2026-07-04 22:05:07,183 - social_media_monitor.email.sender - INFO - Email sent successfully at 2026-07-04 22:05:07.183020
2026-07-04 22:05:07,183 - __main__ - INFO - Email sent successfully
2026-07-04 22:05:07,183 - __main__ - INFO - Cache stats: 0 items in memory, 0 files on disk
2026-07-04 22:05:07,183 - __main__ - INFO - Monitor complete. Found 3 total results: 3 new rss results.
2026-07-04 22:05:07,183 - __main__ - INFO - === Social Media Monitor v2.0 Complete ===
2026-07-04 22:05:07,183 - social_media_monitor.database.connection - INFO - Closing all database connections
2026-07-04 22:05:07,228 - social_media_monitor.database.connection - INFO - Closed 5 database connections
2026-07-04 22:05:07,229 - __main__ - INFO - Closed all database connections
2026-07-04 22:05:07,230 - social_media_monitor.database.connection - INFO - Closing all database connections
2026-07-04 22:05:07,230 - social_media_monitor.database.connection - INFO - Closed 0 database connections
2026-07-04 22:30:00,689 - social_media_monitor.utils.logging_config - INFO - Posts log file: /mnt/data/openclaw-shared/home/shared/Social_Media_Monitor/logs/social_media_monitor_posts_2026-07-04.log
2026-07-04 22:30:00,689 - social_media_monitor.utils.logging_config - INFO - Downloads log file: /mnt/data/openclaw-shared/home/shared/Social_Media_Monitor/logs/social_media_monitor_downloads_2026-07-04.log
2026-07-04 22:30:00,690 - social_media_monitor.utils.logging_config - INFO - Logging configured with level INFO, log file: /mnt/data/openclaw-shared/home/shared/Social_Media_Monitor/logs/social_media_monitor_2026-07-04.log
2026-07-04 22:30:00,690 - __main__ - INFO - === Social Media Monitor v2.0 (Async) Started ===
2026-07-04 22:30:00,723 - social_media_monitor.config.settings - INFO - Configuration loaded from /home/openclaw/Social_Media_Monitor/config.yaml
2026-07-04 22:30:00,723 - __main__ - INFO - Configuration loaded and validated successfully
2026-07-04 22:30:00,725 - social_media_monitor.database.connection - INFO - Database connection pool initialized with 5 connections
2026-07-04 22:30:00,726 - social_media_monitor.database.connection - INFO - add_speaker_filters column already exists
2026-07-04 22:30:00,739 - social_media_monitor.database.connection - INFO - Database setup completed successfully
2026-07-04 22:30:00,739 - __main__ - INFO - Database initialized at: /mnt/data/openclaw-shared/home/shared/Social_Media_Monitor/social_media_monitor.db
2026-07-04 22:30:00,742 - social_media_monitor.scrapers.reddit_scraper - INFO - Reddit API initialized successfully
2026-07-04 22:30:00,742 - __main__ - INFO - GitHub scraper initialized and enabled
2026-07-04 22:30:00,742 - __main__ - INFO - GitHub feed scraper initialized and enabled
2026-07-04 22:30:00,743 - RSSFeedScraper - INFO - RSS scraper initialized: feeds=2, html_urls=0 (0 with custom selectors), include_terms=0, exclude_terms=0
2026-07-04 22:30:00,743 - __main__ - INFO - LinkedIn Feed scraper initialized and enabled
2026-07-04 22:30:00,745 - Yiddish24Scraper - INFO - File downloader initialized with 55 terms, prefix 'Y2', and file type organization enabled
2026-07-04 22:30:00,745 - Yiddish24Scraper - INFO - Yiddish24 scraper initialized: data_id='latest', page_limit=100, total_pages=10, debug_html=True, include_terms=55, exclude_terms=0
2026-07-04 22:30:00,748 - AhBlickLiveScraper - INFO - AhBlickLive scraper initialized: urls=1, max_items=100, include_terms=49, exclude_terms=0
2026-07-04 22:30:00,752 - TorahAnytimeScraper - INFO - File downloader initialized with 53 terms, prefix 'TA', and file type organization enabled
2026-07-04 22:30:00,752 - TorahAnytimeScraper - INFO - TorahAnytime scraper initialized: limit=400, offset=0, project_id=1, include_terms=53, exclude_terms=10
2026-07-04 22:30:00,754 - BatorahScraper - INFO - File downloader initialized with 19 terms, prefix 'BT', and file type organization enabled
2026-07-04 22:30:00,754 - BatorahScraper - INFO - Batorah scraper initialized: api_urls=2, page_size=100, include_terms=19, exclude_terms=1
2026-07-04 22:30:00,754 - KolHalashonScraper - INFO - Kol Halashon scraper initialized: base_url=https://www2.kolhalashon.com, max_shiurim=50, fetch_all_speakers=True, headless=False, exclude_filters=1, include_filters=0
2026-07-04 22:30:00,754 - __main__ - INFO - Running 3 scrapers concurrently...
2026-07-04 22:30:00,755 - GitHubScraper - INFO - Searching GitHub for terms: ['paperclip', '"Hermes Agent"', 'multica']
2026-07-04 22:30:00,851 - LinkedInFeedScraper - INFO - Fetching LinkedIn feed (25 posts)
2026-07-04 22:30:03,581 - LinkedInFeedScraper - ERROR - LinkedIn feed fetch failed: Unauthorized: LinkedIn session cookies are expired or invalid.
2026-07-04 22:30:03,589 - LinkedInFeedScraper - INFO - LinkedIn Feed scrape complete. Found 0 new posts.
2026-07-04 22:30:04,188 - httpx - INFO - HTTP Request: GET https://api.github.com/users/abeperl/received_events?per_page=100&page=1 "HTTP/1.1 200 OK"
2026-07-04 22:30:05,570 - httpx - INFO - HTTP Request: GET https://api.github.com/search/repositories?q=paperclip+in%3Aname%2Cdescription%2Creadme+stars%3A%3E150&sort=updated&order=desc&per_page=20 "HTTP/1.1 200 OK"
2026-07-04 22:30:05,575 - GitHubScraper - INFO - Found 242 GitHub repositories for term: paperclip
2026-07-04 22:30:06,235 - httpx - INFO - HTTP Request: GET https://api.github.com/users/abeperl/received_events?per_page=100&page=2 "HTTP/1.1 200 OK"
2026-07-04 22:30:07,210 - httpx - INFO - HTTP Request: GET https://api.github.com/search/repositories?q=%22Hermes+Agent%22+in%3Aname%2Cdescription%2Creadme+stars%3A%3E150&sort=updated&order=desc&per_page=20 "HTTP/1.1 200 OK"
2026-07-04 22:30:07,214 - GitHubScraper - INFO - Found 420 GitHub repositories for term: "Hermes Agent"
2026-07-04 22:30:08,081 - httpx - INFO - HTTP Request: GET https://api.github.com/search/repositories?q=multica+in%3Aname%2Cdescription%2Creadme+stars%3A%3E150&sort=updated&order=desc&per_page=20 "HTTP/1.1 200 OK"
2026-07-04 22:30:08,164 - GitHubScraper - INFO - Found 57 GitHub repositories for term: multica
2026-07-04 22:30:08,308 - GitHubScraper - INFO - Total GitHub results found: 0
2026-07-04 22:30:08,488 - httpx - INFO - HTTP Request: GET https://api.github.com/users/abeperl/received_events?per_page=100&page=3 "HTTP/1.1 200 OK"
2026-07-04 22:30:09,450 - GitHubScraper - INFO - GitHub feed: 300 fetched across pages, 100 new events for abeperl
2026-07-04 22:30:09,451 - __main__ - INFO - Scraper github completed: 0 results
2026-07-04 22:30:09,451 - __main__ - INFO - Scraper github_feed completed: 100 results
2026-07-04 22:30:09,451 - __main__ - INFO - Scraper linkedin_feed completed: 0 results
2026-07-04 22:30:09,452 - __main__ - WARNING - LinkedIn Feed: 49 consecutive failures (0 results)
2026-07-04 22:30:10,324 - social_media_monitor.email.sender - INFO - Email sent successfully at 2026-07-04 22:30:10.324204
2026-07-04 22:30:10,324 - __main__ - INFO - Email sent successfully
2026-07-04 22:30:10,324 - __main__ - INFO - Cache stats: 0 items in memory, 0 files on disk
2026-07-04 22:30:10,324 - __main__ - INFO - Monitor complete. Found 100 total results: 100 new github feed results.
2026-07-04 22:30:10,325 - __main__ - INFO - === Social Media Monitor v2.0 Complete ===
2026-07-04 22:30:10,325 - social_media_monitor.database.connection - INFO - Closing all database connections
2026-07-04 22:30:10,451 - social_media_monitor.database.connection - INFO - Closed 5 database connections
2026-07-04 22:30:10,452 - __main__ - INFO - Closed all database connections
2026-07-04 22:30:10,454 - social_media_monitor.database.connection - INFO - Closing all database connections
2026-07-04 22:30:10,454 - social_media_monitor.database.connection - INFO - Closed 0 database connections
2026-07-04 23:04:39,311 - social_media_monitor.utils.logging_config - INFO - Posts log file: /mnt/data/openclaw-shared/home/shared/Social_Media_Monitor/logs/social_media_monitor_posts_2026-07-04.log
2026-07-04 23:04:39,312 - social_media_monitor.utils.logging_config - INFO - Downloads log file: /mnt/data/openclaw-shared/home/shared/Social_Media_Monitor/logs/social_media_monitor_downloads_2026-07-04.log
2026-07-04 23:04:39,312 - social_media_monitor.utils.logging_config - INFO - Logging configured with level INFO, log file: /mnt/data/openclaw-shared/home/shared/Social_Media_Monitor/logs/social_media_monitor_2026-07-04.log
2026-07-04 23:04:39,312 - __main__ - INFO - === Social Media Monitor v2.0 (Async) Started ===
2026-07-04 23:04:39,343 - social_media_monitor.config.settings - INFO - Configuration loaded from /home/openclaw/Social_Media_Monitor/config.yaml
2026-07-04 23:04:39,344 - __main__ - INFO - Configuration loaded and validated successfully
2026-07-04 23:04:39,346 - social_media_monitor.database.connection - INFO - Database connection pool initialized with 5 connections
2026-07-04 23:04:39,346 - social_media_monitor.database.connection - INFO - add_speaker_filters column already exists
2026-07-04 23:04:39,363 - social_media_monitor.database.connection - INFO - Database setup completed successfully
2026-07-04 23:04:39,363 - __main__ - INFO - Database initialized at: /mnt/data/openclaw-shared/home/shared/Social_Media_Monitor/social_media_monitor.db
2026-07-04 23:04:39,366 - social_media_monitor.scrapers.reddit_scraper - INFO - Reddit API initialized successfully
2026-07-04 23:04:39,367 - RSSFeedScraper - INFO - RSS scraper initialized: feeds=2, html_urls=0 (0 with custom selectors), include_terms=0, exclude_terms=0
2026-07-04 23:04:39,367 - __main__ - INFO - RSS scraper initialized and enabled
2026-07-04 23:04:39,369 - Yiddish24Scraper - INFO - File downloader initialized with 55 terms, prefix 'Y2', and file type organization enabled
2026-07-04 23:04:39,369 - Yiddish24Scraper - INFO - Yiddish24 scraper initialized: data_id='latest', page_limit=100, total_pages=10, debug_html=True, include_terms=55, exclude_terms=0
2026-07-04 23:04:39,369 - __main__ - INFO - Yiddish24 scraper initialized and enabled
2026-07-04 23:04:39,372 - AhBlickLiveScraper - INFO - AhBlickLive scraper initialized: urls=1, max_items=100, include_terms=49, exclude_terms=0
2026-07-04 23:04:39,377 - TorahAnytimeScraper - INFO - File downloader initialized with 53 terms, prefix 'TA', and file type organization enabled
2026-07-04 23:04:39,377 - TorahAnytimeScraper - INFO - TorahAnytime scraper initialized: limit=400, offset=0, project_id=1, include_terms=53, exclude_terms=10
2026-07-04 23:04:39,379 - BatorahScraper - INFO - File downloader initialized with 19 terms, prefix 'BT', and file type organization enabled
2026-07-04 23:04:39,379 - BatorahScraper - INFO - Batorah scraper initialized: api_urls=2, page_size=100, include_terms=19, exclude_terms=1
2026-07-04 23:04:39,379 - KolHalashonScraper - INFO - Kol Halashon scraper initialized: base_url=https://www2.kolhalashon.com, max_shiurim=50, fetch_all_speakers=True, headless=False, exclude_filters=1, include_filters=0
2026-07-04 23:04:39,379 - __main__ - INFO - Ivelt scraper initialized and enabled
2026-07-04 23:04:39,379 - __main__ - INFO - Running 3 scrapers concurrently...
2026-07-04 23:04:39,380 - Yiddish24Scraper - INFO - Starting Yiddish24 scraping (max 10 pages, limit 100 items)
2026-07-04 23:04:39,471 - Yiddish24Scraper - INFO - Fetching Yiddish24 page 1 with data_id='latest', page_limit=100
2026-07-04 23:04:39,475 - IveltScraper - INFO - Loaded 181 include, 0 exclude filters from DB topic_filters
2026-07-04 23:04:39,475 - IveltScraper - INFO - Fetching ivelt active topics from https://www.ivelt.com/forum/search.php?search_id=active_topics...
2026-07-04 23:04:39,477 - RSSFeedScraper - INFO - Fetching RSS feed: https://www.theyeshivaworld.com/feed
2026-07-04 23:04:39,755 - RSSFeedScraper - INFO - Fetching RSS feed: https://www.thegatewaypundit.com/feed/
2026-07-04 23:04:40,177 - RSSFeedScraper - INFO - Total RSS/HTML results: 3
2026-07-04 23:04:40,557 - httpx - INFO - HTTP Request: POST https://www.yiddish24.com/ajax/cat_pagination.php "HTTP/1.1 200 OK"
2026-07-04 23:04:40,769 - Yiddish24Scraper - INFO - Received response with 454297 characters
2026-07-04 23:04:40,774 - Yiddish24Scraper - INFO - Extracted HTML from JSON result field (293757 chars)
2026-07-04 23:04:40,778 - Yiddish24Scraper - INFO - Saved HTML debug file to: /mnt/data/openclaw-shared/home/shared/Social_Media_Monitor/debug/yiddish24_page1_response.html
2026-07-04 23:04:40,855 - Yiddish24Scraper - INFO - Found 100 potential article elements on page 1
2026-07-04 23:04:40,895 - Yiddish24Scraper - INFO - Page 1 extraction: 100 successful, 0 failed out of 100 elements
2026-07-04 23:04:40,896 - Yiddish24Scraper - INFO - Parsed 100 articles from HTML on page 1
2026-07-04 23:04:41,628 - Yiddish24Scraper - INFO - Page 1 summary: 0 new out of 100 total articles
2026-07-04 23:04:41,628 - Yiddish24Scraper - INFO - Page 1: 0/100 new items
2026-07-04 23:04:41,628 - Yiddish24Scraper - INFO - Low new item ratio (0.0%), stopping pagination
2026-07-04 23:04:41,629 - Yiddish24Scraper - INFO - Total Yiddish24 results found: 0
2026-07-04 23:04:51,599 - IveltScraper - INFO - Found 50 active topics on ivelt
2026-07-04 23:05:11,049 - IveltScraper - INFO - ivelt scrape complete. Checked 5 topics.
2026-07-04 23:05:11,050 - IveltScraper - INFO - All forums scrape complete. Found 1 new posts.
2026-07-04 23:05:11,050 - __main__ - INFO - Scraper rss completed: 3 results
2026-07-04 23:05:11,050 - __main__ - INFO - Scraper yiddish24 completed: 0 results
2026-07-04 23:05:11,050 - __main__ - INFO - Scraper ivelt completed: 1 results
2026-07-04 23:05:11,820 - social_media_monitor.email.sender - INFO - Email sent successfully at 2026-07-04 23:05:11.820440
2026-07-04 23:05:11,820 - __main__ - INFO - Email sent successfully
2026-07-04 23:05:11,821 - __main__ - INFO - Cache stats: 0 items in memory, 0 files on disk
2026-07-04 23:05:11,821 - __main__ - INFO - Monitor complete. Found 4 total results: 3 new rss results, 1 new ivelt results.
2026-07-04 23:05:11,821 - __main__ - INFO - === Social Media Monitor v2.0 Complete ===
2026-07-04 23:05:11,821 - social_media_monitor.database.connection - INFO - Closing all database connections
2026-07-04 23:05:11,859 - social_media_monitor.database.connection - INFO - Closed 5 database connections
2026-07-04 23:05:11,860 - __main__ - INFO - Closed all database connections
2026-07-04 23:05:11,862 - social_media_monitor.database.connection - INFO - Closing all database connections
2026-07-04 23:05:11,862 - social_media_monitor.database.connection - INFO - Closed 0 database connections
2026-07-04 23:30:00,647 - social_media_monitor.utils.logging_config - INFO - Posts log file: /mnt/data/openclaw-shared/home/shared/Social_Media_Monitor/logs/social_media_monitor_posts_2026-07-04.log
2026-07-04 23:30:00,647 - social_media_monitor.utils.logging_config - INFO - Downloads log file: /mnt/data/openclaw-shared/home/shared/Social_Media_Monitor/logs/social_media_monitor_downloads_2026-07-04.log
2026-07-04 23:30:00,647 - social_media_monitor.utils.logging_config - INFO - Logging configured with level INFO, log file: /mnt/data/openclaw-shared/home/shared/Social_Media_Monitor/logs/social_media_monitor_2026-07-04.log
2026-07-04 23:30:00,647 - __main__ - INFO - === Social Media Monitor v2.0 (Async) Started ===
2026-07-04 23:30:00,679 - social_media_monitor.config.settings - INFO - Configuration loaded from /home/openclaw/Social_Media_Monitor/config.yaml
2026-07-04 23:30:00,679 - __main__ - INFO - Configuration loaded and validated successfully
2026-07-04 23:30:00,681 - social_media_monitor.database.connection - INFO - Database connection pool initialized with 5 connections
2026-07-04 23:30:00,681 - social_media_monitor.database.connection - INFO - add_speaker_filters column already exists
2026-07-04 23:30:00,694 - social_media_monitor.database.connection - INFO - Database setup completed successfully
2026-07-04 23:30:00,694 - __main__ - INFO - Database initialized at: /mnt/data/openclaw-shared/home/shared/Social_Media_Monitor/social_media_monitor.db
2026-07-04 23:30:00,742 - social_media_monitor.scrapers.reddit_scraper - INFO - Reddit API initialized successfully
2026-07-04 23:30:00,742 - __main__ - INFO - GitHub scraper initialized and enabled
2026-07-04 23:30:00,742 - __main__ - INFO - GitHub feed scraper initialized and enabled
2026-07-04 23:30:00,742 - RSSFeedScraper - INFO - RSS scraper initialized: feeds=2, html_urls=0 (0 with custom selectors), include_terms=0, exclude_terms=0
2026-07-04 23:30:00,743 - __main__ - INFO - LinkedIn Feed scraper initialized and enabled
2026-07-04 23:30:00,746 - Yiddish24Scraper - INFO - File downloader initialized with 55 terms, prefix 'Y2', and file type organization enabled
2026-07-04 23:30:00,746 - Yiddish24Scraper - INFO - Yiddish24 scraper initialized: data_id='latest', page_limit=100, total_pages=10, debug_html=True, include_terms=55, exclude_terms=0
2026-07-04 23:30:00,749 - AhBlickLiveScraper - INFO - AhBlickLive scraper initialized: urls=1, max_items=100, include_terms=49, exclude_terms=0
2026-07-04 23:30:00,753 - TorahAnytimeScraper - INFO - File downloader initialized with 53 terms, prefix 'TA', and file type organization enabled
2026-07-04 23:30:00,753 - TorahAnytimeScraper - INFO - TorahAnytime scraper initialized: limit=400, offset=0, project_id=1, include_terms=53, exclude_terms=10
2026-07-04 23:30:00,755 - BatorahScraper - INFO - File downloader initialized with 19 terms, prefix 'BT', and file type organization enabled
2026-07-04 23:30:00,755 - BatorahScraper - INFO - Batorah scraper initialized: api_urls=2, page_size=100, include_terms=19, exclude_terms=1
2026-07-04 23:30:00,755 - KolHalashonScraper - INFO - Kol Halashon scraper initialized: base_url=https://www2.kolhalashon.com, max_shiurim=50, fetch_all_speakers=True, headless=False, exclude_filters=1, include_filters=0
2026-07-04 23:30:00,755 - __main__ - INFO - Running 3 scrapers concurrently...
2026-07-04 23:30:00,756 - GitHubScraper - INFO - Searching GitHub for terms: ['paperclip', '"Hermes Agent"', 'multica']
2026-07-04 23:30:00,851 - LinkedInFeedScraper - INFO - Fetching LinkedIn feed (25 posts)
2026-07-04 23:30:05,489 - LinkedInFeedScraper - ERROR - LinkedIn feed fetch failed: Unauthorized: LinkedIn session cookies are expired or invalid.
2026-07-04 23:30:05,498 - LinkedInFeedScraper - INFO - LinkedIn Feed scrape complete. Found 0 new posts.
2026-07-04 23:30:06,092 - httpx - INFO - HTTP Request: GET https://api.github.com/users/abeperl/received_events?per_page=100&page=1 "HTTP/1.1 200 OK"
2026-07-04 23:30:07,248 - httpx - INFO - HTTP Request: GET https://api.github.com/search/repositories?q=paperclip+in%3Aname%2Cdescription%2Creadme+stars%3A%3E150&sort=updated&order=desc&per_page=20 "HTTP/1.1 200 OK"
2026-07-04 23:30:07,254 - GitHubScraper - INFO - Found 242 GitHub repositories for term: paperclip
2026-07-04 23:30:08,209 - httpx - INFO - HTTP Request: GET https://api.github.com/search/repositories?q=%22Hermes+Agent%22+in%3Aname%2Cdescription%2Creadme+stars%3A%3E150&sort=updated&order=desc&per_page=20 "HTTP/1.1 200 OK"
2026-07-04 23:30:08,213 - GitHubScraper - INFO - Found 420 GitHub repositories for term: "Hermes Agent"
2026-07-04 23:30:08,699 - httpx - INFO - HTTP Request: GET https://api.github.com/users/abeperl/received_events?per_page=100&page=2 "HTTP/1.1 200 OK"
2026-07-04 23:30:09,598 - httpx - INFO - HTTP Request: GET https://api.github.com/search/repositories?q=multica+in%3Aname%2Cdescription%2Creadme+stars%3A%3E150&sort=updated&order=desc&per_page=20 "HTTP/1.1 200 OK"
2026-07-04 23:30:09,602 - GitHubScraper - INFO - Found 57 GitHub repositories for term: multica
2026-07-04 23:30:09,742 - GitHubScraper - INFO - Total GitHub results found: 0
2026-07-04 23:30:10,198 - httpx - INFO - HTTP Request: GET https://api.github.com/users/abeperl/received_events?per_page=100&page=3 "HTTP/1.1 200 OK"
2026-07-04 23:30:10,907 - GitHubScraper - INFO - GitHub feed: 300 fetched across pages, 200 new events for abeperl
2026-07-04 23:30:10,908 - __main__ - INFO - Scraper github completed: 0 results
2026-07-04 23:30:10,909 - __main__ - INFO - Scraper github_feed completed: 200 results
2026-07-04 23:30:10,909 - __main__ - INFO - Scraper linkedin_feed completed: 0 results
2026-07-04 23:30:10,909 - __main__ - WARNING - LinkedIn Feed: 50 consecutive failures (0 results)
2026-07-04 23:30:11,892 - social_media_monitor.email.sender - INFO - Email sent successfully at 2026-07-04 23:30:11.892615
2026-07-04 23:30:11,892 - __main__ - INFO - Email sent successfully
2026-07-04 23:30:11,893 - __main__ - INFO - Cache stats: 0 items in memory, 0 files on disk
2026-07-04 23:30:11,893 - __main__ - INFO - Monitor complete. Found 200 total results: 200 new github feed results.
2026-07-04 23:30:11,893 - __main__ - INFO - === Social Media Monitor v2.0 Complete ===
2026-07-04 23:30:11,893 - social_media_monitor.database.connection - INFO - Closing all database connections
2026-07-04 23:30:11,930 - social_media_monitor.database.connection - INFO - Closed 5 database connections
2026-07-04 23:30:11,930 - __main__ - INFO - Closed all database connections
2026-07-04 23:30:11,932 - social_media_monitor.database.connection - INFO - Closing all database connections
2026-07-04 23:30:11,932 - social_media_monitor.database.connection - INFO - Closed 0 database connections