← Logs

social_media_monitor_2026-07-03.log

2026-07-03 19:30:00,683 - __main__ - INFO - Database initialized at: /mnt/data/openclaw-shared/home/shared/Social_Media_Monitor/social_media_monitor.db

2026-07-03 19:30:00,730 - social_media_monitor.scrapers.reddit_scraper - INFO - Reddit API initialized successfully

2026-07-03 19:30:00,730 - __main__ - INFO - GitHub scraper initialized and enabled

2026-07-03 19:30:00,730 - __main__ - INFO - GitHub feed scraper initialized and enabled

2026-07-03 19:30:00,730 - RSSFeedScraper - INFO - RSS scraper initialized: feeds=2, html_urls=0 (0 with custom selectors), include_terms=0, exclude_terms=0

2026-07-03 19:30:00,731 - __main__ - INFO - LinkedIn Feed scraper initialized and enabled

2026-07-03 19:30:00,734 - Yiddish24Scraper - INFO - File downloader initialized with 55 terms, prefix 'Y2', and file type organization enabled

2026-07-03 19:30:00,734 - Yiddish24Scraper - INFO - Yiddish24 scraper initialized: data_id='latest', page_limit=100, total_pages=10, debug_html=True, include_terms=55, exclude_terms=0

2026-07-03 19:30:00,736 - AhBlickLiveScraper - INFO - AhBlickLive scraper initialized: urls=1, max_items=100, include_terms=49, exclude_terms=0

2026-07-03 19:30:00,740 - TorahAnytimeScraper - INFO - File downloader initialized with 53 terms, prefix 'TA', and file type organization enabled

2026-07-03 19:30:00,741 - TorahAnytimeScraper - INFO - TorahAnytime scraper initialized: limit=400, offset=0, project_id=1, include_terms=53, exclude_terms=10

2026-07-03 19:30:00,743 - BatorahScraper - INFO - File downloader initialized with 19 terms, prefix 'BT', and file type organization enabled

2026-07-03 19:30:00,743 - BatorahScraper - INFO - Batorah scraper initialized: api_urls=2, page_size=100, include_terms=19, exclude_terms=1

2026-07-03 19:30:00,744 - KolHalashonScraper - INFO - Kol Halashon scraper initialized: base_url=https://www2.kolhalashon.com, max_shiurim=50, fetch_all_speakers=True, headless=False, exclude_filters=1, include_filters=0

2026-07-03 19:30:00,744 - __main__ - INFO - Running 3 scrapers concurrently...

2026-07-03 19:30:00,744 - GitHubScraper - INFO - Searching GitHub for terms: ['paperclip', '"Hermes Agent"', 'multica']

2026-07-03 19:30:00,840 - LinkedInFeedScraper - INFO - Fetching LinkedIn feed (25 posts)

2026-07-03 19:30:06,542 - LinkedInFeedScraper - ERROR - LinkedIn feed fetch failed: Unauthorized: LinkedIn session cookies are expired or invalid.

2026-07-03 19:30:06,549 - LinkedInFeedScraper - INFO - LinkedIn Feed scrape complete. Found 0 new posts.

2026-07-03 19:30:07,418 - httpx - INFO - HTTP Request: GET https://api.github.com/users/abeperl/received_events?per_page=100&page=1 "HTTP/1.1 200 OK"

2026-07-03 19:30:08,133 - httpx - INFO - HTTP Request: GET https://api.github.com/search/repositories?q=paperclip+in%3Aname%2Cdescription%2Creadme+stars%3A%3E150&sort=updated&order=desc&per_page=20 "HTTP/1.1 200 OK"

2026-07-03 19:30:08,140 - GitHubScraper - INFO - Found 243 GitHub repositories for term: paperclip

2026-07-03 19:30:08,732 - httpx - INFO - HTTP Request: GET https://api.github.com/users/abeperl/received_events?per_page=100&page=2 "HTTP/1.1 200 OK"

2026-07-03 19:30:09,381 - httpx - INFO - HTTP Request: GET https://api.github.com/search/repositories?q=%22Hermes+Agent%22+in%3Aname%2Cdescription%2Creadme+stars%3A%3E150&sort=updated&order=desc&per_page=20 "HTTP/1.1 200 OK"

2026-07-03 19:30:09,385 - GitHubScraper - INFO - Found 420 GitHub repositories for term: "Hermes Agent"

2026-07-03 19:30:10,391 - httpx - INFO - HTTP Request: GET https://api.github.com/search/repositories?q=multica+in%3Aname%2Cdescription%2Creadme+stars%3A%3E150&sort=updated&order=desc&per_page=20 "HTTP/1.1 200 OK"

2026-07-03 19:30:10,484 - GitHubScraper - INFO - Found 56 GitHub repositories for term: multica

2026-07-03 19:30:10,633 - GitHubScraper - INFO - Total GitHub results found: 0

2026-07-03 19:30:11,004 - httpx - INFO - HTTP Request: GET https://api.github.com/users/abeperl/received_events?per_page=100&page=3 "HTTP/1.1 200 OK"

2026-07-03 19:30:11,873 - GitHubScraper - INFO - GitHub feed: 300 fetched across pages, 1 new events for abeperl

2026-07-03 19:30:11,874 - __main__ - INFO - Scraper github completed: 0 results

2026-07-03 19:30:11,874 - __main__ - INFO - Scraper github_feed completed: 1 results

2026-07-03 19:30:11,874 - __main__ - INFO - Scraper linkedin_feed completed: 0 results

2026-07-03 19:30:11,875 - __main__ - WARNING - LinkedIn Feed: 22 consecutive failures (0 results)

2026-07-03 19:30:12,642 - social_media_monitor.email.sender - INFO - Email sent successfully at 2026-07-03 19:30:12.642685

2026-07-03 19:30:12,643 - __main__ - INFO - Email sent successfully

2026-07-03 19:30:12,643 - __main__ - INFO - Cache stats: 0 items in memory, 0 files on disk

2026-07-03 19:30:12,643 - __main__ - INFO - Monitor complete. Found 1 total results: 1 new github feed results.

2026-07-03 19:30:12,643 - __main__ - INFO - === Social Media Monitor v2.0 Complete ===

2026-07-03 19:30:12,643 - social_media_monitor.database.connection - INFO - Closing all database connections

2026-07-03 19:30:12,932 - social_media_monitor.database.connection - INFO - Closed 5 database connections

2026-07-03 19:30:12,932 - __main__ - INFO - Closed all database connections

2026-07-03 19:30:12,934 - social_media_monitor.database.connection - INFO - Closing all database connections

2026-07-03 19:30:12,934 - social_media_monitor.database.connection - INFO - Closed 0 database connections

2026-07-03 20:04:39,282 - social_media_monitor.utils.logging_config - INFO - Posts log file: /mnt/data/openclaw-shared/home/shared/Social_Media_Monitor/logs/social_media_monitor_posts_2026-07-03.log

2026-07-03 20:04:39,282 - social_media_monitor.utils.logging_config - INFO - Downloads log file: /mnt/data/openclaw-shared/home/shared/Social_Media_Monitor/logs/social_media_monitor_downloads_2026-07-03.log

2026-07-03 20:04:39,282 - social_media_monitor.utils.logging_config - INFO - Logging configured with level INFO, log file: /mnt/data/openclaw-shared/home/shared/Social_Media_Monitor/logs/social_media_monitor_2026-07-03.log

2026-07-03 20:04:39,282 - __main__ - INFO - === Social Media Monitor v2.0 (Async) Started ===

2026-07-03 20:04:39,313 - social_media_monitor.config.settings - INFO - Configuration loaded from /home/openclaw/Social_Media_Monitor/config.yaml

2026-07-03 20:04:39,314 - __main__ - INFO - Configuration loaded and validated successfully

2026-07-03 20:04:39,316 - social_media_monitor.database.connection - INFO - Database connection pool initialized with 5 connections

2026-07-03 20:04:39,316 - social_media_monitor.database.connection - INFO - add_speaker_filters column already exists

2026-07-03 20:04:39,330 - social_media_monitor.database.connection - INFO - Database setup completed successfully

2026-07-03 20:04:39,330 - __main__ - INFO - Database initialized at: /mnt/data/openclaw-shared/home/shared/Social_Media_Monitor/social_media_monitor.db

2026-07-03 20:04:39,332 - social_media_monitor.scrapers.reddit_scraper - INFO - Reddit API initialized successfully

2026-07-03 20:04:39,333 - RSSFeedScraper - INFO - RSS scraper initialized: feeds=2, html_urls=0 (0 with custom selectors), include_terms=0, exclude_terms=0

2026-07-03 20:04:39,333 - __main__ - INFO - RSS scraper initialized and enabled

2026-07-03 20:04:39,335 - Yiddish24Scraper - INFO - File downloader initialized with 55 terms, prefix 'Y2', and file type organization enabled

2026-07-03 20:04:39,335 - Yiddish24Scraper - INFO - Yiddish24 scraper initialized: data_id='latest', page_limit=100, total_pages=10, debug_html=True, include_terms=55, exclude_terms=0

2026-07-03 20:04:39,335 - __main__ - INFO - Yiddish24 scraper initialized and enabled

2026-07-03 20:04:39,338 - AhBlickLiveScraper - INFO - AhBlickLive scraper initialized: urls=1, max_items=100, include_terms=49, exclude_terms=0

2026-07-03 20:04:39,342 - TorahAnytimeScraper - INFO - File downloader initialized with 53 terms, prefix 'TA', and file type organization enabled

2026-07-03 20:04:39,343 - TorahAnytimeScraper - INFO - TorahAnytime scraper initialized: limit=400, offset=0, project_id=1, include_terms=53, exclude_terms=10

2026-07-03 20:04:39,345 - BatorahScraper - INFO - File downloader initialized with 19 terms, prefix 'BT', and file type organization enabled

2026-07-03 20:04:39,345 - BatorahScraper - INFO - Batorah scraper initialized: api_urls=2, page_size=100, include_terms=19, exclude_terms=1

2026-07-03 20:04:39,345 - KolHalashonScraper - INFO - Kol Halashon scraper initialized: base_url=https://www2.kolhalashon.com, max_shiurim=50, fetch_all_speakers=True, headless=False, exclude_filters=1, include_filters=0

2026-07-03 20:04:39,345 - __main__ - INFO - Ivelt scraper initialized and enabled

2026-07-03 20:04:39,345 - __main__ - INFO - Running 3 scrapers concurrently...

2026-07-03 20:04:39,345 - Yiddish24Scraper - INFO - Starting Yiddish24 scraping (max 10 pages, limit 100 items)

2026-07-03 20:04:39,437 - Yiddish24Scraper - INFO - Fetching Yiddish24 page 1 with data_id='latest', page_limit=100

2026-07-03 20:04:39,442 - IveltScraper - INFO - Loaded 181 include, 0 exclude filters from DB topic_filters

2026-07-03 20:04:39,442 - IveltScraper - INFO - Fetching ivelt active topics from https://www.ivelt.com/forum/search.php?search_id=active_topics...

2026-07-03 20:04:39,444 - RSSFeedScraper - INFO - Fetching RSS feed: https://www.theyeshivaworld.com/feed

2026-07-03 20:04:39,703 - RSSFeedScraper - INFO - Fetching RSS feed: https://www.thegatewaypundit.com/feed/

2026-07-03 20:04:40,168 - RSSFeedScraper - INFO - Total RSS/HTML results: 4

2026-07-03 20:04:43,529 - httpx - INFO - HTTP Request: POST https://www.yiddish24.com/ajax/cat_pagination.php "HTTP/1.1 200 OK"

2026-07-03 20:04:43,751 - Yiddish24Scraper - INFO - Received response with 443080 characters

2026-07-03 20:04:43,754 - Yiddish24Scraper - INFO - Extracted HTML from JSON result field (289739 chars)

2026-07-03 20:04:43,757 - Yiddish24Scraper - INFO - Saved HTML debug file to: /mnt/data/openclaw-shared/home/shared/Social_Media_Monitor/debug/yiddish24_page1_response.html

2026-07-03 20:04:43,840 - Yiddish24Scraper - INFO - Found 100 potential article elements on page 1

2026-07-03 20:04:43,879 - Yiddish24Scraper - INFO - Page 1 extraction: 100 successful, 0 failed out of 100 elements

2026-07-03 20:04:43,879 - Yiddish24Scraper - INFO - Parsed 100 articles from HTML on page 1

2026-07-03 20:04:44,543 - Yiddish24Scraper - INFO - Page 1 summary: 6 new out of 100 total articles

2026-07-03 20:04:44,543 - Yiddish24Scraper - INFO - Page 1: 6/100 new items

2026-07-03 20:04:44,543 - Yiddish24Scraper - INFO - Low new item ratio (6.0%), stopping pagination

2026-07-03 20:04:44,544 - Yiddish24Scraper - INFO - Total Yiddish24 results found: 6

2026-07-03 20:04:51,536 - IveltScraper - INFO - Found 50 active topics on ivelt

2026-07-03 20:05:05,928 - IveltScraper - INFO - Filtering post containing exclude term: '*ביטע מאכט זיכער בלויז צו באריכטן קראנטע נייעס*'

2026-07-03 20:05:11,916 - IveltScraper - INFO - ivelt scrape complete. Checked 4 topics.

2026-07-03 20:05:11,916 - IveltScraper - INFO - All forums scrape complete. Found 9 new posts.

2026-07-03 20:05:11,916 - __main__ - INFO - Scraper rss completed: 4 results

2026-07-03 20:05:11,916 - __main__ - INFO - Scraper yiddish24 completed: 6 results

2026-07-03 20:05:11,916 - __main__ - INFO - Scraper ivelt completed: 9 results

2026-07-03 20:05:22,067 - social_media_monitor.email.sender - INFO - Email sent successfully at 2026-07-03 20:05:22.067079

2026-07-03 20:05:22,067 - __main__ - INFO - Email sent successfully

2026-07-03 20:05:22,067 - __main__ - INFO - Cache stats: 0 items in memory, 0 files on disk

2026-07-03 20:05:22,068 - __main__ - INFO - Monitor complete. Found 19 total results: 4 new rss results, 6 new yiddish24 results, 9 new ivelt results.

2026-07-03 20:05:22,068 - __main__ - INFO - === Social Media Monitor v2.0 Complete ===

2026-07-03 20:05:22,068 - social_media_monitor.database.connection - INFO - Closing all database connections

2026-07-03 20:05:22,135 - social_media_monitor.database.connection - INFO - Closed 5 database connections

2026-07-03 20:05:22,136 - __main__ - INFO - Closed all database connections

2026-07-03 20:05:22,138 - social_media_monitor.database.connection - INFO - Closing all database connections

2026-07-03 20:05:22,138 - social_media_monitor.database.connection - INFO - Closed 0 database connections

2026-07-03 20:30:00,613 - social_media_monitor.utils.logging_config - INFO - Posts log file: /mnt/data/openclaw-shared/home/shared/Social_Media_Monitor/logs/social_media_monitor_posts_2026-07-03.log

2026-07-03 20:30:00,613 - social_media_monitor.utils.logging_config - INFO - Downloads log file: /mnt/data/openclaw-shared/home/shared/Social_Media_Monitor/logs/social_media_monitor_downloads_2026-07-03.log

2026-07-03 20:30:00,613 - social_media_monitor.utils.logging_config - INFO - Logging configured with level INFO, log file: /mnt/data/openclaw-shared/home/shared/Social_Media_Monitor/logs/social_media_monitor_2026-07-03.log

2026-07-03 20:30:00,613 - __main__ - INFO - === Social Media Monitor v2.0 (Async) Started ===

2026-07-03 20:30:00,643 - social_media_monitor.config.settings - INFO - Configuration loaded from /home/openclaw/Social_Media_Monitor/config.yaml

2026-07-03 20:30:00,643 - __main__ - INFO - Configuration loaded and validated successfully

2026-07-03 20:30:00,645 - social_media_monitor.database.connection - INFO - Database connection pool initialized with 5 connections

2026-07-03 20:30:00,646 - social_media_monitor.database.connection - INFO - add_speaker_filters column already exists

2026-07-03 20:30:00,659 - social_media_monitor.database.connection - INFO - Database setup completed successfully

2026-07-03 20:30:00,659 - __main__ - INFO - Database initialized at: /mnt/data/openclaw-shared/home/shared/Social_Media_Monitor/social_media_monitor.db

2026-07-03 20:30:00,662 - social_media_monitor.scrapers.reddit_scraper - INFO - Reddit API initialized successfully

2026-07-03 20:30:00,662 - __main__ - INFO - GitHub scraper initialized and enabled

2026-07-03 20:30:00,662 - __main__ - INFO - GitHub feed scraper initialized and enabled

2026-07-03 20:30:00,663 - RSSFeedScraper - INFO - RSS scraper initialized: feeds=2, html_urls=0 (0 with custom selectors), include_terms=0, exclude_terms=0

2026-07-03 20:30:00,663 - __main__ - INFO - LinkedIn Feed scraper initialized and enabled

2026-07-03 20:30:00,665 - Yiddish24Scraper - INFO - File downloader initialized with 55 terms, prefix 'Y2', and file type organization enabled

2026-07-03 20:30:00,665 - Yiddish24Scraper - INFO - Yiddish24 scraper initialized: data_id='latest', page_limit=100, total_pages=10, debug_html=True, include_terms=55, exclude_terms=0

2026-07-03 20:30:00,668 - AhBlickLiveScraper - INFO - AhBlickLive scraper initialized: urls=1, max_items=100, include_terms=49, exclude_terms=0

2026-07-03 20:30:00,672 - TorahAnytimeScraper - INFO - File downloader initialized with 53 terms, prefix 'TA', and file type organization enabled

2026-07-03 20:30:00,672 - TorahAnytimeScraper - INFO - TorahAnytime scraper initialized: limit=400, offset=0, project_id=1, include_terms=53, exclude_terms=10

2026-07-03 20:30:00,674 - BatorahScraper - INFO - File downloader initialized with 19 terms, prefix 'BT', and file type organization enabled

2026-07-03 20:30:00,674 - BatorahScraper - INFO - Batorah scraper initialized: api_urls=2, page_size=100, include_terms=19, exclude_terms=1

2026-07-03 20:30:00,674 - KolHalashonScraper - INFO - Kol Halashon scraper initialized: base_url=https://www2.kolhalashon.com, max_shiurim=50, fetch_all_speakers=True, headless=False, exclude_filters=1, include_filters=0

2026-07-03 20:30:00,675 - __main__ - INFO - Running 3 scrapers concurrently...

2026-07-03 20:30:00,675 - GitHubScraper - INFO - Searching GitHub for terms: ['paperclip', '"Hermes Agent"', 'multica']

2026-07-03 20:30:00,769 - LinkedInFeedScraper - INFO - Fetching LinkedIn feed (25 posts)

2026-07-03 20:30:04,324 - LinkedInFeedScraper - ERROR - LinkedIn feed fetch failed: Unauthorized: LinkedIn session cookies are expired or invalid.

2026-07-03 20:30:04,332 - LinkedInFeedScraper - INFO - LinkedIn Feed scrape complete. Found 0 new posts.

2026-07-03 20:30:05,323 - httpx - INFO - HTTP Request: GET https://api.github.com/search/repositories?q=paperclip+in%3Aname%2Cdescription%2Creadme+stars%3A%3E150&sort=updated&order=desc&per_page=20 "HTTP/1.1 200 OK"

2026-07-03 20:30:05,327 - httpx - INFO - HTTP Request: GET https://api.github.com/users/abeperl/received_events?per_page=100&page=1 "HTTP/1.1 200 OK"

2026-07-03 20:30:06,071 - GitHubScraper - INFO - Found 243 GitHub repositories for term: paperclip

2026-07-03 20:30:06,889 - httpx - INFO - HTTP Request: GET https://api.github.com/users/abeperl/received_events?per_page=100&page=2 "HTTP/1.1 200 OK"

2026-07-03 20:30:07,569 - httpx - INFO - HTTP Request: GET https://api.github.com/search/repositories?q=%22Hermes+Agent%22+in%3Aname%2Cdescription%2Creadme+stars%3A%3E150&sort=updated&order=desc&per_page=20 "HTTP/1.1 200 OK"

2026-07-03 20:30:07,573 - GitHubScraper - INFO - Found 420 GitHub repositories for term: "Hermes Agent"

2026-07-03 20:30:08,547 - httpx - INFO - HTTP Request: GET https://api.github.com/search/repositories?q=multica+in%3Aname%2Cdescription%2Creadme+stars%3A%3E150&sort=updated&order=desc&per_page=20 "HTTP/1.1 200 OK"

2026-07-03 20:30:08,552 - GitHubScraper - INFO - Found 56 GitHub repositories for term: multica

2026-07-03 20:30:08,678 - GitHubScraper - INFO - Total GitHub results found: 0

2026-07-03 20:30:09,267 - httpx - INFO - HTTP Request: GET https://api.github.com/users/abeperl/received_events?per_page=100&page=3 "HTTP/1.1 200 OK"

2026-07-03 20:30:10,124 - GitHubScraper - INFO - GitHub feed: 300 fetched across pages, 0 new events for abeperl

2026-07-03 20:30:10,125 - __main__ - INFO - Scraper github completed: 0 results

2026-07-03 20:30:10,125 - __main__ - INFO - Scraper github_feed completed: 0 results

2026-07-03 20:30:10,125 - __main__ - INFO - Scraper linkedin_feed completed: 0 results

2026-07-03 20:30:10,126 - __main__ - WARNING - LinkedIn Feed: 23 consecutive failures (0 results)

2026-07-03 20:30:10,126 - __main__ - INFO - No new results found across all enabled sources

2026-07-03 20:30:10,126 - __main__ - INFO - Cache stats: 0 items in memory, 0 files on disk

2026-07-03 20:30:10,126 - __main__ - INFO - Monitor complete. No new results found across all sources.

2026-07-03 20:30:10,126 - __main__ - INFO - === Social Media Monitor v2.0 Complete ===

2026-07-03 20:30:10,126 - social_media_monitor.database.connection - INFO - Closing all database connections

2026-07-03 20:30:10,216 - social_media_monitor.database.connection - INFO - Closed 5 database connections

2026-07-03 20:30:10,216 - __main__ - INFO - Closed all database connections

2026-07-03 20:30:10,218 - social_media_monitor.database.connection - INFO - Closing all database connections

2026-07-03 20:30:10,218 - social_media_monitor.database.connection - INFO - Closed 0 database connections

2026-07-03 21:04:39,281 - social_media_monitor.utils.logging_config - INFO - Posts log file: /mnt/data/openclaw-shared/home/shared/Social_Media_Monitor/logs/social_media_monitor_posts_2026-07-03.log

2026-07-03 21:04:39,281 - social_media_monitor.utils.logging_config - INFO - Downloads log file: /mnt/data/openclaw-shared/home/shared/Social_Media_Monitor/logs/social_media_monitor_downloads_2026-07-03.log

2026-07-03 21:04:39,281 - social_media_monitor.utils.logging_config - INFO - Logging configured with level INFO, log file: /mnt/data/openclaw-shared/home/shared/Social_Media_Monitor/logs/social_media_monitor_2026-07-03.log

2026-07-03 21:04:39,281 - __main__ - INFO - === Social Media Monitor v2.0 (Async) Started ===

2026-07-03 21:04:39,315 - social_media_monitor.config.settings - INFO - Configuration loaded from /home/openclaw/Social_Media_Monitor/config.yaml

2026-07-03 21:04:39,315 - __main__ - INFO - Configuration loaded and validated successfully

2026-07-03 21:04:39,317 - social_media_monitor.database.connection - INFO - Database connection pool initialized with 5 connections

2026-07-03 21:04:39,318 - social_media_monitor.database.connection - INFO - add_speaker_filters column already exists

2026-07-03 21:04:39,335 - social_media_monitor.database.connection - INFO - Database setup completed successfully

2026-07-03 21:04:39,335 - __main__ - INFO - Database initialized at: /mnt/data/openclaw-shared/home/shared/Social_Media_Monitor/social_media_monitor.db

2026-07-03 21:04:39,381 - social_media_monitor.scrapers.reddit_scraper - INFO - Reddit API initialized successfully

2026-07-03 21:04:39,382 - RSSFeedScraper - INFO - RSS scraper initialized: feeds=2, html_urls=0 (0 with custom selectors), include_terms=0, exclude_terms=0

2026-07-03 21:04:39,382 - __main__ - INFO - RSS scraper initialized and enabled

2026-07-03 21:04:39,386 - Yiddish24Scraper - INFO - File downloader initialized with 55 terms, prefix 'Y2', and file type organization enabled

2026-07-03 21:04:39,386 - Yiddish24Scraper - INFO - Yiddish24 scraper initialized: data_id='latest', page_limit=100, total_pages=10, debug_html=True, include_terms=55, exclude_terms=0

2026-07-03 21:04:39,386 - __main__ - INFO - Yiddish24 scraper initialized and enabled

2026-07-03 21:04:39,388 - AhBlickLiveScraper - INFO - AhBlickLive scraper initialized: urls=1, max_items=100, include_terms=49, exclude_terms=0

2026-07-03 21:04:39,393 - TorahAnytimeScraper - INFO - File downloader initialized with 53 terms, prefix 'TA', and file type organization enabled

2026-07-03 21:04:39,393 - TorahAnytimeScraper - INFO - TorahAnytime scraper initialized: limit=400, offset=0, project_id=1, include_terms=53, exclude_terms=10

2026-07-03 21:04:39,395 - BatorahScraper - INFO - File downloader initialized with 19 terms, prefix 'BT', and file type organization enabled

2026-07-03 21:04:39,396 - BatorahScraper - INFO - Batorah scraper initialized: api_urls=2, page_size=100, include_terms=19, exclude_terms=1

2026-07-03 21:04:39,396 - KolHalashonScraper - INFO - Kol Halashon scraper initialized: base_url=https://www2.kolhalashon.com, max_shiurim=50, fetch_all_speakers=True, headless=False, exclude_filters=1, include_filters=0

2026-07-03 21:04:39,396 - __main__ - INFO - Ivelt scraper initialized and enabled

2026-07-03 21:04:39,396 - __main__ - INFO - Running 3 scrapers concurrently...

2026-07-03 21:04:39,396 - Yiddish24Scraper - INFO - Starting Yiddish24 scraping (max 10 pages, limit 100 items)

2026-07-03 21:04:39,491 - Yiddish24Scraper - INFO - Fetching Yiddish24 page 1 with data_id='latest', page_limit=100

2026-07-03 21:04:39,497 - IveltScraper - INFO - Loaded 181 include, 0 exclude filters from DB topic_filters

2026-07-03 21:04:39,497 - IveltScraper - INFO - Fetching ivelt active topics from https://www.ivelt.com/forum/search.php?search_id=active_topics...

2026-07-03 21:04:39,498 - RSSFeedScraper - INFO - Fetching RSS feed: https://www.theyeshivaworld.com/feed

2026-07-03 21:04:39,795 - RSSFeedScraper - INFO - Fetching RSS feed: https://www.thegatewaypundit.com/feed/

2026-07-03 21:04:40,172 - RSSFeedScraper - INFO - Total RSS/HTML results: 3

2026-07-03 21:04:43,810 - httpx - INFO - HTTP Request: POST https://www.yiddish24.com/ajax/cat_pagination.php "HTTP/1.1 200 OK"

2026-07-03 21:04:44,021 - Yiddish24Scraper - INFO - Received response with 447086 characters

2026-07-03 21:04:44,024 - Yiddish24Scraper - INFO - Extracted HTML from JSON result field (293033 chars)

2026-07-03 21:04:44,027 - Yiddish24Scraper - INFO - Saved HTML debug file to: /mnt/data/openclaw-shared/home/shared/Social_Media_Monitor/debug/yiddish24_page1_response.html

2026-07-03 21:04:44,112 - Yiddish24Scraper - INFO - Found 100 potential article elements on page 1

2026-07-03 21:04:44,152 - Yiddish24Scraper - INFO - Page 1 extraction: 100 successful, 0 failed out of 100 elements

2026-07-03 21:04:44,152 - Yiddish24Scraper - INFO - Parsed 100 articles from HTML on page 1

2026-07-03 21:04:44,827 - Yiddish24Scraper - INFO - Page 1 summary: 3 new out of 100 total articles

2026-07-03 21:04:44,827 - Yiddish24Scraper - INFO - Page 1: 3/100 new items

2026-07-03 21:04:44,827 - Yiddish24Scraper - INFO - Low new item ratio (3.0%), stopping pagination

2026-07-03 21:04:44,828 - Yiddish24Scraper - INFO - Total Yiddish24 results found: 3

2026-07-03 21:04:51,605 - IveltScraper - INFO - Found 50 active topics on ivelt

2026-07-03 21:05:42,541 - IveltScraper - INFO - Filtering post containing exclude term: '*ביטע מאכט זיכער בלויז צו באריכטן קראנטע נייעס*'

2026-07-03 21:05:43,160 - IveltScraper - INFO - ivelt scrape complete. Checked 8 topics.

2026-07-03 21:05:43,160 - IveltScraper - INFO - All forums scrape complete. Found 47 new posts.

2026-07-03 21:05:43,160 - __main__ - INFO - Scraper rss completed: 3 results

2026-07-03 21:05:43,161 - __main__ - INFO - Scraper yiddish24 completed: 3 results

2026-07-03 21:05:43,161 - __main__ - INFO - Scraper ivelt completed: 47 results

2026-07-03 21:05:44,085 - social_media_monitor.email.sender - INFO - Email sent successfully at 2026-07-03 21:05:44.085327

2026-07-03 21:05:44,085 - __main__ - INFO - Email sent successfully

2026-07-03 21:05:44,086 - __main__ - INFO - Cache stats: 0 items in memory, 0 files on disk

2026-07-03 21:05:44,086 - __main__ - INFO - Monitor complete. Found 53 total results: 3 new rss results, 3 new yiddish24 results, 47 new ivelt results.

2026-07-03 21:05:44,086 - __main__ - INFO - === Social Media Monitor v2.0 Complete ===

2026-07-03 21:05:44,086 - social_media_monitor.database.connection - INFO - Closing all database connections

2026-07-03 21:05:44,234 - social_media_monitor.database.connection - INFO - Closed 5 database connections

2026-07-03 21:05:44,234 - __main__ - INFO - Closed all database connections

2026-07-03 21:05:44,236 - social_media_monitor.database.connection - INFO - Closing all database connections

2026-07-03 21:05:44,236 - social_media_monitor.database.connection - INFO - Closed 0 database connections

2026-07-03 21:30:00,653 - social_media_monitor.utils.logging_config - INFO - Posts log file: /mnt/data/openclaw-shared/home/shared/Social_Media_Monitor/logs/social_media_monitor_posts_2026-07-03.log

2026-07-03 21:30:00,653 - social_media_monitor.utils.logging_config - INFO - Downloads log file: /mnt/data/openclaw-shared/home/shared/Social_Media_Monitor/logs/social_media_monitor_downloads_2026-07-03.log

2026-07-03 21:30:00,653 - social_media_monitor.utils.logging_config - INFO - Logging configured with level INFO, log file: /mnt/data/openclaw-shared/home/shared/Social_Media_Monitor/logs/social_media_monitor_2026-07-03.log

2026-07-03 21:30:00,653 - __main__ - INFO - === Social Media Monitor v2.0 (Async) Started ===

2026-07-03 21:30:00,683 - social_media_monitor.config.settings - INFO - Configuration loaded from /home/openclaw/Social_Media_Monitor/config.yaml

2026-07-03 21:30:00,683 - __main__ - INFO - Configuration loaded and validated successfully

2026-07-03 21:30:00,685 - social_media_monitor.database.connection - INFO - Database connection pool initialized with 5 connections

2026-07-03 21:30:00,686 - social_media_monitor.database.connection - INFO - add_speaker_filters column already exists

2026-07-03 21:30:00,699 - social_media_monitor.database.connection - INFO - Database setup completed successfully

2026-07-03 21:30:00,699 - __main__ - INFO - Database initialized at: /mnt/data/openclaw-shared/home/shared/Social_Media_Monitor/social_media_monitor.db

2026-07-03 21:30:00,702 - social_media_monitor.scrapers.reddit_scraper - INFO - Reddit API initialized successfully

2026-07-03 21:30:00,702 - __main__ - INFO - GitHub scraper initialized and enabled

2026-07-03 21:30:00,702 - __main__ - INFO - GitHub feed scraper initialized and enabled

2026-07-03 21:30:00,702 - RSSFeedScraper - INFO - RSS scraper initialized: feeds=2, html_urls=0 (0 with custom selectors), include_terms=0, exclude_terms=0

2026-07-03 21:30:00,703 - __main__ - INFO - LinkedIn Feed scraper initialized and enabled

2026-07-03 21:30:00,705 - Yiddish24Scraper - INFO - File downloader initialized with 55 terms, prefix 'Y2', and file type organization enabled

2026-07-03 21:30:00,705 - Yiddish24Scraper - INFO - Yiddish24 scraper initialized: data_id='latest', page_limit=100, total_pages=10, debug_html=True, include_terms=55, exclude_terms=0

2026-07-03 21:30:00,708 - AhBlickLiveScraper - INFO - AhBlickLive scraper initialized: urls=1, max_items=100, include_terms=49, exclude_terms=0

2026-07-03 21:30:00,712 - TorahAnytimeScraper - INFO - File downloader initialized with 53 terms, prefix 'TA', and file type organization enabled

2026-07-03 21:30:00,712 - TorahAnytimeScraper - INFO - TorahAnytime scraper initialized: limit=400, offset=0, project_id=1, include_terms=53, exclude_terms=10

2026-07-03 21:30:00,714 - BatorahScraper - INFO - File downloader initialized with 19 terms, prefix 'BT', and file type organization enabled

2026-07-03 21:30:00,714 - BatorahScraper - INFO - Batorah scraper initialized: api_urls=2, page_size=100, include_terms=19, exclude_terms=1

2026-07-03 21:30:00,714 - KolHalashonScraper - INFO - Kol Halashon scraper initialized: base_url=https://www2.kolhalashon.com, max_shiurim=50, fetch_all_speakers=True, headless=False, exclude_filters=1, include_filters=0

2026-07-03 21:30:00,714 - __main__ - INFO - Running 3 scrapers concurrently...

2026-07-03 21:30:00,715 - GitHubScraper - INFO - Searching GitHub for terms: ['paperclip', '"Hermes Agent"', 'multica']

2026-07-03 21:30:00,813 - LinkedInFeedScraper - INFO - Fetching LinkedIn feed (25 posts)

2026-07-03 21:30:04,586 - LinkedInFeedScraper - ERROR - LinkedIn feed fetch failed: Unauthorized: LinkedIn session cookies are expired or invalid.

2026-07-03 21:30:04,596 - LinkedInFeedScraper - INFO - LinkedIn Feed scrape complete. Found 0 new posts.

2026-07-03 21:30:05,477 - httpx - INFO - HTTP Request: GET https://api.github.com/users/abeperl/received_events?per_page=100&page=1 "HTTP/1.1 200 OK"

2026-07-03 21:30:06,185 - httpx - INFO - HTTP Request: GET https://api.github.com/search/repositories?q=paperclip+in%3Aname%2Cdescription%2Creadme+stars%3A%3E150&sort=updated&order=desc&per_page=20 "HTTP/1.1 200 OK"

2026-07-03 21:30:06,189 - GitHubScraper - INFO - Found 243 GitHub repositories for term: paperclip

2026-07-03 21:30:06,814 - httpx - INFO - HTTP Request: GET https://api.github.com/users/abeperl/received_events?per_page=100&page=2 "HTTP/1.1 200 OK"

2026-07-03 21:30:07,613 - httpx - INFO - HTTP Request: GET https://api.github.com/search/repositories?q=%22Hermes+Agent%22+in%3Aname%2Cdescription%2Creadme+stars%3A%3E150&sort=updated&order=desc&per_page=20 "HTTP/1.1 200 OK"

2026-07-03 21:30:07,617 - GitHubScraper - INFO - Found 420 GitHub repositories for term: "Hermes Agent"

2026-07-03 21:30:08,502 - httpx - INFO - HTTP Request: GET https://api.github.com/search/repositories?q=multica+in%3Aname%2Cdescription%2Creadme+stars%3A%3E150&sort=updated&order=desc&per_page=20 "HTTP/1.1 200 OK"

2026-07-03 21:30:08,505 - GitHubScraper - INFO - Found 56 GitHub repositories for term: multica

2026-07-03 21:30:08,628 - GitHubScraper - INFO - Total GitHub results found: 0

2026-07-03 21:30:09,520 - httpx - INFO - HTTP Request: GET https://api.github.com/users/abeperl/received_events?per_page=100&page=3 "HTTP/1.1 200 OK"

2026-07-03 21:30:10,201 - GitHubScraper - INFO - GitHub feed: 300 fetched across pages, 0 new events for abeperl

2026-07-03 21:30:10,202 - __main__ - INFO - Scraper github completed: 0 results

2026-07-03 21:30:10,202 - __main__ - INFO - Scraper github_feed completed: 0 results

2026-07-03 21:30:10,202 - __main__ - INFO - Scraper linkedin_feed completed: 0 results

2026-07-03 21:30:10,203 - __main__ - WARNING - LinkedIn Feed: 24 consecutive failures (0 results)

2026-07-03 21:30:10,944 - __main__ - INFO - Sent LinkedIn Feed failure alert email (24 failures)

2026-07-03 21:30:10,945 - __main__ - INFO - No new results found across all enabled sources

2026-07-03 21:30:10,945 - __main__ - INFO - Cache stats: 0 items in memory, 0 files on disk

2026-07-03 21:30:10,945 - __main__ - INFO - Monitor complete. No new results found across all sources.

2026-07-03 21:30:10,945 - __main__ - INFO - === Social Media Monitor v2.0 Complete ===

2026-07-03 21:30:10,945 - social_media_monitor.database.connection - INFO - Closing all database connections

2026-07-03 21:30:11,023 - social_media_monitor.database.connection - INFO - Closed 5 database connections

2026-07-03 21:30:11,023 - __main__ - INFO - Closed all database connections

2026-07-03 21:30:11,025 - social_media_monitor.database.connection - INFO - Closing all database connections

2026-07-03 21:30:11,025 - social_media_monitor.database.connection - INFO - Closed 0 database connections

2026-07-03 22:04:39,261 - social_media_monitor.utils.logging_config - INFO - Posts log file: /mnt/data/openclaw-shared/home/shared/Social_Media_Monitor/logs/social_media_monitor_posts_2026-07-03.log

2026-07-03 22:04:39,261 - social_media_monitor.utils.logging_config - INFO - Downloads log file: /mnt/data/openclaw-shared/home/shared/Social_Media_Monitor/logs/social_media_monitor_downloads_2026-07-03.log

2026-07-03 22:04:39,261 - social_media_monitor.utils.logging_config - INFO - Logging configured with level INFO, log file: /mnt/data/openclaw-shared/home/shared/Social_Media_Monitor/logs/social_media_monitor_2026-07-03.log

2026-07-03 22:04:39,261 - __main__ - INFO - === Social Media Monitor v2.0 (Async) Started ===

2026-07-03 22:04:39,292 - social_media_monitor.config.settings - INFO - Configuration loaded from /home/openclaw/Social_Media_Monitor/config.yaml

2026-07-03 22:04:39,292 - __main__ - INFO - Configuration loaded and validated successfully

2026-07-03 22:04:39,295 - social_media_monitor.database.connection - INFO - Database connection pool initialized with 5 connections

2026-07-03 22:04:39,295 - social_media_monitor.database.connection - INFO - add_speaker_filters column already exists

2026-07-03 22:04:39,309 - social_media_monitor.database.connection - INFO - Database setup completed successfully

2026-07-03 22:04:39,309 - __main__ - INFO - Database initialized at: /mnt/data/openclaw-shared/home/shared/Social_Media_Monitor/social_media_monitor.db

2026-07-03 22:04:39,311 - social_media_monitor.scrapers.reddit_scraper - INFO - Reddit API initialized successfully

2026-07-03 22:04:39,312 - RSSFeedScraper - INFO - RSS scraper initialized: feeds=2, html_urls=0 (0 with custom selectors), include_terms=0, exclude_terms=0

2026-07-03 22:04:39,312 - __main__ - INFO - RSS scraper initialized and enabled

2026-07-03 22:04:39,314 - Yiddish24Scraper - INFO - File downloader initialized with 55 terms, prefix 'Y2', and file type organization enabled

2026-07-03 22:04:39,314 - Yiddish24Scraper - INFO - Yiddish24 scraper initialized: data_id='latest', page_limit=100, total_pages=10, debug_html=True, include_terms=55, exclude_terms=0

2026-07-03 22:04:39,315 - __main__ - INFO - Yiddish24 scraper initialized and enabled

2026-07-03 22:04:39,317 - AhBlickLiveScraper - INFO - AhBlickLive scraper initialized: urls=1, max_items=100, include_terms=49, exclude_terms=0

2026-07-03 22:04:39,321 - TorahAnytimeScraper - INFO - File downloader initialized with 53 terms, prefix 'TA', and file type organization enabled

2026-07-03 22:04:39,322 - TorahAnytimeScraper - INFO - TorahAnytime scraper initialized: limit=400, offset=0, project_id=1, include_terms=53, exclude_terms=10

2026-07-03 22:04:39,324 - BatorahScraper - INFO - File downloader initialized with 19 terms, prefix 'BT', and file type organization enabled

2026-07-03 22:04:39,324 - BatorahScraper - INFO - Batorah scraper initialized: api_urls=2, page_size=100, include_terms=19, exclude_terms=1

2026-07-03 22:04:39,324 - KolHalashonScraper - INFO - Kol Halashon scraper initialized: base_url=https://www2.kolhalashon.com, max_shiurim=50, fetch_all_speakers=True, headless=False, exclude_filters=1, include_filters=0

2026-07-03 22:04:39,324 - __main__ - INFO - Ivelt scraper initialized and enabled

2026-07-03 22:04:39,324 - __main__ - INFO - Running 3 scrapers concurrently...

2026-07-03 22:04:39,324 - Yiddish24Scraper - INFO - Starting Yiddish24 scraping (max 10 pages, limit 100 items)

2026-07-03 22:04:39,411 - Yiddish24Scraper - INFO - Fetching Yiddish24 page 1 with data_id='latest', page_limit=100

2026-07-03 22:04:39,416 - IveltScraper - INFO - Loaded 181 include, 0 exclude filters from DB topic_filters

2026-07-03 22:04:39,416 - IveltScraper - INFO - Fetching ivelt active topics from https://www.ivelt.com/forum/search.php?search_id=active_topics...

2026-07-03 22:04:39,417 - RSSFeedScraper - INFO - Fetching RSS feed: https://www.theyeshivaworld.com/feed

2026-07-03 22:04:39,641 - RSSFeedScraper - INFO - Fetching RSS feed: https://www.thegatewaypundit.com/feed/

2026-07-03 22:04:40,131 - RSSFeedScraper - INFO - Total RSS/HTML results: 3

2026-07-03 22:04:43,413 - httpx - INFO - HTTP Request: POST https://www.yiddish24.com/ajax/cat_pagination.php "HTTP/1.1 200 OK"

2026-07-03 22:04:43,632 - Yiddish24Scraper - INFO - Received response with 457907 characters

2026-07-03 22:04:43,635 - Yiddish24Scraper - INFO - Extracted HTML from JSON result field (296858 chars)

2026-07-03 22:04:43,638 - Yiddish24Scraper - INFO - Saved HTML debug file to: /mnt/data/openclaw-shared/home/shared/Social_Media_Monitor/debug/yiddish24_page1_response.html

2026-07-03 22:04:43,725 - Yiddish24Scraper - INFO - Found 100 potential article elements on page 1

2026-07-03 22:04:43,765 - Yiddish24Scraper - INFO - Page 1 extraction: 100 successful, 0 failed out of 100 elements

2026-07-03 22:04:43,765 - Yiddish24Scraper - INFO - Parsed 100 articles from HTML on page 1

2026-07-03 22:04:43,779 - Yiddish24Scraper - INFO - Term 'שליט' matched in TITLE: 'הגה"צ ר' אלימלך בידערמאן שליט"א'

2026-07-03 22:04:43,779 - Yiddish24Scraper - INFO - Downloading 1 media file(s) for article: הגה"צ ר' אלימלך בידערמאן שליט"א

2026-07-03 22:04:44,279 - social_media_monitor.utils.file_downloader - INFO - Starting download from: https://cloudfront.yiddish24.com/1783116111_140_1783115881195_452_0_260703_575298341.mp3

2026-07-03 22:04:44,879 - social_media_monitor.utils.file_downloader - INFO - Successfully downloaded: Y2_1783116111_140_1783115881195_452_0_260703_575298341.mp3 (77.8 KB)

2026-07-03 22:04:44,879 - social_media_monitor.utils.file_downloader - INFO - Organized audio file to: /mnt/data/openclaw-shared/home/shared/Social_Media_Monitor/downloads/Audio (audio/mpeg)

2026-07-03 22:04:44,882 - social_media_monitor.utils.media_tagger - INFO - Tagged Y2_1783116111_140_1783115881195_452_0_260703_575298341.mp3: title, artist, album, source_url, genre, publisher

2026-07-03 22:04:44,883 - Yiddish24Scraper - INFO - Renamed downloaded file to include match term: Y2_1783116111_140_1783115881195_452_0_260703_575298341_שליט.mp3

2026-07-03 22:04:45,755 - Yiddish24Scraper - INFO - Page 1 summary: 7 new out of 100 total articles

2026-07-03 22:04:45,755 - Yiddish24Scraper - INFO - Page 1: 7/100 new items

2026-07-03 22:04:45,755 - Yiddish24Scraper - INFO - Low new item ratio (7.0%), stopping pagination

2026-07-03 22:04:45,756 - Yiddish24Scraper - INFO - Total Yiddish24 results found: 7

2026-07-03 22:04:51,602 - IveltScraper - INFO - Found 50 active topics on ivelt

2026-07-03 22:05:06,651 - IveltScraper - INFO - ivelt scrape complete. Checked 4 topics.

2026-07-03 22:05:06,651 - IveltScraper - INFO - All forums scrape complete. Found 4 new posts.

2026-07-03 22:05:06,651 - __main__ - INFO - Scraper rss completed: 3 results

2026-07-03 22:05:06,651 - __main__ - INFO - Scraper yiddish24 completed: 7 results

2026-07-03 22:05:06,651 - __main__ - INFO - Scraper ivelt completed: 4 results

2026-07-03 22:05:07,476 - social_media_monitor.email.sender - INFO - Email sent successfully at 2026-07-03 22:05:07.475994

2026-07-03 22:05:07,476 - __main__ - INFO - Email sent successfully

2026-07-03 22:05:07,476 - __main__ - INFO - Cache stats: 0 items in memory, 0 files on disk

2026-07-03 22:05:07,476 - __main__ - INFO - Monitor complete. Found 14 total results: 3 new rss results, 7 new yiddish24 results, 4 new ivelt results.

2026-07-03 22:05:07,476 - __main__ - INFO - === Social Media Monitor v2.0 Complete ===

2026-07-03 22:05:07,476 - social_media_monitor.database.connection - INFO - Closing all database connections

2026-07-03 22:05:07,567 - social_media_monitor.database.connection - INFO - Closed 5 database connections

2026-07-03 22:05:07,568 - __main__ - INFO - Closed all database connections

2026-07-03 22:05:07,570 - social_media_monitor.database.connection - INFO - Closing all database connections

2026-07-03 22:05:07,570 - social_media_monitor.database.connection - INFO - Closed 0 database connections

2026-07-03 22:30:00,652 - social_media_monitor.utils.logging_config - INFO - Posts log file: /mnt/data/openclaw-shared/home/shared/Social_Media_Monitor/logs/social_media_monitor_posts_2026-07-03.log

2026-07-03 22:30:00,653 - social_media_monitor.utils.logging_config - INFO - Downloads log file: /mnt/data/openclaw-shared/home/shared/Social_Media_Monitor/logs/social_media_monitor_downloads_2026-07-03.log

2026-07-03 22:30:00,653 - social_media_monitor.utils.logging_config - INFO - Logging configured with level INFO, log file: /mnt/data/openclaw-shared/home/shared/Social_Media_Monitor/logs/social_media_monitor_2026-07-03.log

2026-07-03 22:30:00,653 - __main__ - INFO - === Social Media Monitor v2.0 (Async) Started ===

2026-07-03 22:30:00,683 - social_media_monitor.config.settings - INFO - Configuration loaded from /home/openclaw/Social_Media_Monitor/config.yaml

2026-07-03 22:30:00,683 - __main__ - INFO - Configuration loaded and validated successfully

2026-07-03 22:30:00,685 - social_media_monitor.database.connection - INFO - Database connection pool initialized with 5 connections

2026-07-03 22:30:00,686 - social_media_monitor.database.connection - INFO - add_speaker_filters column already exists

2026-07-03 22:30:00,700 - social_media_monitor.database.connection - INFO - Database setup completed successfully

2026-07-03 22:30:00,700 - __main__ - INFO - Database initialized at: /mnt/data/openclaw-shared/home/shared/Social_Media_Monitor/social_media_monitor.db

2026-07-03 22:30:00,747 - social_media_monitor.scrapers.reddit_scraper - INFO - Reddit API initialized successfully

2026-07-03 22:30:00,747 - __main__ - INFO - GitHub scraper initialized and enabled

2026-07-03 22:30:00,747 - __main__ - INFO - GitHub feed scraper initialized and enabled

2026-07-03 22:30:00,748 - RSSFeedScraper - INFO - RSS scraper initialized: feeds=2, html_urls=0 (0 with custom selectors), include_terms=0, exclude_terms=0

2026-07-03 22:30:00,748 - __main__ - INFO - LinkedIn Feed scraper initialized and enabled

2026-07-03 22:30:00,751 - Yiddish24Scraper - INFO - File downloader initialized with 55 terms, prefix 'Y2', and file type organization enabled

2026-07-03 22:30:00,751 - Yiddish24Scraper - INFO - Yiddish24 scraper initialized: data_id='latest', page_limit=100, total_pages=10, debug_html=True, include_terms=55, exclude_terms=0

2026-07-03 22:30:00,754 - AhBlickLiveScraper - INFO - AhBlickLive scraper initialized: urls=1, max_items=100, include_terms=49, exclude_terms=0

2026-07-03 22:30:00,758 - TorahAnytimeScraper - INFO - File downloader initialized with 53 terms, prefix 'TA', and file type organization enabled

2026-07-03 22:30:00,758 - TorahAnytimeScraper - INFO - TorahAnytime scraper initialized: limit=400, offset=0, project_id=1, include_terms=53, exclude_terms=10

2026-07-03 22:30:00,760 - BatorahScraper - INFO - File downloader initialized with 19 terms, prefix 'BT', and file type organization enabled

2026-07-03 22:30:00,760 - BatorahScraper - INFO - Batorah scraper initialized: api_urls=2, page_size=100, include_terms=19, exclude_terms=1

2026-07-03 22:30:00,760 - KolHalashonScraper - INFO - Kol Halashon scraper initialized: base_url=https://www2.kolhalashon.com, max_shiurim=50, fetch_all_speakers=True, headless=False, exclude_filters=1, include_filters=0

2026-07-03 22:30:00,760 - __main__ - INFO - Running 3 scrapers concurrently...

2026-07-03 22:30:00,761 - GitHubScraper - INFO - Searching GitHub for terms: ['paperclip', '"Hermes Agent"', 'multica']

2026-07-03 22:30:00,858 - LinkedInFeedScraper - INFO - Fetching LinkedIn feed (25 posts)

2026-07-03 22:30:05,561 - LinkedInFeedScraper - ERROR - LinkedIn feed fetch failed: Unauthorized: LinkedIn session cookies are expired or invalid.

2026-07-03 22:30:05,569 - LinkedInFeedScraper - INFO - LinkedIn Feed scrape complete. Found 0 new posts.

2026-07-03 22:30:06,398 - httpx - INFO - HTTP Request: GET https://api.github.com/users/abeperl/received_events?per_page=100&page=1 "HTTP/1.1 200 OK"

2026-07-03 22:30:07,111 - httpx - INFO - HTTP Request: GET https://api.github.com/search/repositories?q=paperclip+in%3Aname%2Cdescription%2Creadme+stars%3A%3E150&sort=updated&order=desc&per_page=20 "HTTP/1.1 200 OK"

2026-07-03 22:30:07,116 - GitHubScraper - INFO - Found 243 GitHub repositories for term: paperclip

2026-07-03 22:30:07,725 - httpx - INFO - HTTP Request: GET https://api.github.com/users/abeperl/received_events?per_page=100&page=2 "HTTP/1.1 200 OK"

2026-07-03 22:30:08,640 - httpx - INFO - HTTP Request: GET https://api.github.com/search/repositories?q=%22Hermes+Agent%22+in%3Aname%2Cdescription%2Creadme+stars%3A%3E150&sort=updated&order=desc&per_page=20 "HTTP/1.1 200 OK"

2026-07-03 22:30:08,643 - GitHubScraper - INFO - Found 420 GitHub repositories for term: "Hermes Agent"

2026-07-03 22:30:09,550 - httpx - INFO - HTTP Request: GET https://api.github.com/search/repositories?q=multica+in%3Aname%2Cdescription%2Creadme+stars%3A%3E150&sort=updated&order=desc&per_page=20 "HTTP/1.1 200 OK"

2026-07-03 22:30:09,554 - GitHubScraper - INFO - Found 56 GitHub repositories for term: multica

2026-07-03 22:30:09,700 - GitHubScraper - INFO - Total GitHub results found: 0

2026-07-03 22:30:10,253 - httpx - INFO - HTTP Request: GET https://api.github.com/users/abeperl/received_events?per_page=100&page=3 "HTTP/1.1 200 OK"

2026-07-03 22:30:10,942 - GitHubScraper - INFO - GitHub feed: 300 fetched across pages, 0 new events for abeperl

2026-07-03 22:30:10,944 - __main__ - INFO - Scraper github completed: 0 results

2026-07-03 22:30:10,944 - __main__ - INFO - Scraper github_feed completed: 0 results

2026-07-03 22:30:10,944 - __main__ - INFO - Scraper linkedin_feed completed: 0 results

2026-07-03 22:30:10,944 - __main__ - WARNING - LinkedIn Feed: 25 consecutive failures (0 results)

2026-07-03 22:30:10,945 - __main__ - INFO - No new results found across all enabled sources

2026-07-03 22:30:10,945 - __main__ - INFO - Cache stats: 0 items in memory, 0 files on disk

2026-07-03 22:30:10,945 - __main__ - INFO - Monitor complete. No new results found across all sources.

2026-07-03 22:30:10,945 - __main__ - INFO - === Social Media Monitor v2.0 Complete ===

2026-07-03 22:30:10,945 - social_media_monitor.database.connection - INFO - Closing all database connections

2026-07-03 22:30:11,004 - social_media_monitor.database.connection - INFO - Closed 5 database connections

2026-07-03 22:30:11,004 - __main__ - INFO - Closed all database connections

2026-07-03 22:30:11,006 - social_media_monitor.database.connection - INFO - Closing all database connections

2026-07-03 22:30:11,006 - social_media_monitor.database.connection - INFO - Closed 0 database connections

2026-07-03 23:04:39,272 - social_media_monitor.utils.logging_config - INFO - Posts log file: /mnt/data/openclaw-shared/home/shared/Social_Media_Monitor/logs/social_media_monitor_posts_2026-07-03.log

2026-07-03 23:04:39,273 - social_media_monitor.utils.logging_config - INFO - Downloads log file: /mnt/data/openclaw-shared/home/shared/Social_Media_Monitor/logs/social_media_monitor_downloads_2026-07-03.log

2026-07-03 23:04:39,273 - social_media_monitor.utils.logging_config - INFO - Logging configured with level INFO, log file: /mnt/data/openclaw-shared/home/shared/Social_Media_Monitor/logs/social_media_monitor_2026-07-03.log

2026-07-03 23:04:39,273 - __main__ - INFO - === Social Media Monitor v2.0 (Async) Started ===

2026-07-03 23:04:39,303 - social_media_monitor.config.settings - INFO - Configuration loaded from /home/openclaw/Social_Media_Monitor/config.yaml

2026-07-03 23:04:39,303 - __main__ - INFO - Configuration loaded and validated successfully

2026-07-03 23:04:39,305 - social_media_monitor.database.connection - INFO - Database connection pool initialized with 5 connections

2026-07-03 23:04:39,306 - social_media_monitor.database.connection - INFO - add_speaker_filters column already exists

2026-07-03 23:04:39,320 - social_media_monitor.database.connection - INFO - Database setup completed successfully

2026-07-03 23:04:39,320 - __main__ - INFO - Database initialized at: /mnt/data/openclaw-shared/home/shared/Social_Media_Monitor/social_media_monitor.db

2026-07-03 23:04:39,322 - social_media_monitor.scrapers.reddit_scraper - INFO - Reddit API initialized successfully

2026-07-03 23:04:39,323 - RSSFeedScraper - INFO - RSS scraper initialized: feeds=2, html_urls=0 (0 with custom selectors), include_terms=0, exclude_terms=0

2026-07-03 23:04:39,323 - __main__ - INFO - RSS scraper initialized and enabled

2026-07-03 23:04:39,325 - Yiddish24Scraper - INFO - File downloader initialized with 55 terms, prefix 'Y2', and file type organization enabled

2026-07-03 23:04:39,325 - Yiddish24Scraper - INFO - Yiddish24 scraper initialized: data_id='latest', page_limit=100, total_pages=10, debug_html=True, include_terms=55, exclude_terms=0

2026-07-03 23:04:39,325 - __main__ - INFO - Yiddish24 scraper initialized and enabled

2026-07-03 23:04:39,328 - AhBlickLiveScraper - INFO - AhBlickLive scraper initialized: urls=1, max_items=100, include_terms=49, exclude_terms=0

2026-07-03 23:04:39,333 - TorahAnytimeScraper - INFO - File downloader initialized with 53 terms, prefix 'TA', and file type organization enabled

2026-07-03 23:04:39,333 - TorahAnytimeScraper - INFO - TorahAnytime scraper initialized: limit=400, offset=0, project_id=1, include_terms=53, exclude_terms=10

2026-07-03 23:04:39,335 - BatorahScraper - INFO - File downloader initialized with 19 terms, prefix 'BT', and file type organization enabled

2026-07-03 23:04:39,335 - BatorahScraper - INFO - Batorah scraper initialized: api_urls=2, page_size=100, include_terms=19, exclude_terms=1

2026-07-03 23:04:39,335 - KolHalashonScraper - INFO - Kol Halashon scraper initialized: base_url=https://www2.kolhalashon.com, max_shiurim=50, fetch_all_speakers=True, headless=False, exclude_filters=1, include_filters=0

2026-07-03 23:04:39,335 - __main__ - INFO - Ivelt scraper initialized and enabled

2026-07-03 23:04:39,335 - __main__ - INFO - Running 3 scrapers concurrently...

2026-07-03 23:04:39,335 - Yiddish24Scraper - INFO - Starting Yiddish24 scraping (max 10 pages, limit 100 items)

2026-07-03 23:04:39,421 - Yiddish24Scraper - INFO - Fetching Yiddish24 page 1 with data_id='latest', page_limit=100

2026-07-03 23:04:39,425 - IveltScraper - INFO - Loaded 181 include, 0 exclude filters from DB topic_filters

2026-07-03 23:04:39,425 - IveltScraper - INFO - Fetching ivelt active topics from https://www.ivelt.com/forum/search.php?search_id=active_topics...

2026-07-03 23:04:39,426 - RSSFeedScraper - INFO - Fetching RSS feed: https://www.theyeshivaworld.com/feed

2026-07-03 23:04:39,637 - RSSFeedScraper - INFO - Fetching RSS feed: https://www.thegatewaypundit.com/feed/

2026-07-03 23:04:40,035 - RSSFeedScraper - INFO - Total RSS/HTML results: 3

2026-07-03 23:04:43,562 - httpx - INFO - HTTP Request: POST https://www.yiddish24.com/ajax/cat_pagination.php "HTTP/1.1 200 OK"

2026-07-03 23:04:43,783 - Yiddish24Scraper - INFO - Received response with 454297 characters

2026-07-03 23:04:43,787 - Yiddish24Scraper - INFO - Extracted HTML from JSON result field (293757 chars)

2026-07-03 23:04:43,790 - Yiddish24Scraper - INFO - Saved HTML debug file to: /mnt/data/openclaw-shared/home/shared/Social_Media_Monitor/debug/yiddish24_page1_response.html

2026-07-03 23:04:43,872 - Yiddish24Scraper - INFO - Found 100 potential article elements on page 1

2026-07-03 23:04:43,912 - Yiddish24Scraper - INFO - Page 1 extraction: 100 successful, 0 failed out of 100 elements

2026-07-03 23:04:43,912 - Yiddish24Scraper - INFO - Parsed 100 articles from HTML on page 1

2026-07-03 23:04:44,604 - Yiddish24Scraper - INFO - Page 1 summary: 4 new out of 100 total articles

2026-07-03 23:04:44,605 - Yiddish24Scraper - INFO - Page 1: 4/100 new items

2026-07-03 23:04:44,605 - Yiddish24Scraper - INFO - Low new item ratio (4.0%), stopping pagination

2026-07-03 23:04:44,605 - Yiddish24Scraper - INFO - Total Yiddish24 results found: 4

2026-07-03 23:04:51,606 - IveltScraper - INFO - Found 50 active topics on ivelt

2026-07-03 23:05:05,989 - IveltScraper - INFO - Filtering post containing exclude term: '*ביטע מאכט זיכער בלויז צו באריכטן קראנטע נייעס*'

2026-07-03 23:05:10,402 - IveltScraper - INFO - ivelt scrape complete. Checked 5 topics.

2026-07-03 23:05:10,402 - IveltScraper - INFO - All forums scrape complete. Found 8 new posts.

2026-07-03 23:05:10,403 - __main__ - INFO - Scraper rss completed: 3 results

2026-07-03 23:05:10,403 - __main__ - INFO - Scraper yiddish24 completed: 4 results

2026-07-03 23:05:10,403 - __main__ - INFO - Scraper ivelt completed: 8 results

2026-07-03 23:05:11,260 - social_media_monitor.email.sender - INFO - Email sent successfully at 2026-07-03 23:05:11.260531

2026-07-03 23:05:11,260 - __main__ - INFO - Email sent successfully

2026-07-03 23:05:11,261 - __main__ - INFO - Cache stats: 0 items in memory, 0 files on disk

2026-07-03 23:05:11,261 - __main__ - INFO - Monitor complete. Found 15 total results: 3 new rss results, 4 new yiddish24 results, 8 new ivelt results.

2026-07-03 23:05:11,261 - __main__ - INFO - === Social Media Monitor v2.0 Complete ===

2026-07-03 23:05:11,261 - social_media_monitor.database.connection - INFO - Closing all database connections

2026-07-03 23:05:11,324 - social_media_monitor.database.connection - INFO - Closed 5 database connections

2026-07-03 23:05:11,325 - __main__ - INFO - Closed all database connections

2026-07-03 23:05:11,327 - social_media_monitor.database.connection - INFO - Closing all database connections

2026-07-03 23:05:11,327 - social_media_monitor.database.connection - INFO - Closed 0 database connections

2026-07-03 23:30:00,637 - social_media_monitor.utils.logging_config - INFO - Posts log file: /mnt/data/openclaw-shared/home/shared/Social_Media_Monitor/logs/social_media_monitor_posts_2026-07-03.log

2026-07-03 23:30:00,637 - social_media_monitor.utils.logging_config - INFO - Downloads log file: /mnt/data/openclaw-shared/home/shared/Social_Media_Monitor/logs/social_media_monitor_downloads_2026-07-03.log

2026-07-03 23:30:00,637 - social_media_monitor.utils.logging_config - INFO - Logging configured with level INFO, log file: /mnt/data/openclaw-shared/home/shared/Social_Media_Monitor/logs/social_media_monitor_2026-07-03.log

2026-07-03 23:30:00,637 - __main__ - INFO - === Social Media Monitor v2.0 (Async) Started ===

2026-07-03 23:30:00,678 - social_media_monitor.config.settings - INFO - Configuration loaded from /home/openclaw/Social_Media_Monitor/config.yaml

2026-07-03 23:30:00,678 - __main__ - INFO - Configuration loaded and validated successfully

2026-07-03 23:30:00,680 - social_media_monitor.database.connection - INFO - Database connection pool initialized with 5 connections

2026-07-03 23:30:00,681 - social_media_monitor.database.connection - INFO - add_speaker_filters column already exists

2026-07-03 23:30:00,697 - social_media_monitor.database.connection - INFO - Database setup completed successfully

2026-07-03 23:30:00,698 - __main__ - INFO - Database initialized at: /mnt/data/openclaw-shared/home/shared/Social_Media_Monitor/social_media_monitor.db

2026-07-03 23:30:00,700 - social_media_monitor.scrapers.reddit_scraper - INFO - Reddit API initialized successfully

2026-07-03 23:30:00,700 - __main__ - INFO - GitHub scraper initialized and enabled

2026-07-03 23:30:00,700 - __main__ - INFO - GitHub feed scraper initialized and enabled

2026-07-03 23:30:00,701 - RSSFeedScraper - INFO - RSS scraper initialized: feeds=2, html_urls=0 (0 with custom selectors), include_terms=0, exclude_terms=0

2026-07-03 23:30:00,701 - __main__ - INFO - LinkedIn Feed scraper initialized and enabled

2026-07-03 23:30:00,703 - Yiddish24Scraper - INFO - File downloader initialized with 55 terms, prefix 'Y2', and file type organization enabled

2026-07-03 23:30:00,704 - Yiddish24Scraper - INFO - Yiddish24 scraper initialized: data_id='latest', page_limit=100, total_pages=10, debug_html=True, include_terms=55, exclude_terms=0

2026-07-03 23:30:00,706 - AhBlickLiveScraper - INFO - AhBlickLive scraper initialized: urls=1, max_items=100, include_terms=49, exclude_terms=0

2026-07-03 23:30:00,711 - TorahAnytimeScraper - INFO - File downloader initialized with 53 terms, prefix 'TA', and file type organization enabled

2026-07-03 23:30:00,711 - TorahAnytimeScraper - INFO - TorahAnytime scraper initialized: limit=400, offset=0, project_id=1, include_terms=53, exclude_terms=10

2026-07-03 23:30:00,713 - BatorahScraper - INFO - File downloader initialized with 19 terms, prefix 'BT', and file type organization enabled

2026-07-03 23:30:00,713 - BatorahScraper - INFO - Batorah scraper initialized: api_urls=2, page_size=100, include_terms=19, exclude_terms=1

2026-07-03 23:30:00,713 - KolHalashonScraper - INFO - Kol Halashon scraper initialized: base_url=https://www2.kolhalashon.com, max_shiurim=50, fetch_all_speakers=True, headless=False, exclude_filters=1, include_filters=0

2026-07-03 23:30:00,713 - __main__ - INFO - Running 3 scrapers concurrently...

2026-07-03 23:30:00,713 - GitHubScraper - INFO - Searching GitHub for terms: ['paperclip', '"Hermes Agent"', 'multica']

2026-07-03 23:30:00,808 - LinkedInFeedScraper - INFO - Fetching LinkedIn feed (25 posts)

2026-07-03 23:30:03,425 - LinkedInFeedScraper - ERROR - LinkedIn feed fetch failed: Unauthorized: LinkedIn session cookies are expired or invalid.

2026-07-03 23:30:03,433 - LinkedInFeedScraper - INFO - LinkedIn Feed scrape complete. Found 0 new posts.

2026-07-03 23:30:04,141 - httpx - INFO - HTTP Request: GET https://api.github.com/users/abeperl/received_events?per_page=100&page=1 "HTTP/1.1 200 OK"

2026-07-03 23:30:04,833 - httpx - INFO - HTTP Request: GET https://api.github.com/search/repositories?q=paperclip+in%3Aname%2Cdescription%2Creadme+stars%3A%3E150&sort=updated&order=desc&per_page=20 "HTTP/1.1 200 OK"

2026-07-03 23:30:04,839 - GitHubScraper - INFO - Found 243 GitHub repositories for term: paperclip

2026-07-03 23:30:05,516 - httpx - INFO - HTTP Request: GET https://api.github.com/users/abeperl/received_events?per_page=100&page=2 "HTTP/1.1 200 OK"

2026-07-03 23:30:06,197 - httpx - INFO - HTTP Request: GET https://api.github.com/search/repositories?q=%22Hermes+Agent%22+in%3Aname%2Cdescription%2Creadme+stars%3A%3E150&sort=updated&order=desc&per_page=20 "HTTP/1.1 200 OK"

2026-07-03 23:30:06,201 - GitHubScraper - INFO - Found 419 GitHub repositories for term: "Hermes Agent"

2026-07-03 23:30:07,188 - httpx - INFO - HTTP Request: GET https://api.github.com/search/repositories?q=multica+in%3Aname%2Cdescription%2Creadme+stars%3A%3E150&sort=updated&order=desc&per_page=20 "HTTP/1.1 200 OK"

2026-07-03 23:30:07,281 - GitHubScraper - INFO - Found 56 GitHub repositories for term: multica

2026-07-03 23:30:07,410 - GitHubScraper - INFO - Total GitHub results found: 0

2026-07-03 23:30:07,510 - httpx - INFO - HTTP Request: GET https://api.github.com/users/abeperl/received_events?per_page=100&page=3 "HTTP/1.1 200 OK"

2026-07-03 23:30:08,191 - GitHubScraper - INFO - GitHub feed: 300 fetched across pages, 0 new events for abeperl

2026-07-03 23:30:08,192 - __main__ - INFO - Scraper github completed: 0 results

2026-07-03 23:30:08,192 - __main__ - INFO - Scraper github_feed completed: 0 results

2026-07-03 23:30:08,192 - __main__ - INFO - Scraper linkedin_feed completed: 0 results

2026-07-03 23:30:08,193 - __main__ - WARNING - LinkedIn Feed: 26 consecutive failures (0 results)

2026-07-03 23:30:08,193 - __main__ - INFO - No new results found across all enabled sources

2026-07-03 23:30:08,194 - __main__ - INFO - Cache stats: 0 items in memory, 0 files on disk

2026-07-03 23:30:08,194 - __main__ - INFO - Monitor complete. No new results found across all sources.

2026-07-03 23:30:08,194 - __main__ - INFO - === Social Media Monitor v2.0 Complete ===

2026-07-03 23:30:08,194 - social_media_monitor.database.connection - INFO - Closing all database connections

2026-07-03 23:30:08,277 - social_media_monitor.database.connection - INFO - Closed 5 database connections

2026-07-03 23:30:08,277 - __main__ - INFO - Closed all database connections

2026-07-03 23:30:08,279 - social_media_monitor.database.connection - INFO - Closing all database connections

2026-07-03 23:30:08,279 - social_media_monitor.database.connection - INFO - Closed 0 database connections