← Logs

social_media_monitor_2026-07-01.log

2026-07-01 19:30:00,711 - social_media_monitor.scrapers.reddit_scraper - INFO - Reddit API initialized successfully

2026-07-01 19:30:00,711 - __main__ - INFO - GitHub scraper initialized and enabled

2026-07-01 19:30:00,711 - __main__ - INFO - GitHub feed scraper initialized and enabled

2026-07-01 19:30:00,712 - RSSFeedScraper - INFO - RSS scraper initialized: feeds=2, html_urls=0 (0 with custom selectors), include_terms=0, exclude_terms=0

2026-07-01 19:30:00,712 - __main__ - INFO - LinkedIn Feed scraper initialized and enabled

2026-07-01 19:30:00,715 - Yiddish24Scraper - INFO - File downloader initialized with 55 terms, prefix 'Y2', and file type organization enabled

2026-07-01 19:30:00,715 - Yiddish24Scraper - INFO - Yiddish24 scraper initialized: data_id='latest', page_limit=100, total_pages=10, debug_html=True, include_terms=55, exclude_terms=0

2026-07-01 19:30:00,718 - AhBlickLiveScraper - INFO - AhBlickLive scraper initialized: urls=1, max_items=100, include_terms=49, exclude_terms=0

2026-07-01 19:30:00,723 - TorahAnytimeScraper - INFO - File downloader initialized with 53 terms, prefix 'TA', and file type organization enabled

2026-07-01 19:30:00,723 - TorahAnytimeScraper - INFO - TorahAnytime scraper initialized: limit=400, offset=0, project_id=1, include_terms=53, exclude_terms=10

2026-07-01 19:30:00,725 - BatorahScraper - INFO - File downloader initialized with 19 terms, prefix 'BT', and file type organization enabled

2026-07-01 19:30:00,725 - BatorahScraper - INFO - Batorah scraper initialized: api_urls=2, page_size=100, include_terms=19, exclude_terms=1

2026-07-01 19:30:00,725 - KolHalashonScraper - INFO - Kol Halashon scraper initialized: base_url=https://www2.kolhalashon.com, max_shiurim=50, fetch_all_speakers=True, headless=False, exclude_filters=1, include_filters=0

2026-07-01 19:30:00,725 - __main__ - INFO - Running 3 scrapers concurrently...

2026-07-01 19:30:00,726 - GitHubScraper - INFO - Searching GitHub for terms: ['paperclip', '"Hermes Agent"', 'multica']

2026-07-01 19:30:00,829 - LinkedInFeedScraper - INFO - Fetching LinkedIn feed (25 posts)

2026-07-01 19:30:06,477 - LinkedInFeedScraper - ERROR - LinkedIn feed fetch failed: Unauthorized: LinkedIn session cookies are expired or invalid.

2026-07-01 19:30:06,485 - LinkedInFeedScraper - INFO - LinkedIn Feed scrape complete. Found 0 new posts.

2026-07-01 19:30:07,169 - httpx - INFO - HTTP Request: GET https://api.github.com/users/abeperl/received_events?per_page=100&page=1 "HTTP/1.1 200 OK"

2026-07-01 19:30:07,381 - GitHubScraper - INFO - GitHub feed: 29 fetched across pages, 0 new events for abeperl

2026-07-01 19:30:07,382 - httpx - INFO - HTTP Request: GET https://api.github.com/search/repositories?q=paperclip+in%3Aname%2Cdescription%2Creadme+stars%3A%3E150&sort=updated&order=desc&per_page=20 "HTTP/1.1 200 OK"

2026-07-01 19:30:07,427 - GitHubScraper - INFO - Found 242 GitHub repositories for term: paperclip

2026-07-01 19:30:08,555 - httpx - INFO - HTTP Request: GET https://api.github.com/search/repositories?q=%22Hermes+Agent%22+in%3Aname%2Cdescription%2Creadme+stars%3A%3E150&sort=updated&order=desc&per_page=20 "HTTP/1.1 200 OK"

2026-07-01 19:30:08,559 - GitHubScraper - INFO - Found 413 GitHub repositories for term: "Hermes Agent"

2026-07-01 19:30:09,607 - httpx - INFO - HTTP Request: GET https://api.github.com/search/repositories?q=multica+in%3Aname%2Cdescription%2Creadme+stars%3A%3E150&sort=updated&order=desc&per_page=20 "HTTP/1.1 200 OK"

2026-07-01 19:30:09,611 - GitHubScraper - INFO - Found 54 GitHub repositories for term: multica

2026-07-01 19:30:09,741 - GitHubScraper - INFO - Total GitHub results found: 0

2026-07-01 19:30:09,741 - __main__ - INFO - Scraper github completed: 0 results

2026-07-01 19:30:09,741 - __main__ - INFO - Scraper github_feed completed: 0 results

2026-07-01 19:30:09,741 - __main__ - INFO - Scraper linkedin_feed completed: 0 results

2026-07-01 19:30:09,742 - __main__ - WARNING - LinkedIn Feed: 8 consecutive failures (0 results)

2026-07-01 19:30:09,742 - __main__ - INFO - No new results found across all enabled sources

2026-07-01 19:30:09,743 - __main__ - INFO - Cache stats: 0 items in memory, 0 files on disk

2026-07-01 19:30:09,743 - __main__ - INFO - Monitor complete. No new results found across all sources.

2026-07-01 19:30:09,743 - __main__ - INFO - === Social Media Monitor v2.0 Complete ===

2026-07-01 19:30:09,743 - social_media_monitor.database.connection - INFO - Closing all database connections

2026-07-01 19:30:09,758 - social_media_monitor.database.connection - INFO - Closed 5 database connections

2026-07-01 19:30:09,758 - __main__ - INFO - Closed all database connections

2026-07-01 19:30:09,760 - social_media_monitor.database.connection - INFO - Closing all database connections

2026-07-01 19:30:09,760 - social_media_monitor.database.connection - INFO - Closed 0 database connections

2026-07-01 20:04:39,255 - social_media_monitor.utils.logging_config - INFO - Posts log file: /mnt/data/openclaw-shared/home/shared/Social_Media_Monitor/logs/social_media_monitor_posts_2026-07-01.log

2026-07-01 20:04:39,255 - social_media_monitor.utils.logging_config - INFO - Downloads log file: /mnt/data/openclaw-shared/home/shared/Social_Media_Monitor/logs/social_media_monitor_downloads_2026-07-01.log

2026-07-01 20:04:39,255 - social_media_monitor.utils.logging_config - INFO - Logging configured with level INFO, log file: /mnt/data/openclaw-shared/home/shared/Social_Media_Monitor/logs/social_media_monitor_2026-07-01.log

2026-07-01 20:04:39,255 - __main__ - INFO - === Social Media Monitor v2.0 (Async) Started ===

2026-07-01 20:04:39,286 - social_media_monitor.config.settings - INFO - Configuration loaded from /home/openclaw/Social_Media_Monitor/config.yaml

2026-07-01 20:04:39,286 - __main__ - INFO - Configuration loaded and validated successfully

2026-07-01 20:04:39,289 - social_media_monitor.database.connection - INFO - Database connection pool initialized with 5 connections

2026-07-01 20:04:39,289 - social_media_monitor.database.connection - INFO - add_speaker_filters column already exists

2026-07-01 20:04:39,304 - social_media_monitor.database.connection - INFO - Database setup completed successfully

2026-07-01 20:04:39,304 - __main__ - INFO - Database initialized at: /mnt/data/openclaw-shared/home/shared/Social_Media_Monitor/social_media_monitor.db

2026-07-01 20:04:39,306 - social_media_monitor.scrapers.reddit_scraper - INFO - Reddit API initialized successfully

2026-07-01 20:04:39,307 - RSSFeedScraper - INFO - RSS scraper initialized: feeds=2, html_urls=0 (0 with custom selectors), include_terms=0, exclude_terms=0

2026-07-01 20:04:39,307 - __main__ - INFO - RSS scraper initialized and enabled

2026-07-01 20:04:39,310 - Yiddish24Scraper - INFO - File downloader initialized with 55 terms, prefix 'Y2', and file type organization enabled

2026-07-01 20:04:39,310 - Yiddish24Scraper - INFO - Yiddish24 scraper initialized: data_id='latest', page_limit=100, total_pages=10, debug_html=True, include_terms=55, exclude_terms=0

2026-07-01 20:04:39,310 - __main__ - INFO - Yiddish24 scraper initialized and enabled

2026-07-01 20:04:39,312 - AhBlickLiveScraper - INFO - AhBlickLive scraper initialized: urls=1, max_items=100, include_terms=49, exclude_terms=0

2026-07-01 20:04:39,317 - TorahAnytimeScraper - INFO - File downloader initialized with 53 terms, prefix 'TA', and file type organization enabled

2026-07-01 20:04:39,317 - TorahAnytimeScraper - INFO - TorahAnytime scraper initialized: limit=400, offset=0, project_id=1, include_terms=53, exclude_terms=10

2026-07-01 20:04:39,319 - BatorahScraper - INFO - File downloader initialized with 19 terms, prefix 'BT', and file type organization enabled

2026-07-01 20:04:39,319 - BatorahScraper - INFO - Batorah scraper initialized: api_urls=2, page_size=100, include_terms=19, exclude_terms=1

2026-07-01 20:04:39,319 - KolHalashonScraper - INFO - Kol Halashon scraper initialized: base_url=https://www2.kolhalashon.com, max_shiurim=50, fetch_all_speakers=True, headless=False, exclude_filters=1, include_filters=0

2026-07-01 20:04:39,319 - __main__ - INFO - Ivelt scraper initialized and enabled

2026-07-01 20:04:39,319 - __main__ - INFO - Running 3 scrapers concurrently...

2026-07-01 20:04:39,319 - Yiddish24Scraper - INFO - Starting Yiddish24 scraping (max 10 pages, limit 100 items)

2026-07-01 20:04:39,408 - Yiddish24Scraper - INFO - Fetching Yiddish24 page 1 with data_id='latest', page_limit=100

2026-07-01 20:04:39,413 - IveltScraper - INFO - Loaded 181 include, 0 exclude filters from DB topic_filters

2026-07-01 20:04:39,413 - IveltScraper - INFO - Fetching ivelt active topics from https://www.ivelt.com/forum/search.php?search_id=active_topics...

2026-07-01 20:04:39,414 - RSSFeedScraper - INFO - Fetching RSS feed: https://www.theyeshivaworld.com/feed

2026-07-01 20:04:39,656 - RSSFeedScraper - INFO - Fetching RSS feed: https://www.thegatewaypundit.com/feed/

2026-07-01 20:04:40,027 - RSSFeedScraper - INFO - Total RSS/HTML results: 6

2026-07-01 20:04:43,539 - httpx - INFO - HTTP Request: POST https://www.yiddish24.com/ajax/cat_pagination.php "HTTP/1.1 200 OK"

2026-07-01 20:04:43,773 - Yiddish24Scraper - INFO - Received response with 514097 characters

2026-07-01 20:04:43,776 - Yiddish24Scraper - INFO - Extracted HTML from JSON result field (316722 chars)

2026-07-01 20:04:43,780 - Yiddish24Scraper - INFO - Saved HTML debug file to: /mnt/data/openclaw-shared/home/shared/Social_Media_Monitor/debug/yiddish24_page1_response.html

2026-07-01 20:04:43,870 - Yiddish24Scraper - INFO - Found 100 potential article elements on page 1

2026-07-01 20:04:43,911 - Yiddish24Scraper - INFO - Page 1 extraction: 100 successful, 0 failed out of 100 elements

2026-07-01 20:04:43,911 - Yiddish24Scraper - INFO - Parsed 100 articles from HTML on page 1

2026-07-01 20:04:44,640 - Yiddish24Scraper - INFO - Page 1 summary: 12 new out of 100 total articles

2026-07-01 20:04:44,640 - Yiddish24Scraper - INFO - Page 1: 12/100 new items

2026-07-01 20:04:44,640 - Yiddish24Scraper - INFO - Low new item ratio (12.0%), stopping pagination

2026-07-01 20:04:44,641 - Yiddish24Scraper - INFO - Total Yiddish24 results found: 12

2026-07-01 20:04:51,564 - IveltScraper - INFO - Found 50 active topics on ivelt

2026-07-01 20:05:23,776 - IveltScraper - INFO - ivelt scrape complete. Checked 5 topics.

2026-07-01 20:05:23,776 - IveltScraper - INFO - All forums scrape complete. Found 32 new posts.

2026-07-01 20:05:23,777 - __main__ - INFO - Scraper rss completed: 6 results

2026-07-01 20:05:23,777 - __main__ - INFO - Scraper yiddish24 completed: 12 results

2026-07-01 20:05:23,777 - __main__ - INFO - Scraper ivelt completed: 32 results

2026-07-01 20:05:24,897 - social_media_monitor.email.sender - INFO - Email sent successfully at 2026-07-01 20:05:24.897148

2026-07-01 20:05:24,897 - __main__ - INFO - Email sent successfully

2026-07-01 20:05:24,897 - __main__ - INFO - Cache stats: 0 items in memory, 0 files on disk

2026-07-01 20:05:24,898 - __main__ - INFO - Monitor complete. Found 50 total results: 6 new rss results, 12 new yiddish24 results, 32 new ivelt results.

2026-07-01 20:05:24,898 - __main__ - INFO - === Social Media Monitor v2.0 Complete ===

2026-07-01 20:05:24,898 - social_media_monitor.database.connection - INFO - Closing all database connections

2026-07-01 20:05:24,970 - social_media_monitor.database.connection - INFO - Closed 5 database connections

2026-07-01 20:05:24,970 - __main__ - INFO - Closed all database connections

2026-07-01 20:05:24,972 - social_media_monitor.database.connection - INFO - Closing all database connections

2026-07-01 20:05:24,972 - social_media_monitor.database.connection - INFO - Closed 0 database connections

2026-07-01 20:30:00,633 - social_media_monitor.utils.logging_config - INFO - Posts log file: /mnt/data/openclaw-shared/home/shared/Social_Media_Monitor/logs/social_media_monitor_posts_2026-07-01.log

2026-07-01 20:30:00,633 - social_media_monitor.utils.logging_config - INFO - Downloads log file: /mnt/data/openclaw-shared/home/shared/Social_Media_Monitor/logs/social_media_monitor_downloads_2026-07-01.log

2026-07-01 20:30:00,633 - social_media_monitor.utils.logging_config - INFO - Logging configured with level INFO, log file: /mnt/data/openclaw-shared/home/shared/Social_Media_Monitor/logs/social_media_monitor_2026-07-01.log

2026-07-01 20:30:00,634 - __main__ - INFO - === Social Media Monitor v2.0 (Async) Started ===

2026-07-01 20:30:00,665 - social_media_monitor.config.settings - INFO - Configuration loaded from /home/openclaw/Social_Media_Monitor/config.yaml

2026-07-01 20:30:00,665 - __main__ - INFO - Configuration loaded and validated successfully

2026-07-01 20:30:00,667 - social_media_monitor.database.connection - INFO - Database connection pool initialized with 5 connections

2026-07-01 20:30:00,667 - social_media_monitor.database.connection - INFO - add_speaker_filters column already exists

2026-07-01 20:30:00,681 - social_media_monitor.database.connection - INFO - Database setup completed successfully

2026-07-01 20:30:00,681 - __main__ - INFO - Database initialized at: /mnt/data/openclaw-shared/home/shared/Social_Media_Monitor/social_media_monitor.db

2026-07-01 20:30:00,730 - social_media_monitor.scrapers.reddit_scraper - INFO - Reddit API initialized successfully

2026-07-01 20:30:00,731 - __main__ - INFO - GitHub scraper initialized and enabled

2026-07-01 20:30:00,731 - __main__ - INFO - GitHub feed scraper initialized and enabled

2026-07-01 20:30:00,731 - RSSFeedScraper - INFO - RSS scraper initialized: feeds=2, html_urls=0 (0 with custom selectors), include_terms=0, exclude_terms=0

2026-07-01 20:30:00,731 - __main__ - INFO - LinkedIn Feed scraper initialized and enabled

2026-07-01 20:30:00,735 - Yiddish24Scraper - INFO - File downloader initialized with 55 terms, prefix 'Y2', and file type organization enabled

2026-07-01 20:30:00,735 - Yiddish24Scraper - INFO - Yiddish24 scraper initialized: data_id='latest', page_limit=100, total_pages=10, debug_html=True, include_terms=55, exclude_terms=0

2026-07-01 20:30:00,737 - AhBlickLiveScraper - INFO - AhBlickLive scraper initialized: urls=1, max_items=100, include_terms=49, exclude_terms=0

2026-07-01 20:30:00,742 - TorahAnytimeScraper - INFO - File downloader initialized with 53 terms, prefix 'TA', and file type organization enabled

2026-07-01 20:30:00,742 - TorahAnytimeScraper - INFO - TorahAnytime scraper initialized: limit=400, offset=0, project_id=1, include_terms=53, exclude_terms=10

2026-07-01 20:30:00,744 - BatorahScraper - INFO - File downloader initialized with 19 terms, prefix 'BT', and file type organization enabled

2026-07-01 20:30:00,744 - BatorahScraper - INFO - Batorah scraper initialized: api_urls=2, page_size=100, include_terms=19, exclude_terms=1

2026-07-01 20:30:00,744 - KolHalashonScraper - INFO - Kol Halashon scraper initialized: base_url=https://www2.kolhalashon.com, max_shiurim=50, fetch_all_speakers=True, headless=False, exclude_filters=1, include_filters=0

2026-07-01 20:30:00,744 - __main__ - INFO - Running 3 scrapers concurrently...

2026-07-01 20:30:00,744 - GitHubScraper - INFO - Searching GitHub for terms: ['paperclip', '"Hermes Agent"', 'multica']

2026-07-01 20:30:00,838 - LinkedInFeedScraper - INFO - Fetching LinkedIn feed (25 posts)

2026-07-01 20:30:05,417 - LinkedInFeedScraper - ERROR - LinkedIn feed fetch failed: Unauthorized: LinkedIn session cookies are expired or invalid.

2026-07-01 20:30:05,426 - LinkedInFeedScraper - INFO - LinkedIn Feed scrape complete. Found 0 new posts.

2026-07-01 20:30:06,140 - httpx - INFO - HTTP Request: GET https://api.github.com/users/abeperl/received_events?per_page=100&page=1 "HTTP/1.1 200 OK"

2026-07-01 20:30:06,505 - GitHubScraper - INFO - GitHub feed: 39 fetched across pages, 11 new events for abeperl

2026-07-01 20:30:06,507 - httpx - INFO - HTTP Request: GET https://api.github.com/search/repositories?q=paperclip+in%3Aname%2Cdescription%2Creadme+stars%3A%3E150&sort=updated&order=desc&per_page=20 "HTTP/1.1 200 OK"

2026-07-01 20:30:06,512 - GitHubScraper - INFO - Found 242 GitHub repositories for term: paperclip

2026-07-01 20:30:07,616 - httpx - INFO - HTTP Request: GET https://api.github.com/search/repositories?q=%22Hermes+Agent%22+in%3Aname%2Cdescription%2Creadme+stars%3A%3E150&sort=updated&order=desc&per_page=20 "HTTP/1.1 200 OK"

2026-07-01 20:30:07,620 - GitHubScraper - INFO - Found 413 GitHub repositories for term: "Hermes Agent"

2026-07-01 20:30:08,561 - httpx - INFO - HTTP Request: GET https://api.github.com/search/repositories?q=multica+in%3Aname%2Cdescription%2Creadme+stars%3A%3E150&sort=updated&order=desc&per_page=20 "HTTP/1.1 200 OK"

2026-07-01 20:30:08,651 - GitHubScraper - INFO - Found 54 GitHub repositories for term: multica

2026-07-01 20:30:08,788 - GitHubScraper - INFO - Total GitHub results found: 0

2026-07-01 20:30:08,788 - __main__ - INFO - Scraper github completed: 0 results

2026-07-01 20:30:08,788 - __main__ - INFO - Scraper github_feed completed: 11 results

2026-07-01 20:30:08,788 - __main__ - INFO - Scraper linkedin_feed completed: 0 results

2026-07-01 20:30:08,789 - __main__ - WARNING - LinkedIn Feed: 9 consecutive failures (0 results)

2026-07-01 20:30:09,926 - social_media_monitor.email.sender - INFO - Email sent successfully at 2026-07-01 20:30:09.926410

2026-07-01 20:30:09,926 - __main__ - INFO - Email sent successfully

2026-07-01 20:30:09,927 - __main__ - INFO - Cache stats: 0 items in memory, 0 files on disk

2026-07-01 20:30:09,927 - __main__ - INFO - Monitor complete. Found 11 total results: 11 new github feed results.

2026-07-01 20:30:09,927 - __main__ - INFO - === Social Media Monitor v2.0 Complete ===

2026-07-01 20:30:09,927 - social_media_monitor.database.connection - INFO - Closing all database connections

2026-07-01 20:30:09,959 - social_media_monitor.database.connection - INFO - Closed 5 database connections

2026-07-01 20:30:09,959 - __main__ - INFO - Closed all database connections

2026-07-01 20:30:09,961 - social_media_monitor.database.connection - INFO - Closing all database connections

2026-07-01 20:30:09,961 - social_media_monitor.database.connection - INFO - Closed 0 database connections

2026-07-01 21:04:39,281 - social_media_monitor.utils.logging_config - INFO - Posts log file: /mnt/data/openclaw-shared/home/shared/Social_Media_Monitor/logs/social_media_monitor_posts_2026-07-01.log

2026-07-01 21:04:39,281 - social_media_monitor.utils.logging_config - INFO - Downloads log file: /mnt/data/openclaw-shared/home/shared/Social_Media_Monitor/logs/social_media_monitor_downloads_2026-07-01.log

2026-07-01 21:04:39,281 - social_media_monitor.utils.logging_config - INFO - Logging configured with level INFO, log file: /mnt/data/openclaw-shared/home/shared/Social_Media_Monitor/logs/social_media_monitor_2026-07-01.log

2026-07-01 21:04:39,281 - __main__ - INFO - === Social Media Monitor v2.0 (Async) Started ===

2026-07-01 21:04:39,315 - social_media_monitor.config.settings - INFO - Configuration loaded from /home/openclaw/Social_Media_Monitor/config.yaml

2026-07-01 21:04:39,315 - __main__ - INFO - Configuration loaded and validated successfully

2026-07-01 21:04:39,317 - social_media_monitor.database.connection - INFO - Database connection pool initialized with 5 connections

2026-07-01 21:04:39,318 - social_media_monitor.database.connection - INFO - add_speaker_filters column already exists

2026-07-01 21:04:39,332 - social_media_monitor.database.connection - INFO - Database setup completed successfully

2026-07-01 21:04:39,332 - __main__ - INFO - Database initialized at: /mnt/data/openclaw-shared/home/shared/Social_Media_Monitor/social_media_monitor.db

2026-07-01 21:04:39,335 - social_media_monitor.scrapers.reddit_scraper - INFO - Reddit API initialized successfully

2026-07-01 21:04:39,336 - RSSFeedScraper - INFO - RSS scraper initialized: feeds=2, html_urls=0 (0 with custom selectors), include_terms=0, exclude_terms=0

2026-07-01 21:04:39,336 - __main__ - INFO - RSS scraper initialized and enabled

2026-07-01 21:04:39,338 - Yiddish24Scraper - INFO - File downloader initialized with 55 terms, prefix 'Y2', and file type organization enabled

2026-07-01 21:04:39,338 - Yiddish24Scraper - INFO - Yiddish24 scraper initialized: data_id='latest', page_limit=100, total_pages=10, debug_html=True, include_terms=55, exclude_terms=0

2026-07-01 21:04:39,338 - __main__ - INFO - Yiddish24 scraper initialized and enabled

2026-07-01 21:04:39,342 - AhBlickLiveScraper - INFO - AhBlickLive scraper initialized: urls=1, max_items=100, include_terms=49, exclude_terms=0

2026-07-01 21:04:39,346 - TorahAnytimeScraper - INFO - File downloader initialized with 53 terms, prefix 'TA', and file type organization enabled

2026-07-01 21:04:39,346 - TorahAnytimeScraper - INFO - TorahAnytime scraper initialized: limit=400, offset=0, project_id=1, include_terms=53, exclude_terms=10

2026-07-01 21:04:39,348 - BatorahScraper - INFO - File downloader initialized with 19 terms, prefix 'BT', and file type organization enabled

2026-07-01 21:04:39,348 - BatorahScraper - INFO - Batorah scraper initialized: api_urls=2, page_size=100, include_terms=19, exclude_terms=1

2026-07-01 21:04:39,348 - KolHalashonScraper - INFO - Kol Halashon scraper initialized: base_url=https://www2.kolhalashon.com, max_shiurim=50, fetch_all_speakers=True, headless=False, exclude_filters=1, include_filters=0

2026-07-01 21:04:39,348 - __main__ - INFO - Ivelt scraper initialized and enabled

2026-07-01 21:04:39,348 - __main__ - INFO - Running 3 scrapers concurrently...

2026-07-01 21:04:39,349 - Yiddish24Scraper - INFO - Starting Yiddish24 scraping (max 10 pages, limit 100 items)

2026-07-01 21:04:39,438 - Yiddish24Scraper - INFO - Fetching Yiddish24 page 1 with data_id='latest', page_limit=100

2026-07-01 21:04:39,442 - IveltScraper - INFO - Loaded 181 include, 0 exclude filters from DB topic_filters

2026-07-01 21:04:39,442 - IveltScraper - INFO - Fetching ivelt active topics from https://www.ivelt.com/forum/search.php?search_id=active_topics...

2026-07-01 21:04:39,443 - RSSFeedScraper - INFO - Fetching RSS feed: https://www.theyeshivaworld.com/feed

2026-07-01 21:04:39,691 - RSSFeedScraper - INFO - Fetching RSS feed: https://www.thegatewaypundit.com/feed/

2026-07-01 21:04:41,548 - RSSFeedScraper - INFO - Total RSS/HTML results: 4

2026-07-01 21:04:51,730 - IveltScraper - INFO - Found 50 active topics on ivelt

2026-07-01 21:05:07,299 - httpx - INFO - HTTP Request: POST https://www.yiddish24.com/ajax/cat_pagination.php "HTTP/1.1 200 OK"

2026-07-01 21:05:07,517 - Yiddish24Scraper - INFO - Received response with 526233 characters

2026-07-01 21:05:07,521 - Yiddish24Scraper - INFO - Extracted HTML from JSON result field (322240 chars)

2026-07-01 21:05:07,524 - Yiddish24Scraper - INFO - Saved HTML debug file to: /mnt/data/openclaw-shared/home/shared/Social_Media_Monitor/debug/yiddish24_page1_response.html

2026-07-01 21:05:07,604 - Yiddish24Scraper - INFO - Found 100 potential article elements on page 1

2026-07-01 21:05:07,645 - Yiddish24Scraper - INFO - Page 1 extraction: 100 successful, 0 failed out of 100 elements

2026-07-01 21:05:07,645 - Yiddish24Scraper - INFO - Parsed 100 articles from HTML on page 1

2026-07-01 21:05:07,688 - Yiddish24Scraper - INFO - Term 'אינטערעסאנטס -' matched in DESCRIPTION: 'אינטערעסאנטס - מאדנע חידושים און טשיקאווע איבערלעבענישן'

2026-07-01 21:05:07,688 - Yiddish24Scraper - INFO - Downloading 1 media file(s) for article: הכל בכל

2026-07-01 21:05:08,466 - social_media_monitor.utils.file_downloader - INFO - Starting download from: https://cloudfront.yiddish24.com/BM__260_260701033133.mp3

2026-07-01 21:05:09,823 - social_media_monitor.utils.file_downloader - INFO - Successfully downloaded: Y2_BM__260_260701033133.mp3 (15.5 MB)

2026-07-01 21:05:09,824 - social_media_monitor.utils.file_downloader - INFO - Organized audio file to: /mnt/data/openclaw-shared/home/shared/Social_Media_Monitor/downloads/Audio (audio/mpeg)

2026-07-01 21:05:09,844 - social_media_monitor.utils.media_tagger - INFO - Tagged Y2_BM__260_260701033133.mp3: title, artist, album, comment, source_url, genre, publisher

2026-07-01 21:05:09,844 - Yiddish24Scraper - INFO - Renamed downloaded file to include match term: Y2_BM__260_260701033133_אינטערעסאנטס_-.mp3

2026-07-01 21:05:10,499 - Yiddish24Scraper - INFO - Page 1 summary: 6 new out of 100 total articles

2026-07-01 21:05:10,499 - Yiddish24Scraper - INFO - Page 1: 6/100 new items

2026-07-01 21:05:10,499 - Yiddish24Scraper - INFO - Low new item ratio (6.0%), stopping pagination

2026-07-01 21:05:10,500 - Yiddish24Scraper - INFO - Total Yiddish24 results found: 6

2026-07-01 21:05:14,554 - IveltScraper - INFO - ivelt scrape complete. Checked 4 topics.

2026-07-01 21:05:14,554 - IveltScraper - INFO - All forums scrape complete. Found 8 new posts.

2026-07-01 21:05:14,554 - __main__ - INFO - Scraper rss completed: 4 results

2026-07-01 21:05:14,554 - __main__ - INFO - Scraper yiddish24 completed: 6 results

2026-07-01 21:05:14,554 - __main__ - INFO - Scraper ivelt completed: 8 results

2026-07-01 21:05:15,442 - social_media_monitor.email.sender - INFO - Email sent successfully at 2026-07-01 21:05:15.442027

2026-07-01 21:05:15,442 - __main__ - INFO - Email sent successfully

2026-07-01 21:05:15,442 - __main__ - INFO - Cache stats: 0 items in memory, 0 files on disk

2026-07-01 21:05:15,442 - __main__ - INFO - Monitor complete. Found 18 total results: 4 new rss results, 6 new yiddish24 results, 8 new ivelt results.

2026-07-01 21:05:15,442 - __main__ - INFO - === Social Media Monitor v2.0 Complete ===

2026-07-01 21:05:15,442 - social_media_monitor.database.connection - INFO - Closing all database connections

2026-07-01 21:05:15,551 - social_media_monitor.database.connection - INFO - Closed 5 database connections

2026-07-01 21:05:15,551 - __main__ - INFO - Closed all database connections

2026-07-01 21:05:15,553 - social_media_monitor.database.connection - INFO - Closing all database connections

2026-07-01 21:05:15,553 - social_media_monitor.database.connection - INFO - Closed 0 database connections

2026-07-01 21:30:00,656 - social_media_monitor.utils.logging_config - INFO - Posts log file: /mnt/data/openclaw-shared/home/shared/Social_Media_Monitor/logs/social_media_monitor_posts_2026-07-01.log

2026-07-01 21:30:00,656 - social_media_monitor.utils.logging_config - INFO - Downloads log file: /mnt/data/openclaw-shared/home/shared/Social_Media_Monitor/logs/social_media_monitor_downloads_2026-07-01.log

2026-07-01 21:30:00,656 - social_media_monitor.utils.logging_config - INFO - Logging configured with level INFO, log file: /mnt/data/openclaw-shared/home/shared/Social_Media_Monitor/logs/social_media_monitor_2026-07-01.log

2026-07-01 21:30:00,656 - __main__ - INFO - === Social Media Monitor v2.0 (Async) Started ===

2026-07-01 21:30:00,687 - social_media_monitor.config.settings - INFO - Configuration loaded from /home/openclaw/Social_Media_Monitor/config.yaml

2026-07-01 21:30:00,687 - __main__ - INFO - Configuration loaded and validated successfully

2026-07-01 21:30:00,689 - social_media_monitor.database.connection - INFO - Database connection pool initialized with 5 connections

2026-07-01 21:30:00,689 - social_media_monitor.database.connection - INFO - add_speaker_filters column already exists

2026-07-01 21:30:00,703 - social_media_monitor.database.connection - INFO - Database setup completed successfully

2026-07-01 21:30:00,703 - __main__ - INFO - Database initialized at: /mnt/data/openclaw-shared/home/shared/Social_Media_Monitor/social_media_monitor.db

2026-07-01 21:30:00,706 - social_media_monitor.scrapers.reddit_scraper - INFO - Reddit API initialized successfully

2026-07-01 21:30:00,706 - __main__ - INFO - GitHub scraper initialized and enabled

2026-07-01 21:30:00,706 - __main__ - INFO - GitHub feed scraper initialized and enabled

2026-07-01 21:30:00,706 - RSSFeedScraper - INFO - RSS scraper initialized: feeds=2, html_urls=0 (0 with custom selectors), include_terms=0, exclude_terms=0

2026-07-01 21:30:00,707 - __main__ - INFO - LinkedIn Feed scraper initialized and enabled

2026-07-01 21:30:00,709 - Yiddish24Scraper - INFO - File downloader initialized with 55 terms, prefix 'Y2', and file type organization enabled

2026-07-01 21:30:00,709 - Yiddish24Scraper - INFO - Yiddish24 scraper initialized: data_id='latest', page_limit=100, total_pages=10, debug_html=True, include_terms=55, exclude_terms=0

2026-07-01 21:30:00,712 - AhBlickLiveScraper - INFO - AhBlickLive scraper initialized: urls=1, max_items=100, include_terms=49, exclude_terms=0

2026-07-01 21:30:00,716 - TorahAnytimeScraper - INFO - File downloader initialized with 53 terms, prefix 'TA', and file type organization enabled

2026-07-01 21:30:00,716 - TorahAnytimeScraper - INFO - TorahAnytime scraper initialized: limit=400, offset=0, project_id=1, include_terms=53, exclude_terms=10

2026-07-01 21:30:00,718 - BatorahScraper - INFO - File downloader initialized with 19 terms, prefix 'BT', and file type organization enabled

2026-07-01 21:30:00,718 - BatorahScraper - INFO - Batorah scraper initialized: api_urls=2, page_size=100, include_terms=19, exclude_terms=1

2026-07-01 21:30:00,719 - KolHalashonScraper - INFO - Kol Halashon scraper initialized: base_url=https://www2.kolhalashon.com, max_shiurim=50, fetch_all_speakers=True, headless=False, exclude_filters=1, include_filters=0

2026-07-01 21:30:00,719 - __main__ - INFO - Running 3 scrapers concurrently...

2026-07-01 21:30:00,719 - GitHubScraper - INFO - Searching GitHub for terms: ['paperclip', '"Hermes Agent"', 'multica']

2026-07-01 21:30:00,818 - LinkedInFeedScraper - INFO - Fetching LinkedIn feed (25 posts)

2026-07-01 21:30:04,413 - LinkedInFeedScraper - ERROR - LinkedIn feed fetch failed: Unauthorized: LinkedIn session cookies are expired or invalid.

2026-07-01 21:30:04,457 - LinkedInFeedScraper - INFO - LinkedIn Feed scrape complete. Found 0 new posts.

2026-07-01 21:30:05,338 - httpx - INFO - HTTP Request: GET https://api.github.com/users/abeperl/received_events?per_page=100&page=1 "HTTP/1.1 200 OK"

2026-07-01 21:30:05,619 - GitHubScraper - INFO - GitHub feed: 39 fetched across pages, 0 new events for abeperl

2026-07-01 21:30:05,620 - httpx - INFO - HTTP Request: GET https://api.github.com/search/repositories?q=paperclip+in%3Aname%2Cdescription%2Creadme+stars%3A%3E150&sort=updated&order=desc&per_page=20 "HTTP/1.1 200 OK"

2026-07-01 21:30:05,625 - GitHubScraper - INFO - Found 242 GitHub repositories for term: paperclip

2026-07-01 21:30:06,864 - httpx - INFO - HTTP Request: GET https://api.github.com/search/repositories?q=%22Hermes+Agent%22+in%3Aname%2Cdescription%2Creadme+stars%3A%3E150&sort=updated&order=desc&per_page=20 "HTTP/1.1 200 OK"

2026-07-01 21:30:06,954 - GitHubScraper - INFO - Found 413 GitHub repositories for term: "Hermes Agent"

2026-07-01 21:30:08,179 - httpx - INFO - HTTP Request: GET https://api.github.com/search/repositories?q=multica+in%3Aname%2Cdescription%2Creadme+stars%3A%3E150&sort=updated&order=desc&per_page=20 "HTTP/1.1 200 OK"

2026-07-01 21:30:08,265 - GitHubScraper - INFO - Found 54 GitHub repositories for term: multica

2026-07-01 21:30:08,402 - GitHubScraper - INFO - Total GitHub results found: 0

2026-07-01 21:30:08,403 - __main__ - INFO - Scraper github completed: 0 results

2026-07-01 21:30:08,403 - __main__ - INFO - Scraper github_feed completed: 0 results

2026-07-01 21:30:08,403 - __main__ - INFO - Scraper linkedin_feed completed: 0 results

2026-07-01 21:30:08,404 - __main__ - WARNING - LinkedIn Feed: 10 consecutive failures (0 results)

2026-07-01 21:30:08,404 - __main__ - INFO - No new results found across all enabled sources

2026-07-01 21:30:08,404 - __main__ - INFO - Cache stats: 0 items in memory, 0 files on disk

2026-07-01 21:30:08,404 - __main__ - INFO - Monitor complete. No new results found across all sources.

2026-07-01 21:30:08,404 - __main__ - INFO - === Social Media Monitor v2.0 Complete ===

2026-07-01 21:30:08,404 - social_media_monitor.database.connection - INFO - Closing all database connections

2026-07-01 21:30:08,433 - social_media_monitor.database.connection - INFO - Closed 5 database connections

2026-07-01 21:30:08,433 - __main__ - INFO - Closed all database connections

2026-07-01 21:30:08,435 - social_media_monitor.database.connection - INFO - Closing all database connections

2026-07-01 21:30:08,435 - social_media_monitor.database.connection - INFO - Closed 0 database connections

2026-07-01 22:04:39,284 - social_media_monitor.utils.logging_config - INFO - Posts log file: /mnt/data/openclaw-shared/home/shared/Social_Media_Monitor/logs/social_media_monitor_posts_2026-07-01.log

2026-07-01 22:04:39,284 - social_media_monitor.utils.logging_config - INFO - Downloads log file: /mnt/data/openclaw-shared/home/shared/Social_Media_Monitor/logs/social_media_monitor_downloads_2026-07-01.log

2026-07-01 22:04:39,284 - social_media_monitor.utils.logging_config - INFO - Logging configured with level INFO, log file: /mnt/data/openclaw-shared/home/shared/Social_Media_Monitor/logs/social_media_monitor_2026-07-01.log

2026-07-01 22:04:39,284 - __main__ - INFO - === Social Media Monitor v2.0 (Async) Started ===

2026-07-01 22:04:39,317 - social_media_monitor.config.settings - INFO - Configuration loaded from /home/openclaw/Social_Media_Monitor/config.yaml

2026-07-01 22:04:39,318 - __main__ - INFO - Configuration loaded and validated successfully

2026-07-01 22:04:39,320 - social_media_monitor.database.connection - INFO - Database connection pool initialized with 5 connections

2026-07-01 22:04:39,321 - social_media_monitor.database.connection - INFO - add_speaker_filters column already exists

2026-07-01 22:04:39,338 - social_media_monitor.database.connection - INFO - Database setup completed successfully

2026-07-01 22:04:39,338 - __main__ - INFO - Database initialized at: /mnt/data/openclaw-shared/home/shared/Social_Media_Monitor/social_media_monitor.db

2026-07-01 22:04:39,387 - social_media_monitor.scrapers.reddit_scraper - INFO - Reddit API initialized successfully

2026-07-01 22:04:39,388 - RSSFeedScraper - INFO - RSS scraper initialized: feeds=2, html_urls=0 (0 with custom selectors), include_terms=0, exclude_terms=0

2026-07-01 22:04:39,388 - __main__ - INFO - RSS scraper initialized and enabled

2026-07-01 22:04:39,392 - Yiddish24Scraper - INFO - File downloader initialized with 55 terms, prefix 'Y2', and file type organization enabled

2026-07-01 22:04:39,392 - Yiddish24Scraper - INFO - Yiddish24 scraper initialized: data_id='latest', page_limit=100, total_pages=10, debug_html=True, include_terms=55, exclude_terms=0

2026-07-01 22:04:39,392 - __main__ - INFO - Yiddish24 scraper initialized and enabled

2026-07-01 22:04:39,395 - AhBlickLiveScraper - INFO - AhBlickLive scraper initialized: urls=1, max_items=100, include_terms=49, exclude_terms=0

2026-07-01 22:04:39,400 - TorahAnytimeScraper - INFO - File downloader initialized with 53 terms, prefix 'TA', and file type organization enabled

2026-07-01 22:04:39,400 - TorahAnytimeScraper - INFO - TorahAnytime scraper initialized: limit=400, offset=0, project_id=1, include_terms=53, exclude_terms=10

2026-07-01 22:04:39,402 - BatorahScraper - INFO - File downloader initialized with 19 terms, prefix 'BT', and file type organization enabled

2026-07-01 22:04:39,402 - BatorahScraper - INFO - Batorah scraper initialized: api_urls=2, page_size=100, include_terms=19, exclude_terms=1

2026-07-01 22:04:39,402 - KolHalashonScraper - INFO - Kol Halashon scraper initialized: base_url=https://www2.kolhalashon.com, max_shiurim=50, fetch_all_speakers=True, headless=False, exclude_filters=1, include_filters=0

2026-07-01 22:04:39,402 - __main__ - INFO - Ivelt scraper initialized and enabled

2026-07-01 22:04:39,403 - __main__ - INFO - Running 3 scrapers concurrently...

2026-07-01 22:04:39,403 - Yiddish24Scraper - INFO - Starting Yiddish24 scraping (max 10 pages, limit 100 items)

2026-07-01 22:04:39,497 - Yiddish24Scraper - INFO - Fetching Yiddish24 page 1 with data_id='latest', page_limit=100

2026-07-01 22:04:39,501 - IveltScraper - INFO - Loaded 181 include, 0 exclude filters from DB topic_filters

2026-07-01 22:04:39,501 - IveltScraper - INFO - Fetching ivelt active topics from https://www.ivelt.com/forum/search.php?search_id=active_topics...

2026-07-01 22:04:39,502 - RSSFeedScraper - INFO - Fetching RSS feed: https://www.theyeshivaworld.com/feed

2026-07-01 22:04:39,805 - RSSFeedScraper - INFO - Fetching RSS feed: https://www.thegatewaypundit.com/feed/

2026-07-01 22:04:40,214 - RSSFeedScraper - INFO - Total RSS/HTML results: 3

2026-07-01 22:04:51,605 - httpx - INFO - HTTP Request: POST https://www.yiddish24.com/ajax/cat_pagination.php "HTTP/1.1 200 OK"

2026-07-01 22:04:51,830 - Yiddish24Scraper - INFO - Received response with 495598 characters

2026-07-01 22:04:51,834 - Yiddish24Scraper - INFO - Extracted HTML from JSON result field (309057 chars)

2026-07-01 22:04:51,837 - Yiddish24Scraper - INFO - Saved HTML debug file to: /mnt/data/openclaw-shared/home/shared/Social_Media_Monitor/debug/yiddish24_page1_response.html

2026-07-01 22:04:51,925 - Yiddish24Scraper - INFO - Found 100 potential article elements on page 1

2026-07-01 22:04:51,966 - Yiddish24Scraper - INFO - Page 1 extraction: 100 successful, 0 failed out of 100 elements

2026-07-01 22:04:51,966 - Yiddish24Scraper - INFO - Parsed 100 articles from HTML on page 1

2026-07-01 22:04:51,992 - Yiddish24Scraper - INFO - Term 'צום פינטל -' matched in DESCRIPTION: 'צום פינטל - טאנצן נארמאל ביי חתונות'

2026-07-01 22:04:51,992 - Yiddish24Scraper - INFO - Downloading 1 media file(s) for article: הכל בכל

2026-07-01 22:04:52,492 - social_media_monitor.utils.file_downloader - INFO - Starting download from: https://cloudfront.yiddish24.com/_________________1_260_260701045519.mp3

2026-07-01 22:04:52,573 - social_media_monitor.utils.file_downloader - INFO - Successfully downloaded: Y2__________________1_260_260701045519.mp3 (4.1 MB)

2026-07-01 22:04:52,574 - social_media_monitor.utils.file_downloader - INFO - Organized audio file to: /mnt/data/openclaw-shared/home/shared/Social_Media_Monitor/downloads/Audio (audio/mpeg)

2026-07-01 22:04:52,586 - social_media_monitor.utils.media_tagger - INFO - Tagged Y2__________________1_260_260701045519.mp3: title, artist, album, comment, source_url, genre, publisher

2026-07-01 22:04:52,586 - Yiddish24Scraper - INFO - Renamed downloaded file to include match term: Y2__________________1_260_260701045519_צום_פינטל_-.mp3

2026-07-01 22:04:52,698 - Yiddish24Scraper - INFO - Term 'הרה"ג' matched in TITLE: 'הרה"ג ר' מרדכי הערש שפיטצער'

2026-07-01 22:04:52,698 - Yiddish24Scraper - INFO - Downloading 1 media file(s) for article: הרה"ג ר' מרדכי הערש שפיטצער

2026-07-01 22:04:53,493 - social_media_monitor.utils.file_downloader - INFO - Starting download from: https://cloudfront.yiddish24.com/__228_1782938792992_453_0_260701045047_0__388824429.mp3

2026-07-01 22:04:53,769 - social_media_monitor.utils.file_downloader - INFO - Successfully downloaded: Y2___228_1782938792992_453_0_260701045047_0__388824429.mp3 (46.3 MB)

2026-07-01 22:04:53,769 - social_media_monitor.utils.file_downloader - INFO - Organized audio file to: /mnt/data/openclaw-shared/home/shared/Social_Media_Monitor/downloads/Audio (audio/mpeg)

2026-07-01 22:04:53,786 - social_media_monitor.utils.media_tagger - INFO - Tagged Y2___228_1782938792992_453_0_260701045047_0__388824429.mp3: title, artist, album, source_url, genre, publisher

2026-07-01 22:04:53,786 - Yiddish24Scraper - INFO - Renamed downloaded file to include match term: Y2___228_1782938792992_453_0_260701045047_0__388824429_הרה_ג.mp3

2026-07-01 22:04:54,414 - Yiddish24Scraper - INFO - Page 1 summary: 10 new out of 100 total articles

2026-07-01 22:04:54,414 - Yiddish24Scraper - INFO - Page 1: 10/100 new items

2026-07-01 22:04:54,414 - Yiddish24Scraper - INFO - Low new item ratio (10.0%), stopping pagination

2026-07-01 22:04:54,415 - Yiddish24Scraper - INFO - Total Yiddish24 results found: 10

2026-07-01 22:04:54,423 - IveltScraper - INFO - Found 50 active topics on ivelt

2026-07-01 22:05:21,758 - IveltScraper - INFO - ivelt scrape complete. Checked 6 topics.

2026-07-01 22:05:21,758 - IveltScraper - INFO - All forums scrape complete. Found 39 new posts.

2026-07-01 22:05:21,758 - __main__ - INFO - Scraper rss completed: 3 results

2026-07-01 22:05:21,758 - __main__ - INFO - Scraper yiddish24 completed: 10 results

2026-07-01 22:05:21,758 - __main__ - INFO - Scraper ivelt completed: 39 results

2026-07-01 22:05:22,849 - social_media_monitor.email.sender - INFO - Email sent successfully at 2026-07-01 22:05:22.849831

2026-07-01 22:05:22,850 - __main__ - INFO - Email sent successfully

2026-07-01 22:05:22,850 - __main__ - INFO - Cache stats: 0 items in memory, 0 files on disk

2026-07-01 22:05:22,850 - __main__ - INFO - Monitor complete. Found 52 total results: 3 new rss results, 10 new yiddish24 results, 39 new ivelt results.

2026-07-01 22:05:22,850 - __main__ - INFO - === Social Media Monitor v2.0 Complete ===

2026-07-01 22:05:22,850 - social_media_monitor.database.connection - INFO - Closing all database connections

2026-07-01 22:05:22,952 - social_media_monitor.database.connection - INFO - Closed 5 database connections

2026-07-01 22:05:22,952 - __main__ - INFO - Closed all database connections

2026-07-01 22:05:22,954 - social_media_monitor.database.connection - INFO - Closing all database connections

2026-07-01 22:05:22,954 - social_media_monitor.database.connection - INFO - Closed 0 database connections

2026-07-01 22:30:00,641 - social_media_monitor.utils.logging_config - INFO - Posts log file: /mnt/data/openclaw-shared/home/shared/Social_Media_Monitor/logs/social_media_monitor_posts_2026-07-01.log

2026-07-01 22:30:00,641 - social_media_monitor.utils.logging_config - INFO - Downloads log file: /mnt/data/openclaw-shared/home/shared/Social_Media_Monitor/logs/social_media_monitor_downloads_2026-07-01.log

2026-07-01 22:30:00,641 - social_media_monitor.utils.logging_config - INFO - Logging configured with level INFO, log file: /mnt/data/openclaw-shared/home/shared/Social_Media_Monitor/logs/social_media_monitor_2026-07-01.log

2026-07-01 22:30:00,641 - __main__ - INFO - === Social Media Monitor v2.0 (Async) Started ===

2026-07-01 22:30:00,672 - social_media_monitor.config.settings - INFO - Configuration loaded from /home/openclaw/Social_Media_Monitor/config.yaml

2026-07-01 22:30:00,672 - __main__ - INFO - Configuration loaded and validated successfully

2026-07-01 22:30:00,674 - social_media_monitor.database.connection - INFO - Database connection pool initialized with 5 connections

2026-07-01 22:30:00,675 - social_media_monitor.database.connection - INFO - add_speaker_filters column already exists

2026-07-01 22:30:00,688 - social_media_monitor.database.connection - INFO - Database setup completed successfully

2026-07-01 22:30:00,689 - __main__ - INFO - Database initialized at: /mnt/data/openclaw-shared/home/shared/Social_Media_Monitor/social_media_monitor.db

2026-07-01 22:30:00,691 - social_media_monitor.scrapers.reddit_scraper - INFO - Reddit API initialized successfully

2026-07-01 22:30:00,691 - __main__ - INFO - GitHub scraper initialized and enabled

2026-07-01 22:30:00,691 - __main__ - INFO - GitHub feed scraper initialized and enabled

2026-07-01 22:30:00,692 - RSSFeedScraper - INFO - RSS scraper initialized: feeds=2, html_urls=0 (0 with custom selectors), include_terms=0, exclude_terms=0

2026-07-01 22:30:00,692 - __main__ - INFO - LinkedIn Feed scraper initialized and enabled

2026-07-01 22:30:00,694 - Yiddish24Scraper - INFO - File downloader initialized with 55 terms, prefix 'Y2', and file type organization enabled

2026-07-01 22:30:00,694 - Yiddish24Scraper - INFO - Yiddish24 scraper initialized: data_id='latest', page_limit=100, total_pages=10, debug_html=True, include_terms=55, exclude_terms=0

2026-07-01 22:30:00,697 - AhBlickLiveScraper - INFO - AhBlickLive scraper initialized: urls=1, max_items=100, include_terms=49, exclude_terms=0

2026-07-01 22:30:00,701 - TorahAnytimeScraper - INFO - File downloader initialized with 53 terms, prefix 'TA', and file type organization enabled

2026-07-01 22:30:00,701 - TorahAnytimeScraper - INFO - TorahAnytime scraper initialized: limit=400, offset=0, project_id=1, include_terms=53, exclude_terms=10

2026-07-01 22:30:00,703 - BatorahScraper - INFO - File downloader initialized with 19 terms, prefix 'BT', and file type organization enabled

2026-07-01 22:30:00,703 - BatorahScraper - INFO - Batorah scraper initialized: api_urls=2, page_size=100, include_terms=19, exclude_terms=1

2026-07-01 22:30:00,704 - KolHalashonScraper - INFO - Kol Halashon scraper initialized: base_url=https://www2.kolhalashon.com, max_shiurim=50, fetch_all_speakers=True, headless=False, exclude_filters=1, include_filters=0

2026-07-01 22:30:00,704 - __main__ - INFO - Running 3 scrapers concurrently...

2026-07-01 22:30:00,704 - GitHubScraper - INFO - Searching GitHub for terms: ['paperclip', '"Hermes Agent"', 'multica']

2026-07-01 22:30:00,798 - LinkedInFeedScraper - INFO - Fetching LinkedIn feed (25 posts)

2026-07-01 22:30:03,420 - LinkedInFeedScraper - ERROR - LinkedIn feed fetch failed: Unauthorized: LinkedIn session cookies are expired or invalid.

2026-07-01 22:30:03,430 - LinkedInFeedScraper - INFO - LinkedIn Feed scrape complete. Found 0 new posts.

2026-07-01 22:30:04,034 - httpx - INFO - HTTP Request: GET https://api.github.com/users/abeperl/received_events?per_page=100&page=1 "HTTP/1.1 200 OK"

2026-07-01 22:30:04,317 - GitHubScraper - INFO - GitHub feed: 40 fetched across pages, 13 new events for abeperl

2026-07-01 22:30:04,428 - httpx - INFO - HTTP Request: GET https://api.github.com/search/repositories?q=paperclip+in%3Aname%2Cdescription%2Creadme+stars%3A%3E150&sort=updated&order=desc&per_page=20 "HTTP/1.1 200 OK"

2026-07-01 22:30:04,453 - GitHubScraper - INFO - Found 242 GitHub repositories for term: paperclip

2026-07-01 22:30:05,524 - httpx - INFO - HTTP Request: GET https://api.github.com/search/repositories?q=%22Hermes+Agent%22+in%3Aname%2Cdescription%2Creadme+stars%3A%3E150&sort=updated&order=desc&per_page=20 "HTTP/1.1 200 OK"

2026-07-01 22:30:05,526 - GitHubScraper - INFO - Found 413 GitHub repositories for term: "Hermes Agent"

2026-07-01 22:30:06,584 - httpx - INFO - HTTP Request: GET https://api.github.com/search/repositories?q=multica+in%3Aname%2Cdescription%2Creadme+stars%3A%3E150&sort=updated&order=desc&per_page=20 "HTTP/1.1 200 OK"

2026-07-01 22:30:06,587 - GitHubScraper - INFO - Found 54 GitHub repositories for term: multica

2026-07-01 22:30:06,740 - GitHubScraper - INFO - Total GitHub results found: 0

2026-07-01 22:30:06,740 - __main__ - INFO - Scraper github completed: 0 results

2026-07-01 22:30:06,740 - __main__ - INFO - Scraper github_feed completed: 13 results

2026-07-01 22:30:06,740 - __main__ - INFO - Scraper linkedin_feed completed: 0 results

2026-07-01 22:30:06,741 - __main__ - WARNING - LinkedIn Feed: 11 consecutive failures (0 results)

2026-07-01 22:30:07,536 - social_media_monitor.email.sender - INFO - Email sent successfully at 2026-07-01 22:30:07.536360

2026-07-01 22:30:07,536 - __main__ - INFO - Email sent successfully

2026-07-01 22:30:07,536 - __main__ - INFO - Cache stats: 0 items in memory, 0 files on disk

2026-07-01 22:30:07,536 - __main__ - INFO - Monitor complete. Found 13 total results: 13 new github feed results.

2026-07-01 22:30:07,536 - __main__ - INFO - === Social Media Monitor v2.0 Complete ===

2026-07-01 22:30:07,536 - social_media_monitor.database.connection - INFO - Closing all database connections

2026-07-01 22:30:07,566 - social_media_monitor.database.connection - INFO - Closed 5 database connections

2026-07-01 22:30:07,566 - __main__ - INFO - Closed all database connections

2026-07-01 22:30:07,568 - social_media_monitor.database.connection - INFO - Closing all database connections

2026-07-01 22:30:07,568 - social_media_monitor.database.connection - INFO - Closed 0 database connections

2026-07-01 23:04:39,300 - social_media_monitor.utils.logging_config - INFO - Posts log file: /mnt/data/openclaw-shared/home/shared/Social_Media_Monitor/logs/social_media_monitor_posts_2026-07-01.log

2026-07-01 23:04:39,300 - social_media_monitor.utils.logging_config - INFO - Downloads log file: /mnt/data/openclaw-shared/home/shared/Social_Media_Monitor/logs/social_media_monitor_downloads_2026-07-01.log

2026-07-01 23:04:39,300 - social_media_monitor.utils.logging_config - INFO - Logging configured with level INFO, log file: /mnt/data/openclaw-shared/home/shared/Social_Media_Monitor/logs/social_media_monitor_2026-07-01.log

2026-07-01 23:04:39,300 - __main__ - INFO - === Social Media Monitor v2.0 (Async) Started ===

2026-07-01 23:04:39,330 - social_media_monitor.config.settings - INFO - Configuration loaded from /home/openclaw/Social_Media_Monitor/config.yaml

2026-07-01 23:04:39,330 - __main__ - INFO - Configuration loaded and validated successfully

2026-07-01 23:04:39,332 - social_media_monitor.database.connection - INFO - Database connection pool initialized with 5 connections

2026-07-01 23:04:39,333 - social_media_monitor.database.connection - INFO - add_speaker_filters column already exists

2026-07-01 23:04:39,362 - social_media_monitor.database.connection - INFO - Database setup completed successfully

2026-07-01 23:04:39,362 - __main__ - INFO - Database initialized at: /mnt/data/openclaw-shared/home/shared/Social_Media_Monitor/social_media_monitor.db

2026-07-01 23:04:39,365 - social_media_monitor.scrapers.reddit_scraper - INFO - Reddit API initialized successfully

2026-07-01 23:04:39,365 - RSSFeedScraper - INFO - RSS scraper initialized: feeds=2, html_urls=0 (0 with custom selectors), include_terms=0, exclude_terms=0

2026-07-01 23:04:39,365 - __main__ - INFO - RSS scraper initialized and enabled

2026-07-01 23:04:39,368 - Yiddish24Scraper - INFO - File downloader initialized with 55 terms, prefix 'Y2', and file type organization enabled

2026-07-01 23:04:39,368 - Yiddish24Scraper - INFO - Yiddish24 scraper initialized: data_id='latest', page_limit=100, total_pages=10, debug_html=True, include_terms=55, exclude_terms=0

2026-07-01 23:04:39,368 - __main__ - INFO - Yiddish24 scraper initialized and enabled

2026-07-01 23:04:39,371 - AhBlickLiveScraper - INFO - AhBlickLive scraper initialized: urls=1, max_items=100, include_terms=49, exclude_terms=0

2026-07-01 23:04:39,375 - TorahAnytimeScraper - INFO - File downloader initialized with 53 terms, prefix 'TA', and file type organization enabled

2026-07-01 23:04:39,375 - TorahAnytimeScraper - INFO - TorahAnytime scraper initialized: limit=400, offset=0, project_id=1, include_terms=53, exclude_terms=10

2026-07-01 23:04:39,377 - BatorahScraper - INFO - File downloader initialized with 19 terms, prefix 'BT', and file type organization enabled

2026-07-01 23:04:39,377 - BatorahScraper - INFO - Batorah scraper initialized: api_urls=2, page_size=100, include_terms=19, exclude_terms=1

2026-07-01 23:04:39,377 - KolHalashonScraper - INFO - Kol Halashon scraper initialized: base_url=https://www2.kolhalashon.com, max_shiurim=50, fetch_all_speakers=True, headless=False, exclude_filters=1, include_filters=0

2026-07-01 23:04:39,377 - __main__ - INFO - Ivelt scraper initialized and enabled

2026-07-01 23:04:39,377 - __main__ - INFO - Running 3 scrapers concurrently...

2026-07-01 23:04:39,378 - Yiddish24Scraper - INFO - Starting Yiddish24 scraping (max 10 pages, limit 100 items)

2026-07-01 23:04:39,463 - Yiddish24Scraper - INFO - Fetching Yiddish24 page 1 with data_id='latest', page_limit=100

2026-07-01 23:04:39,467 - IveltScraper - INFO - Loaded 181 include, 0 exclude filters from DB topic_filters

2026-07-01 23:04:39,467 - IveltScraper - INFO - Fetching ivelt active topics from https://www.ivelt.com/forum/search.php?search_id=active_topics...

2026-07-01 23:04:39,469 - RSSFeedScraper - INFO - Fetching RSS feed: https://www.theyeshivaworld.com/feed

2026-07-01 23:04:39,713 - RSSFeedScraper - INFO - Fetching RSS feed: https://www.thegatewaypundit.com/feed/

2026-07-01 23:04:40,122 - RSSFeedScraper - INFO - Total RSS/HTML results: 3

2026-07-01 23:04:46,099 - httpx - INFO - HTTP Request: POST https://www.yiddish24.com/ajax/cat_pagination.php "HTTP/1.1 200 OK"

2026-07-01 23:04:46,325 - Yiddish24Scraper - INFO - Received response with 498916 characters

2026-07-01 23:04:46,328 - Yiddish24Scraper - INFO - Extracted HTML from JSON result field (309680 chars)

2026-07-01 23:04:46,331 - Yiddish24Scraper - INFO - Saved HTML debug file to: /mnt/data/openclaw-shared/home/shared/Social_Media_Monitor/debug/yiddish24_page1_response.html

2026-07-01 23:04:46,412 - Yiddish24Scraper - INFO - Found 100 potential article elements on page 1

2026-07-01 23:04:46,453 - Yiddish24Scraper - INFO - Page 1 extraction: 100 successful, 0 failed out of 100 elements

2026-07-01 23:04:46,453 - Yiddish24Scraper - INFO - Parsed 100 articles from HTML on page 1

2026-07-01 23:04:47,264 - Yiddish24Scraper - INFO - Page 1 summary: 11 new out of 100 total articles

2026-07-01 23:04:47,265 - Yiddish24Scraper - INFO - Page 1: 11/100 new items

2026-07-01 23:04:47,265 - Yiddish24Scraper - INFO - Low new item ratio (11.0%), stopping pagination

2026-07-01 23:04:47,265 - Yiddish24Scraper - INFO - Total Yiddish24 results found: 11

2026-07-01 23:04:51,874 - IveltScraper - INFO - Found 50 active topics on ivelt

2026-07-01 23:05:11,618 - IveltScraper - INFO - ivelt scrape complete. Checked 5 topics.

2026-07-01 23:05:11,619 - IveltScraper - INFO - All forums scrape complete. Found 11 new posts.

2026-07-01 23:05:11,619 - __main__ - INFO - Scraper rss completed: 3 results

2026-07-01 23:05:11,619 - __main__ - INFO - Scraper yiddish24 completed: 11 results

2026-07-01 23:05:11,619 - __main__ - INFO - Scraper ivelt completed: 11 results

2026-07-01 23:05:12,528 - social_media_monitor.email.sender - INFO - Email sent successfully at 2026-07-01 23:05:12.527983

2026-07-01 23:05:12,528 - __main__ - INFO - Email sent successfully

2026-07-01 23:05:12,528 - __main__ - INFO - Cache stats: 0 items in memory, 0 files on disk

2026-07-01 23:05:12,528 - __main__ - INFO - Monitor complete. Found 25 total results: 3 new rss results, 11 new yiddish24 results, 11 new ivelt results.

2026-07-01 23:05:12,528 - __main__ - INFO - === Social Media Monitor v2.0 Complete ===

2026-07-01 23:05:12,528 - social_media_monitor.database.connection - INFO - Closing all database connections

2026-07-01 23:05:12,576 - social_media_monitor.database.connection - INFO - Closed 5 database connections

2026-07-01 23:05:12,576 - __main__ - INFO - Closed all database connections

2026-07-01 23:05:12,577 - social_media_monitor.database.connection - INFO - Closing all database connections

2026-07-01 23:05:12,577 - social_media_monitor.database.connection - INFO - Closed 0 database connections

2026-07-01 23:30:00,627 - social_media_monitor.utils.logging_config - INFO - Posts log file: /mnt/data/openclaw-shared/home/shared/Social_Media_Monitor/logs/social_media_monitor_posts_2026-07-01.log

2026-07-01 23:30:00,627 - social_media_monitor.utils.logging_config - INFO - Downloads log file: /mnt/data/openclaw-shared/home/shared/Social_Media_Monitor/logs/social_media_monitor_downloads_2026-07-01.log

2026-07-01 23:30:00,627 - social_media_monitor.utils.logging_config - INFO - Logging configured with level INFO, log file: /mnt/data/openclaw-shared/home/shared/Social_Media_Monitor/logs/social_media_monitor_2026-07-01.log

2026-07-01 23:30:00,627 - __main__ - INFO - === Social Media Monitor v2.0 (Async) Started ===

2026-07-01 23:30:00,662 - social_media_monitor.config.settings - INFO - Configuration loaded from /home/openclaw/Social_Media_Monitor/config.yaml

2026-07-01 23:30:00,662 - __main__ - INFO - Configuration loaded and validated successfully

2026-07-01 23:30:00,664 - social_media_monitor.database.connection - INFO - Database connection pool initialized with 5 connections

2026-07-01 23:30:00,665 - social_media_monitor.database.connection - INFO - add_speaker_filters column already exists

2026-07-01 23:30:00,679 - social_media_monitor.database.connection - INFO - Database setup completed successfully

2026-07-01 23:30:00,679 - __main__ - INFO - Database initialized at: /mnt/data/openclaw-shared/home/shared/Social_Media_Monitor/social_media_monitor.db

2026-07-01 23:30:00,723 - social_media_monitor.scrapers.reddit_scraper - INFO - Reddit API initialized successfully

2026-07-01 23:30:00,723 - __main__ - INFO - GitHub scraper initialized and enabled

2026-07-01 23:30:00,723 - __main__ - INFO - GitHub feed scraper initialized and enabled

2026-07-01 23:30:00,724 - RSSFeedScraper - INFO - RSS scraper initialized: feeds=2, html_urls=0 (0 with custom selectors), include_terms=0, exclude_terms=0

2026-07-01 23:30:00,724 - __main__ - INFO - LinkedIn Feed scraper initialized and enabled

2026-07-01 23:30:00,727 - Yiddish24Scraper - INFO - File downloader initialized with 55 terms, prefix 'Y2', and file type organization enabled

2026-07-01 23:30:00,727 - Yiddish24Scraper - INFO - Yiddish24 scraper initialized: data_id='latest', page_limit=100, total_pages=10, debug_html=True, include_terms=55, exclude_terms=0

2026-07-01 23:30:00,730 - AhBlickLiveScraper - INFO - AhBlickLive scraper initialized: urls=1, max_items=100, include_terms=49, exclude_terms=0

2026-07-01 23:30:00,734 - TorahAnytimeScraper - INFO - File downloader initialized with 53 terms, prefix 'TA', and file type organization enabled

2026-07-01 23:30:00,734 - TorahAnytimeScraper - INFO - TorahAnytime scraper initialized: limit=400, offset=0, project_id=1, include_terms=53, exclude_terms=10

2026-07-01 23:30:00,736 - BatorahScraper - INFO - File downloader initialized with 19 terms, prefix 'BT', and file type organization enabled

2026-07-01 23:30:00,736 - BatorahScraper - INFO - Batorah scraper initialized: api_urls=2, page_size=100, include_terms=19, exclude_terms=1

2026-07-01 23:30:00,737 - KolHalashonScraper - INFO - Kol Halashon scraper initialized: base_url=https://www2.kolhalashon.com, max_shiurim=50, fetch_all_speakers=True, headless=False, exclude_filters=1, include_filters=0

2026-07-01 23:30:00,737 - __main__ - INFO - Running 3 scrapers concurrently...

2026-07-01 23:30:00,737 - GitHubScraper - INFO - Searching GitHub for terms: ['paperclip', '"Hermes Agent"', 'multica']

2026-07-01 23:30:00,833 - LinkedInFeedScraper - INFO - Fetching LinkedIn feed (25 posts)

2026-07-01 23:30:04,476 - LinkedInFeedScraper - ERROR - LinkedIn feed fetch failed: Unauthorized: LinkedIn session cookies are expired or invalid.

2026-07-01 23:30:04,483 - LinkedInFeedScraper - INFO - LinkedIn Feed scrape complete. Found 0 new posts.

2026-07-01 23:30:05,060 - httpx - INFO - HTTP Request: GET https://api.github.com/users/abeperl/received_events?per_page=100&page=1 "HTTP/1.1 200 OK"

2026-07-01 23:30:05,335 - GitHubScraper - INFO - GitHub feed: 40 fetched across pages, 0 new events for abeperl

2026-07-01 23:30:05,501 - httpx - INFO - HTTP Request: GET https://api.github.com/search/repositories?q=paperclip+in%3Aname%2Cdescription%2Creadme+stars%3A%3E150&sort=updated&order=desc&per_page=20 "HTTP/1.1 200 OK"

2026-07-01 23:30:05,601 - GitHubScraper - INFO - Found 242 GitHub repositories for term: paperclip

2026-07-01 23:30:06,706 - httpx - INFO - HTTP Request: GET https://api.github.com/search/repositories?q=%22Hermes+Agent%22+in%3Aname%2Cdescription%2Creadme+stars%3A%3E150&sort=updated&order=desc&per_page=20 "HTTP/1.1 200 OK"

2026-07-01 23:30:06,711 - GitHubScraper - INFO - Found 413 GitHub repositories for term: "Hermes Agent"

2026-07-01 23:30:08,092 - httpx - INFO - HTTP Request: GET https://api.github.com/search/repositories?q=multica+in%3Aname%2Cdescription%2Creadme+stars%3A%3E150&sort=updated&order=desc&per_page=20 "HTTP/1.1 200 OK"

2026-07-01 23:30:08,184 - GitHubScraper - INFO - Found 54 GitHub repositories for term: multica

2026-07-01 23:30:08,325 - GitHubScraper - INFO - Total GitHub results found: 0

2026-07-01 23:30:08,325 - __main__ - INFO - Scraper github completed: 0 results

2026-07-01 23:30:08,325 - __main__ - INFO - Scraper github_feed completed: 0 results

2026-07-01 23:30:08,325 - __main__ - INFO - Scraper linkedin_feed completed: 0 results

2026-07-01 23:30:08,326 - __main__ - WARNING - LinkedIn Feed: 12 consecutive failures (0 results)

2026-07-01 23:30:08,326 - __main__ - INFO - No new results found across all enabled sources

2026-07-01 23:30:08,326 - __main__ - INFO - Cache stats: 0 items in memory, 0 files on disk

2026-07-01 23:30:08,326 - __main__ - INFO - Monitor complete. No new results found across all sources.

2026-07-01 23:30:08,326 - __main__ - INFO - === Social Media Monitor v2.0 Complete ===

2026-07-01 23:30:08,326 - social_media_monitor.database.connection - INFO - Closing all database connections

2026-07-01 23:30:08,359 - social_media_monitor.database.connection - INFO - Closed 5 database connections

2026-07-01 23:30:08,360 - __main__ - INFO - Closed all database connections

2026-07-01 23:30:08,361 - social_media_monitor.database.connection - INFO - Closing all database connections

2026-07-01 23:30:08,362 - social_media_monitor.database.connection - INFO - Closed 0 database connections