← Logs

social_media_monitor_2026-07-05.log

2026-07-05 19:04:44,879 - Yiddish24Scraper - INFO - Page 1 summary: 2 new out of 100 total articles

2026-07-05 19:04:44,879 - Yiddish24Scraper - INFO - Page 1: 2/100 new items

2026-07-05 19:04:44,879 - Yiddish24Scraper - INFO - Low new item ratio (2.0%), stopping pagination

2026-07-05 19:04:44,880 - Yiddish24Scraper - INFO - Total Yiddish24 results found: 2

2026-07-05 19:04:51,711 - IveltScraper - INFO - Found 50 active topics on ivelt

2026-07-05 19:05:29,008 - IveltScraper - INFO - Filtering post containing exclude term: '*ביטע מאכט זיכער בלויז צו באריכטן קראנטע נייעס*'

2026-07-05 19:05:34,868 - IveltScraper - INFO - ivelt scrape complete. Checked 8 topics.

2026-07-05 19:05:34,868 - IveltScraper - INFO - All forums scrape complete. Found 50 new posts.

2026-07-05 19:05:34,868 - __main__ - INFO - Scraper rss completed: 3 results

2026-07-05 19:05:34,868 - __main__ - INFO - Scraper yiddish24 completed: 2 results

2026-07-05 19:05:34,869 - __main__ - INFO - Scraper ivelt completed: 50 results

2026-07-05 19:05:35,766 - social_media_monitor.email.sender - INFO - Email sent successfully at 2026-07-05 19:05:35.766818

2026-07-05 19:05:35,767 - __main__ - INFO - Email sent successfully

2026-07-05 19:05:35,767 - __main__ - INFO - Cache stats: 0 items in memory, 0 files on disk

2026-07-05 19:05:35,767 - __main__ - INFO - Monitor complete. Found 55 total results: 3 new rss results, 2 new yiddish24 results, 50 new ivelt results.

2026-07-05 19:05:35,767 - __main__ - INFO - === Social Media Monitor v2.0 Complete ===

2026-07-05 19:05:35,767 - social_media_monitor.database.connection - INFO - Closing all database connections

2026-07-05 19:05:35,907 - social_media_monitor.database.connection - INFO - Closed 5 database connections

2026-07-05 19:05:35,907 - __main__ - INFO - Closed all database connections

2026-07-05 19:05:35,909 - social_media_monitor.database.connection - INFO - Closing all database connections

2026-07-05 19:05:35,909 - social_media_monitor.database.connection - INFO - Closed 0 database connections

2026-07-05 19:30:00,652 - social_media_monitor.utils.logging_config - INFO - Posts log file: /mnt/data/openclaw-shared/home/shared/Social_Media_Monitor/logs/social_media_monitor_posts_2026-07-05.log

2026-07-05 19:30:00,653 - social_media_monitor.utils.logging_config - INFO - Downloads log file: /mnt/data/openclaw-shared/home/shared/Social_Media_Monitor/logs/social_media_monitor_downloads_2026-07-05.log

2026-07-05 19:30:00,653 - social_media_monitor.utils.logging_config - INFO - Logging configured with level INFO, log file: /mnt/data/openclaw-shared/home/shared/Social_Media_Monitor/logs/social_media_monitor_2026-07-05.log

2026-07-05 19:30:00,653 - __main__ - INFO - === Social Media Monitor v2.0 (Async) Started ===

2026-07-05 19:30:00,684 - social_media_monitor.config.settings - INFO - Configuration loaded from /home/openclaw/Social_Media_Monitor/config.yaml

2026-07-05 19:30:00,685 - __main__ - INFO - Configuration loaded and validated successfully

2026-07-05 19:30:00,687 - social_media_monitor.database.connection - INFO - Database connection pool initialized with 5 connections

2026-07-05 19:30:00,688 - social_media_monitor.database.connection - INFO - add_speaker_filters column already exists

2026-07-05 19:30:00,702 - social_media_monitor.database.connection - INFO - Database setup completed successfully

2026-07-05 19:30:00,703 - __main__ - INFO - Database initialized at: /mnt/data/openclaw-shared/home/shared/Social_Media_Monitor/social_media_monitor.db

2026-07-05 19:30:00,705 - social_media_monitor.scrapers.reddit_scraper - INFO - Reddit API initialized successfully

2026-07-05 19:30:00,705 - __main__ - INFO - GitHub scraper initialized and enabled

2026-07-05 19:30:00,706 - RSSFeedScraper - INFO - RSS scraper initialized: feeds=2, html_urls=0 (0 with custom selectors), include_terms=0, exclude_terms=0

2026-07-05 19:30:00,706 - __main__ - INFO - LinkedIn Feed scraper initialized and enabled

2026-07-05 19:30:00,709 - Yiddish24Scraper - INFO - File downloader initialized with 55 terms, prefix 'Y2', and file type organization enabled

2026-07-05 19:30:00,709 - Yiddish24Scraper - INFO - Yiddish24 scraper initialized: data_id='latest', page_limit=100, total_pages=10, debug_html=True, include_terms=55, exclude_terms=0

2026-07-05 19:30:00,712 - AhBlickLiveScraper - INFO - AhBlickLive scraper initialized: urls=1, max_items=100, include_terms=49, exclude_terms=0

2026-07-05 19:30:00,718 - TorahAnytimeScraper - INFO - File downloader initialized with 53 terms, prefix 'TA', and file type organization enabled

2026-07-05 19:30:00,718 - TorahAnytimeScraper - INFO - TorahAnytime scraper initialized: limit=400, offset=0, project_id=1, include_terms=53, exclude_terms=10

2026-07-05 19:30:00,720 - BatorahScraper - INFO - File downloader initialized with 19 terms, prefix 'BT', and file type organization enabled

2026-07-05 19:30:00,720 - BatorahScraper - INFO - Batorah scraper initialized: api_urls=2, page_size=100, include_terms=19, exclude_terms=1

2026-07-05 19:30:00,721 - KolHalashonScraper - INFO - Kol Halashon scraper initialized: base_url=https://www2.kolhalashon.com, max_shiurim=50, fetch_all_speakers=True, headless=False, exclude_filters=1, include_filters=0

2026-07-05 19:30:00,721 - __main__ - INFO - Running 2 scrapers concurrently...

2026-07-05 19:30:00,721 - GitHubScraper - INFO - Searching GitHub for terms: ['paperclip', '"Hermes Agent"', 'multica']

2026-07-05 19:30:00,818 - LinkedInFeedScraper - INFO - Fetching LinkedIn feed (25 posts)

2026-07-05 19:30:06,409 - LinkedInFeedScraper - ERROR - LinkedIn feed fetch failed: Unauthorized: LinkedIn session cookies are expired or invalid.

2026-07-05 19:30:06,416 - LinkedInFeedScraper - INFO - LinkedIn Feed scrape complete. Found 0 new posts.

2026-07-05 19:30:07,536 - httpx - INFO - HTTP Request: GET https://api.github.com/search/repositories?q=paperclip+in%3Aname%2Cdescription%2Creadme+stars%3A%3E150&sort=updated&order=desc&per_page=20 "HTTP/1.1 200 OK"

2026-07-05 19:30:07,562 - GitHubScraper - INFO - Found 242 GitHub repositories for term: paperclip

2026-07-05 19:30:08,546 - httpx - INFO - HTTP Request: GET https://api.github.com/search/repositories?q=%22Hermes+Agent%22+in%3Aname%2Cdescription%2Creadme+stars%3A%3E150&sort=updated&order=desc&per_page=20 "HTTP/1.1 200 OK"

2026-07-05 19:30:08,550 - GitHubScraper - INFO - Found 426 GitHub repositories for term: "Hermes Agent"

2026-07-05 19:30:09,391 - httpx - INFO - HTTP Request: GET https://api.github.com/search/repositories?q=multica+in%3Aname%2Cdescription%2Creadme+stars%3A%3E150&sort=updated&order=desc&per_page=20 "HTTP/1.1 200 OK"

2026-07-05 19:30:09,394 - GitHubScraper - INFO - Found 57 GitHub repositories for term: multica

2026-07-05 19:30:09,521 - GitHubScraper - INFO - Total GitHub results found: 0

2026-07-05 19:30:09,522 - __main__ - INFO - Scraper github completed: 0 results

2026-07-05 19:30:09,522 - __main__ - INFO - Scraper linkedin_feed completed: 0 results

2026-07-05 19:30:09,522 - __main__ - WARNING - LinkedIn Feed: 14 consecutive failures (fetch failed)

2026-07-05 19:30:09,523 - __main__ - INFO - No new results found across all enabled sources

2026-07-05 19:30:09,523 - __main__ - INFO - Cache stats: 0 items in memory, 0 files on disk

2026-07-05 19:30:09,523 - __main__ - INFO - Monitor complete. No new results found across all sources.

2026-07-05 19:30:09,523 - __main__ - INFO - === Social Media Monitor v2.0 Complete ===

2026-07-05 19:30:09,523 - social_media_monitor.database.connection - INFO - Closing all database connections

2026-07-05 19:30:09,536 - social_media_monitor.database.connection - INFO - Closed 5 database connections

2026-07-05 19:30:09,536 - __main__ - INFO - Closed all database connections

2026-07-05 19:30:09,538 - social_media_monitor.database.connection - INFO - Closing all database connections

2026-07-05 19:30:09,538 - social_media_monitor.database.connection - INFO - Closed 0 database connections

2026-07-05 20:04:39,254 - social_media_monitor.utils.logging_config - INFO - Posts log file: /mnt/data/openclaw-shared/home/shared/Social_Media_Monitor/logs/social_media_monitor_posts_2026-07-05.log

2026-07-05 20:04:39,255 - social_media_monitor.utils.logging_config - INFO - Downloads log file: /mnt/data/openclaw-shared/home/shared/Social_Media_Monitor/logs/social_media_monitor_downloads_2026-07-05.log

2026-07-05 20:04:39,255 - social_media_monitor.utils.logging_config - INFO - Logging configured with level INFO, log file: /mnt/data/openclaw-shared/home/shared/Social_Media_Monitor/logs/social_media_monitor_2026-07-05.log

2026-07-05 20:04:39,255 - __main__ - INFO - === Social Media Monitor v2.0 (Async) Started ===

2026-07-05 20:04:39,285 - social_media_monitor.config.settings - INFO - Configuration loaded from /home/openclaw/Social_Media_Monitor/config.yaml

2026-07-05 20:04:39,285 - __main__ - INFO - Configuration loaded and validated successfully

2026-07-05 20:04:39,287 - social_media_monitor.database.connection - INFO - Database connection pool initialized with 5 connections

2026-07-05 20:04:39,288 - social_media_monitor.database.connection - INFO - add_speaker_filters column already exists

2026-07-05 20:04:39,313 - social_media_monitor.database.connection - INFO - Database setup completed successfully

2026-07-05 20:04:39,313 - __main__ - INFO - Database initialized at: /mnt/data/openclaw-shared/home/shared/Social_Media_Monitor/social_media_monitor.db

2026-07-05 20:04:39,315 - social_media_monitor.scrapers.reddit_scraper - INFO - Reddit API initialized successfully

2026-07-05 20:04:39,316 - RSSFeedScraper - INFO - RSS scraper initialized: feeds=2, html_urls=0 (0 with custom selectors), include_terms=0, exclude_terms=0

2026-07-05 20:04:39,316 - __main__ - INFO - RSS scraper initialized and enabled

2026-07-05 20:04:39,319 - Yiddish24Scraper - INFO - File downloader initialized with 55 terms, prefix 'Y2', and file type organization enabled

2026-07-05 20:04:39,319 - Yiddish24Scraper - INFO - Yiddish24 scraper initialized: data_id='latest', page_limit=100, total_pages=10, debug_html=True, include_terms=55, exclude_terms=0

2026-07-05 20:04:39,319 - __main__ - INFO - Yiddish24 scraper initialized and enabled

2026-07-05 20:04:39,321 - AhBlickLiveScraper - INFO - AhBlickLive scraper initialized: urls=1, max_items=100, include_terms=49, exclude_terms=0

2026-07-05 20:04:39,325 - TorahAnytimeScraper - INFO - File downloader initialized with 53 terms, prefix 'TA', and file type organization enabled

2026-07-05 20:04:39,326 - TorahAnytimeScraper - INFO - TorahAnytime scraper initialized: limit=400, offset=0, project_id=1, include_terms=53, exclude_terms=10

2026-07-05 20:04:39,328 - BatorahScraper - INFO - File downloader initialized with 19 terms, prefix 'BT', and file type organization enabled

2026-07-05 20:04:39,328 - BatorahScraper - INFO - Batorah scraper initialized: api_urls=2, page_size=100, include_terms=19, exclude_terms=1

2026-07-05 20:04:39,328 - KolHalashonScraper - INFO - Kol Halashon scraper initialized: base_url=https://www2.kolhalashon.com, max_shiurim=50, fetch_all_speakers=True, headless=False, exclude_filters=1, include_filters=0

2026-07-05 20:04:39,328 - __main__ - INFO - Ivelt scraper initialized and enabled

2026-07-05 20:04:39,328 - __main__ - INFO - Running 3 scrapers concurrently...

2026-07-05 20:04:39,328 - Yiddish24Scraper - INFO - Starting Yiddish24 scraping (max 10 pages, limit 100 items)

2026-07-05 20:04:39,420 - Yiddish24Scraper - INFO - Fetching Yiddish24 page 1 with data_id='latest', page_limit=100

2026-07-05 20:04:39,425 - IveltScraper - INFO - Loaded 181 include, 0 exclude filters from DB topic_filters

2026-07-05 20:04:39,425 - IveltScraper - INFO - Fetching ivelt active topics from https://www.ivelt.com/forum/search.php?search_id=active_topics...

2026-07-05 20:04:39,427 - RSSFeedScraper - INFO - Fetching RSS feed: https://www.theyeshivaworld.com/feed

2026-07-05 20:04:39,674 - RSSFeedScraper - INFO - Fetching RSS feed: https://www.thegatewaypundit.com/feed/

2026-07-05 20:04:40,125 - RSSFeedScraper - INFO - Total RSS/HTML results: 3

2026-07-05 20:04:43,573 - httpx - INFO - HTTP Request: POST https://www.yiddish24.com/ajax/cat_pagination.php "HTTP/1.1 200 OK"

2026-07-05 20:04:43,786 - Yiddish24Scraper - INFO - Received response with 438654 characters

2026-07-05 20:04:43,789 - Yiddish24Scraper - INFO - Extracted HTML from JSON result field (291923 chars)

2026-07-05 20:04:43,793 - Yiddish24Scraper - INFO - Saved HTML debug file to: /mnt/data/openclaw-shared/home/shared/Social_Media_Monitor/debug/yiddish24_page1_response.html

2026-07-05 20:04:43,894 - Yiddish24Scraper - INFO - Found 100 potential article elements on page 1

2026-07-05 20:04:43,934 - Yiddish24Scraper - INFO - Page 1 extraction: 100 successful, 0 failed out of 100 elements

2026-07-05 20:04:43,934 - Yiddish24Scraper - INFO - Parsed 100 articles from HTML on page 1

2026-07-05 20:04:44,682 - Yiddish24Scraper - INFO - Page 1 summary: 9 new out of 100 total articles

2026-07-05 20:04:44,683 - Yiddish24Scraper - INFO - Page 1: 9/100 new items

2026-07-05 20:04:44,683 - Yiddish24Scraper - INFO - Low new item ratio (9.0%), stopping pagination

2026-07-05 20:04:44,683 - Yiddish24Scraper - INFO - Total Yiddish24 results found: 9

2026-07-05 20:04:51,644 - IveltScraper - INFO - Found 50 active topics on ivelt

2026-07-05 20:05:02,077 - IveltScraper - INFO - Filtering post containing exclude term: '*ביטע מאכט זיכער בלויז צו באריכטן קראנטע נייעס*'

2026-07-05 20:05:02,672 - IveltScraper - INFO - ivelt scrape complete. Checked 2 topics.

2026-07-05 20:05:02,673 - IveltScraper - INFO - All forums scrape complete. Found 9 new posts.

2026-07-05 20:05:02,673 - __main__ - INFO - Scraper rss completed: 3 results

2026-07-05 20:05:02,673 - __main__ - INFO - Scraper yiddish24 completed: 9 results

2026-07-05 20:05:02,673 - __main__ - INFO - Scraper ivelt completed: 9 results

2026-07-05 20:05:03,478 - social_media_monitor.email.sender - INFO - Email sent successfully at 2026-07-05 20:05:03.478782

2026-07-05 20:05:03,479 - __main__ - INFO - Email sent successfully

2026-07-05 20:05:03,479 - __main__ - INFO - Cache stats: 0 items in memory, 0 files on disk

2026-07-05 20:05:03,479 - __main__ - INFO - Monitor complete. Found 21 total results: 3 new rss results, 9 new yiddish24 results, 9 new ivelt results.

2026-07-05 20:05:03,479 - __main__ - INFO - === Social Media Monitor v2.0 Complete ===

2026-07-05 20:05:03,479 - social_media_monitor.database.connection - INFO - Closing all database connections

2026-07-05 20:05:03,536 - social_media_monitor.database.connection - INFO - Closed 5 database connections

2026-07-05 20:05:03,537 - __main__ - INFO - Closed all database connections

2026-07-05 20:05:03,538 - social_media_monitor.database.connection - INFO - Closing all database connections

2026-07-05 20:05:03,538 - social_media_monitor.database.connection - INFO - Closed 0 database connections

2026-07-05 20:30:00,623 - social_media_monitor.utils.logging_config - INFO - Posts log file: /mnt/data/openclaw-shared/home/shared/Social_Media_Monitor/logs/social_media_monitor_posts_2026-07-05.log

2026-07-05 20:30:00,623 - social_media_monitor.utils.logging_config - INFO - Downloads log file: /mnt/data/openclaw-shared/home/shared/Social_Media_Monitor/logs/social_media_monitor_downloads_2026-07-05.log

2026-07-05 20:30:00,623 - social_media_monitor.utils.logging_config - INFO - Logging configured with level INFO, log file: /mnt/data/openclaw-shared/home/shared/Social_Media_Monitor/logs/social_media_monitor_2026-07-05.log

2026-07-05 20:30:00,623 - __main__ - INFO - === Social Media Monitor v2.0 (Async) Started ===

2026-07-05 20:30:00,655 - social_media_monitor.config.settings - INFO - Configuration loaded from /home/openclaw/Social_Media_Monitor/config.yaml

2026-07-05 20:30:00,655 - __main__ - INFO - Configuration loaded and validated successfully

2026-07-05 20:30:00,658 - social_media_monitor.database.connection - INFO - Database connection pool initialized with 5 connections

2026-07-05 20:30:00,658 - social_media_monitor.database.connection - INFO - add_speaker_filters column already exists

2026-07-05 20:30:00,671 - social_media_monitor.database.connection - INFO - Database setup completed successfully

2026-07-05 20:30:00,671 - __main__ - INFO - Database initialized at: /mnt/data/openclaw-shared/home/shared/Social_Media_Monitor/social_media_monitor.db

2026-07-05 20:30:00,719 - social_media_monitor.scrapers.reddit_scraper - INFO - Reddit API initialized successfully

2026-07-05 20:30:00,720 - __main__ - INFO - GitHub scraper initialized and enabled

2026-07-05 20:30:00,721 - RSSFeedScraper - INFO - RSS scraper initialized: feeds=2, html_urls=0 (0 with custom selectors), include_terms=0, exclude_terms=0

2026-07-05 20:30:00,721 - __main__ - INFO - LinkedIn Feed scraper initialized and enabled

2026-07-05 20:30:00,726 - Yiddish24Scraper - INFO - File downloader initialized with 55 terms, prefix 'Y2', and file type organization enabled

2026-07-05 20:30:00,726 - Yiddish24Scraper - INFO - Yiddish24 scraper initialized: data_id='latest', page_limit=100, total_pages=10, debug_html=True, include_terms=55, exclude_terms=0

2026-07-05 20:30:00,730 - AhBlickLiveScraper - INFO - AhBlickLive scraper initialized: urls=1, max_items=100, include_terms=49, exclude_terms=0

2026-07-05 20:30:00,737 - TorahAnytimeScraper - INFO - File downloader initialized with 53 terms, prefix 'TA', and file type organization enabled

2026-07-05 20:30:00,737 - TorahAnytimeScraper - INFO - TorahAnytime scraper initialized: limit=400, offset=0, project_id=1, include_terms=53, exclude_terms=10

2026-07-05 20:30:00,741 - BatorahScraper - INFO - File downloader initialized with 19 terms, prefix 'BT', and file type organization enabled

2026-07-05 20:30:00,741 - BatorahScraper - INFO - Batorah scraper initialized: api_urls=2, page_size=100, include_terms=19, exclude_terms=1

2026-07-05 20:30:00,741 - KolHalashonScraper - INFO - Kol Halashon scraper initialized: base_url=https://www2.kolhalashon.com, max_shiurim=50, fetch_all_speakers=True, headless=False, exclude_filters=1, include_filters=0

2026-07-05 20:30:00,741 - __main__ - INFO - Running 2 scrapers concurrently...

2026-07-05 20:30:00,741 - GitHubScraper - INFO - Searching GitHub for terms: ['paperclip', '"Hermes Agent"', 'multica']

2026-07-05 20:30:00,839 - LinkedInFeedScraper - INFO - Fetching LinkedIn feed (25 posts)

2026-07-05 20:30:06,428 - LinkedInFeedScraper - ERROR - LinkedIn feed fetch failed: Unauthorized: LinkedIn session cookies are expired or invalid.

2026-07-05 20:30:06,436 - LinkedInFeedScraper - INFO - LinkedIn Feed scrape complete. Found 0 new posts.

2026-07-05 20:30:07,360 - httpx - INFO - HTTP Request: GET https://api.github.com/search/repositories?q=paperclip+in%3Aname%2Cdescription%2Creadme+stars%3A%3E150&sort=updated&order=desc&per_page=20 "HTTP/1.1 200 OK"

2026-07-05 20:30:07,386 - GitHubScraper - INFO - Found 242 GitHub repositories for term: paperclip

2026-07-05 20:30:08,421 - httpx - INFO - HTTP Request: GET https://api.github.com/search/repositories?q=%22Hermes+Agent%22+in%3Aname%2Cdescription%2Creadme+stars%3A%3E150&sort=updated&order=desc&per_page=20 "HTTP/1.1 200 OK"

2026-07-05 20:30:08,424 - GitHubScraper - INFO - Found 426 GitHub repositories for term: "Hermes Agent"

2026-07-05 20:30:09,308 - httpx - INFO - HTTP Request: GET https://api.github.com/search/repositories?q=multica+in%3Aname%2Cdescription%2Creadme+stars%3A%3E150&sort=updated&order=desc&per_page=20 "HTTP/1.1 200 OK"

2026-07-05 20:30:09,402 - GitHubScraper - INFO - Found 57 GitHub repositories for term: multica

2026-07-05 20:30:09,542 - GitHubScraper - INFO - Total GitHub results found: 0

2026-07-05 20:30:09,542 - __main__ - INFO - Scraper github completed: 0 results

2026-07-05 20:30:09,542 - __main__ - INFO - Scraper linkedin_feed completed: 0 results

2026-07-05 20:30:09,543 - __main__ - WARNING - LinkedIn Feed: 15 consecutive failures (fetch failed)

2026-07-05 20:30:09,543 - __main__ - INFO - No new results found across all enabled sources

2026-07-05 20:30:09,544 - __main__ - INFO - Cache stats: 0 items in memory, 0 files on disk

2026-07-05 20:30:09,544 - __main__ - INFO - Monitor complete. No new results found across all sources.

2026-07-05 20:30:09,544 - __main__ - INFO - === Social Media Monitor v2.0 Complete ===

2026-07-05 20:30:09,544 - social_media_monitor.database.connection - INFO - Closing all database connections

2026-07-05 20:30:09,566 - social_media_monitor.database.connection - INFO - Closed 5 database connections

2026-07-05 20:30:09,566 - __main__ - INFO - Closed all database connections

2026-07-05 20:30:09,568 - social_media_monitor.database.connection - INFO - Closing all database connections

2026-07-05 20:30:09,568 - social_media_monitor.database.connection - INFO - Closed 0 database connections

2026-07-05 21:04:39,294 - social_media_monitor.utils.logging_config - INFO - Posts log file: /mnt/data/openclaw-shared/home/shared/Social_Media_Monitor/logs/social_media_monitor_posts_2026-07-05.log

2026-07-05 21:04:39,295 - social_media_monitor.utils.logging_config - INFO - Downloads log file: /mnt/data/openclaw-shared/home/shared/Social_Media_Monitor/logs/social_media_monitor_downloads_2026-07-05.log

2026-07-05 21:04:39,295 - social_media_monitor.utils.logging_config - INFO - Logging configured with level INFO, log file: /mnt/data/openclaw-shared/home/shared/Social_Media_Monitor/logs/social_media_monitor_2026-07-05.log

2026-07-05 21:04:39,295 - __main__ - INFO - === Social Media Monitor v2.0 (Async) Started ===

2026-07-05 21:04:39,326 - social_media_monitor.config.settings - INFO - Configuration loaded from /home/openclaw/Social_Media_Monitor/config.yaml

2026-07-05 21:04:39,326 - __main__ - INFO - Configuration loaded and validated successfully

2026-07-05 21:04:39,328 - social_media_monitor.database.connection - INFO - Database connection pool initialized with 5 connections

2026-07-05 21:04:39,329 - social_media_monitor.database.connection - INFO - add_speaker_filters column already exists

2026-07-05 21:04:39,346 - social_media_monitor.database.connection - INFO - Database setup completed successfully

2026-07-05 21:04:39,347 - __main__ - INFO - Database initialized at: /mnt/data/openclaw-shared/home/shared/Social_Media_Monitor/social_media_monitor.db

2026-07-05 21:04:39,349 - social_media_monitor.scrapers.reddit_scraper - INFO - Reddit API initialized successfully

2026-07-05 21:04:39,350 - RSSFeedScraper - INFO - RSS scraper initialized: feeds=2, html_urls=0 (0 with custom selectors), include_terms=0, exclude_terms=0

2026-07-05 21:04:39,350 - __main__ - INFO - RSS scraper initialized and enabled

2026-07-05 21:04:39,352 - Yiddish24Scraper - INFO - File downloader initialized with 55 terms, prefix 'Y2', and file type organization enabled

2026-07-05 21:04:39,352 - Yiddish24Scraper - INFO - Yiddish24 scraper initialized: data_id='latest', page_limit=100, total_pages=10, debug_html=True, include_terms=55, exclude_terms=0

2026-07-05 21:04:39,352 - __main__ - INFO - Yiddish24 scraper initialized and enabled

2026-07-05 21:04:39,355 - AhBlickLiveScraper - INFO - AhBlickLive scraper initialized: urls=1, max_items=100, include_terms=49, exclude_terms=0

2026-07-05 21:04:39,359 - TorahAnytimeScraper - INFO - File downloader initialized with 53 terms, prefix 'TA', and file type organization enabled

2026-07-05 21:04:39,359 - TorahAnytimeScraper - INFO - TorahAnytime scraper initialized: limit=400, offset=0, project_id=1, include_terms=53, exclude_terms=10

2026-07-05 21:04:39,362 - BatorahScraper - INFO - File downloader initialized with 19 terms, prefix 'BT', and file type organization enabled

2026-07-05 21:04:39,362 - BatorahScraper - INFO - Batorah scraper initialized: api_urls=2, page_size=100, include_terms=19, exclude_terms=1

2026-07-05 21:04:39,362 - KolHalashonScraper - INFO - Kol Halashon scraper initialized: base_url=https://www2.kolhalashon.com, max_shiurim=50, fetch_all_speakers=True, headless=False, exclude_filters=1, include_filters=0

2026-07-05 21:04:39,362 - __main__ - INFO - Ivelt scraper initialized and enabled

2026-07-05 21:04:39,362 - __main__ - INFO - Running 3 scrapers concurrently...

2026-07-05 21:04:39,362 - Yiddish24Scraper - INFO - Starting Yiddish24 scraping (max 10 pages, limit 100 items)

2026-07-05 21:04:39,452 - Yiddish24Scraper - INFO - Fetching Yiddish24 page 1 with data_id='latest', page_limit=100

2026-07-05 21:04:39,457 - IveltScraper - INFO - Loaded 181 include, 0 exclude filters from DB topic_filters

2026-07-05 21:04:39,457 - IveltScraper - INFO - Fetching ivelt active topics from https://www.ivelt.com/forum/search.php?search_id=active_topics...

2026-07-05 21:04:39,459 - RSSFeedScraper - INFO - Fetching RSS feed: https://www.theyeshivaworld.com/feed

2026-07-05 21:04:39,764 - RSSFeedScraper - INFO - Fetching RSS feed: https://www.thegatewaypundit.com/feed/

2026-07-05 21:04:40,201 - RSSFeedScraper - INFO - Total RSS/HTML results: 3

2026-07-05 21:04:43,494 - httpx - INFO - HTTP Request: POST https://www.yiddish24.com/ajax/cat_pagination.php "HTTP/1.1 200 OK"

2026-07-05 21:04:43,722 - Yiddish24Scraper - INFO - Received response with 438687 characters

2026-07-05 21:04:43,725 - Yiddish24Scraper - INFO - Extracted HTML from JSON result field (291981 chars)

2026-07-05 21:04:43,729 - Yiddish24Scraper - INFO - Saved HTML debug file to: /mnt/data/openclaw-shared/home/shared/Social_Media_Monitor/debug/yiddish24_page1_response.html

2026-07-05 21:04:43,823 - Yiddish24Scraper - INFO - Found 100 potential article elements on page 1

2026-07-05 21:04:43,862 - Yiddish24Scraper - INFO - Page 1 extraction: 100 successful, 0 failed out of 100 elements

2026-07-05 21:04:43,863 - Yiddish24Scraper - INFO - Parsed 100 articles from HTML on page 1

2026-07-05 21:04:43,893 - Yiddish24Scraper - INFO - Term 'מערקווירדיג -' matched in DESCRIPTION: 'מערקווירדיג - יעדער טאג מאכט אויס'

2026-07-05 21:04:43,893 - Yiddish24Scraper - INFO - Downloading 1 media file(s) for article: הכל בכל

2026-07-05 21:04:44,393 - social_media_monitor.utils.file_downloader - INFO - Starting download from: https://cloudfront.yiddish24.com/DROSSMAN_260_260705045016.mp3

2026-07-05 21:04:45,658 - social_media_monitor.utils.file_downloader - INFO - Successfully downloaded: Y2_DROSSMAN_260_260705045016.mp3 (12.5 MB)

2026-07-05 21:04:45,659 - social_media_monitor.utils.file_downloader - INFO - Organized audio file to: /mnt/data/openclaw-shared/home/shared/Social_Media_Monitor/downloads/Audio (audio/mpeg)

2026-07-05 21:04:45,675 - social_media_monitor.utils.media_tagger - INFO - Tagged Y2_DROSSMAN_260_260705045016.mp3: title, artist, album, comment, source_url, genre, publisher

2026-07-05 21:04:45,676 - Yiddish24Scraper - INFO - Renamed downloaded file to include match term: Y2_DROSSMAN_260_260705045016_מערקווירדיג_-.mp3

2026-07-05 21:04:46,379 - Yiddish24Scraper - INFO - Page 1 summary: 6 new out of 100 total articles

2026-07-05 21:04:46,380 - Yiddish24Scraper - INFO - Page 1: 6/100 new items

2026-07-05 21:04:46,380 - Yiddish24Scraper - INFO - Low new item ratio (6.0%), stopping pagination

2026-07-05 21:04:46,380 - Yiddish24Scraper - INFO - Total Yiddish24 results found: 6

2026-07-05 21:04:51,533 - IveltScraper - INFO - Found 50 active topics on ivelt

2026-07-05 21:05:09,714 - IveltScraper - INFO - Filtering post containing exclude term: '*ביטע מאכט זיכער בלויז צו באריכטן קראנטע נייעס*'

2026-07-05 21:05:15,463 - IveltScraper - INFO - ivelt scrape complete. Checked 6 topics.

2026-07-05 21:05:15,463 - IveltScraper - INFO - All forums scrape complete. Found 13 new posts.

2026-07-05 21:05:15,463 - __main__ - INFO - Scraper rss completed: 3 results

2026-07-05 21:05:15,463 - __main__ - INFO - Scraper yiddish24 completed: 6 results

2026-07-05 21:05:15,463 - __main__ - INFO - Scraper ivelt completed: 13 results

2026-07-05 21:05:16,459 - social_media_monitor.email.sender - INFO - Email sent successfully at 2026-07-05 21:05:16.459045

2026-07-05 21:05:16,459 - __main__ - INFO - Email sent successfully

2026-07-05 21:05:16,459 - __main__ - INFO - Cache stats: 0 items in memory, 0 files on disk

2026-07-05 21:05:16,459 - __main__ - INFO - Monitor complete. Found 22 total results: 3 new rss results, 6 new yiddish24 results, 13 new ivelt results.

2026-07-05 21:05:16,460 - __main__ - INFO - === Social Media Monitor v2.0 Complete ===

2026-07-05 21:05:16,460 - social_media_monitor.database.connection - INFO - Closing all database connections

2026-07-05 21:05:16,622 - social_media_monitor.database.connection - INFO - Closed 5 database connections

2026-07-05 21:05:16,623 - __main__ - INFO - Closed all database connections

2026-07-05 21:05:16,625 - social_media_monitor.database.connection - INFO - Closing all database connections

2026-07-05 21:05:16,625 - social_media_monitor.database.connection - INFO - Closed 0 database connections

2026-07-05 21:30:00,666 - social_media_monitor.utils.logging_config - INFO - Posts log file: /mnt/data/openclaw-shared/home/shared/Social_Media_Monitor/logs/social_media_monitor_posts_2026-07-05.log

2026-07-05 21:30:00,666 - social_media_monitor.utils.logging_config - INFO - Downloads log file: /mnt/data/openclaw-shared/home/shared/Social_Media_Monitor/logs/social_media_monitor_downloads_2026-07-05.log

2026-07-05 21:30:00,666 - social_media_monitor.utils.logging_config - INFO - Logging configured with level INFO, log file: /mnt/data/openclaw-shared/home/shared/Social_Media_Monitor/logs/social_media_monitor_2026-07-05.log

2026-07-05 21:30:00,667 - __main__ - INFO - === Social Media Monitor v2.0 (Async) Started ===

2026-07-05 21:30:00,698 - social_media_monitor.config.settings - INFO - Configuration loaded from /home/openclaw/Social_Media_Monitor/config.yaml

2026-07-05 21:30:00,698 - __main__ - INFO - Configuration loaded and validated successfully

2026-07-05 21:30:00,701 - social_media_monitor.database.connection - INFO - Database connection pool initialized with 5 connections

2026-07-05 21:30:00,701 - social_media_monitor.database.connection - INFO - add_speaker_filters column already exists

2026-07-05 21:30:00,717 - social_media_monitor.database.connection - INFO - Database setup completed successfully

2026-07-05 21:30:00,717 - __main__ - INFO - Database initialized at: /mnt/data/openclaw-shared/home/shared/Social_Media_Monitor/social_media_monitor.db

2026-07-05 21:30:00,763 - social_media_monitor.scrapers.reddit_scraper - INFO - Reddit API initialized successfully

2026-07-05 21:30:00,764 - __main__ - INFO - GitHub scraper initialized and enabled

2026-07-05 21:30:00,764 - RSSFeedScraper - INFO - RSS scraper initialized: feeds=2, html_urls=0 (0 with custom selectors), include_terms=0, exclude_terms=0

2026-07-05 21:30:00,764 - __main__ - INFO - LinkedIn Feed scraper initialized and enabled

2026-07-05 21:30:00,768 - Yiddish24Scraper - INFO - File downloader initialized with 55 terms, prefix 'Y2', and file type organization enabled

2026-07-05 21:30:00,768 - Yiddish24Scraper - INFO - Yiddish24 scraper initialized: data_id='latest', page_limit=100, total_pages=10, debug_html=True, include_terms=55, exclude_terms=0

2026-07-05 21:30:00,770 - AhBlickLiveScraper - INFO - AhBlickLive scraper initialized: urls=1, max_items=100, include_terms=49, exclude_terms=0

2026-07-05 21:30:00,774 - TorahAnytimeScraper - INFO - File downloader initialized with 53 terms, prefix 'TA', and file type organization enabled

2026-07-05 21:30:00,775 - TorahAnytimeScraper - INFO - TorahAnytime scraper initialized: limit=400, offset=0, project_id=1, include_terms=53, exclude_terms=10

2026-07-05 21:30:00,777 - BatorahScraper - INFO - File downloader initialized with 19 terms, prefix 'BT', and file type organization enabled

2026-07-05 21:30:00,777 - BatorahScraper - INFO - Batorah scraper initialized: api_urls=2, page_size=100, include_terms=19, exclude_terms=1

2026-07-05 21:30:00,777 - KolHalashonScraper - INFO - Kol Halashon scraper initialized: base_url=https://www2.kolhalashon.com, max_shiurim=50, fetch_all_speakers=True, headless=False, exclude_filters=1, include_filters=0

2026-07-05 21:30:00,777 - __main__ - INFO - Running 2 scrapers concurrently...

2026-07-05 21:30:00,777 - GitHubScraper - INFO - Searching GitHub for terms: ['paperclip', '"Hermes Agent"', 'multica']

2026-07-05 21:30:00,871 - LinkedInFeedScraper - INFO - Fetching LinkedIn feed (25 posts)

2026-07-05 21:30:06,521 - LinkedInFeedScraper - ERROR - LinkedIn feed fetch failed: Unauthorized: LinkedIn session cookies are expired or invalid.

2026-07-05 21:30:06,533 - LinkedInFeedScraper - INFO - LinkedIn Feed scrape complete. Found 0 new posts.

2026-07-05 21:30:07,440 - httpx - INFO - HTTP Request: GET https://api.github.com/search/repositories?q=paperclip+in%3Aname%2Cdescription%2Creadme+stars%3A%3E150&sort=updated&order=desc&per_page=20 "HTTP/1.1 200 OK"

2026-07-05 21:30:07,465 - GitHubScraper - INFO - Found 242 GitHub repositories for term: paperclip

2026-07-05 21:30:08,512 - httpx - INFO - HTTP Request: GET https://api.github.com/search/repositories?q=%22Hermes+Agent%22+in%3Aname%2Cdescription%2Creadme+stars%3A%3E150&sort=updated&order=desc&per_page=20 "HTTP/1.1 200 OK"

2026-07-05 21:30:08,516 - GitHubScraper - INFO - Found 426 GitHub repositories for term: "Hermes Agent"

2026-07-05 21:30:09,406 - httpx - INFO - HTTP Request: GET https://api.github.com/search/repositories?q=multica+in%3Aname%2Cdescription%2Creadme+stars%3A%3E150&sort=updated&order=desc&per_page=20 "HTTP/1.1 200 OK"

2026-07-05 21:30:09,505 - GitHubScraper - INFO - Found 57 GitHub repositories for term: multica

2026-07-05 21:30:09,680 - GitHubScraper - INFO - Total GitHub results found: 0

2026-07-05 21:30:09,680 - __main__ - INFO - Scraper github completed: 0 results

2026-07-05 21:30:09,681 - __main__ - INFO - Scraper linkedin_feed completed: 0 results

2026-07-05 21:30:09,681 - __main__ - WARNING - LinkedIn Feed: 16 consecutive failures (fetch failed)

2026-07-05 21:30:09,681 - __main__ - INFO - No new results found across all enabled sources

2026-07-05 21:30:09,682 - __main__ - INFO - Cache stats: 0 items in memory, 0 files on disk

2026-07-05 21:30:09,682 - __main__ - INFO - Monitor complete. No new results found across all sources.

2026-07-05 21:30:09,682 - __main__ - INFO - === Social Media Monitor v2.0 Complete ===

2026-07-05 21:30:09,682 - social_media_monitor.database.connection - INFO - Closing all database connections

2026-07-05 21:30:09,701 - social_media_monitor.database.connection - INFO - Closed 5 database connections

2026-07-05 21:30:09,701 - __main__ - INFO - Closed all database connections

2026-07-05 21:30:09,703 - social_media_monitor.database.connection - INFO - Closing all database connections

2026-07-05 21:30:09,703 - social_media_monitor.database.connection - INFO - Closed 0 database connections

2026-07-05 22:04:39,292 - social_media_monitor.utils.logging_config - INFO - Posts log file: /mnt/data/openclaw-shared/home/shared/Social_Media_Monitor/logs/social_media_monitor_posts_2026-07-05.log

2026-07-05 22:04:39,293 - social_media_monitor.utils.logging_config - INFO - Downloads log file: /mnt/data/openclaw-shared/home/shared/Social_Media_Monitor/logs/social_media_monitor_downloads_2026-07-05.log

2026-07-05 22:04:39,293 - social_media_monitor.utils.logging_config - INFO - Logging configured with level INFO, log file: /mnt/data/openclaw-shared/home/shared/Social_Media_Monitor/logs/social_media_monitor_2026-07-05.log

2026-07-05 22:04:39,293 - __main__ - INFO - === Social Media Monitor v2.0 (Async) Started ===

2026-07-05 22:04:39,324 - social_media_monitor.config.settings - INFO - Configuration loaded from /home/openclaw/Social_Media_Monitor/config.yaml

2026-07-05 22:04:39,324 - __main__ - INFO - Configuration loaded and validated successfully

2026-07-05 22:04:39,326 - social_media_monitor.database.connection - INFO - Database connection pool initialized with 5 connections

2026-07-05 22:04:39,326 - social_media_monitor.database.connection - INFO - add_speaker_filters column already exists

2026-07-05 22:04:39,345 - social_media_monitor.database.connection - INFO - Database setup completed successfully

2026-07-05 22:04:39,345 - __main__ - INFO - Database initialized at: /mnt/data/openclaw-shared/home/shared/Social_Media_Monitor/social_media_monitor.db

2026-07-05 22:04:39,348 - social_media_monitor.scrapers.reddit_scraper - INFO - Reddit API initialized successfully

2026-07-05 22:04:39,348 - RSSFeedScraper - INFO - RSS scraper initialized: feeds=2, html_urls=0 (0 with custom selectors), include_terms=0, exclude_terms=0

2026-07-05 22:04:39,348 - __main__ - INFO - RSS scraper initialized and enabled

2026-07-05 22:04:39,351 - Yiddish24Scraper - INFO - File downloader initialized with 55 terms, prefix 'Y2', and file type organization enabled

2026-07-05 22:04:39,351 - Yiddish24Scraper - INFO - Yiddish24 scraper initialized: data_id='latest', page_limit=100, total_pages=10, debug_html=True, include_terms=55, exclude_terms=0

2026-07-05 22:04:39,351 - __main__ - INFO - Yiddish24 scraper initialized and enabled

2026-07-05 22:04:39,354 - AhBlickLiveScraper - INFO - AhBlickLive scraper initialized: urls=1, max_items=100, include_terms=49, exclude_terms=0

2026-07-05 22:04:39,358 - TorahAnytimeScraper - INFO - File downloader initialized with 53 terms, prefix 'TA', and file type organization enabled

2026-07-05 22:04:39,358 - TorahAnytimeScraper - INFO - TorahAnytime scraper initialized: limit=400, offset=0, project_id=1, include_terms=53, exclude_terms=10

2026-07-05 22:04:39,361 - BatorahScraper - INFO - File downloader initialized with 19 terms, prefix 'BT', and file type organization enabled

2026-07-05 22:04:39,361 - BatorahScraper - INFO - Batorah scraper initialized: api_urls=2, page_size=100, include_terms=19, exclude_terms=1

2026-07-05 22:04:39,361 - KolHalashonScraper - INFO - Kol Halashon scraper initialized: base_url=https://www2.kolhalashon.com, max_shiurim=50, fetch_all_speakers=True, headless=False, exclude_filters=1, include_filters=0

2026-07-05 22:04:39,361 - __main__ - INFO - Ivelt scraper initialized and enabled

2026-07-05 22:04:39,361 - __main__ - INFO - Running 3 scrapers concurrently...

2026-07-05 22:04:39,361 - Yiddish24Scraper - INFO - Starting Yiddish24 scraping (max 10 pages, limit 100 items)

2026-07-05 22:04:39,455 - Yiddish24Scraper - INFO - Fetching Yiddish24 page 1 with data_id='latest', page_limit=100

2026-07-05 22:04:39,460 - IveltScraper - INFO - Loaded 181 include, 0 exclude filters from DB topic_filters

2026-07-05 22:04:39,461 - IveltScraper - INFO - Fetching ivelt active topics from https://www.ivelt.com/forum/search.php?search_id=active_topics...

2026-07-05 22:04:39,462 - RSSFeedScraper - INFO - Fetching RSS feed: https://www.theyeshivaworld.com/feed

2026-07-05 22:04:39,665 - RSSFeedScraper - INFO - Fetching RSS feed: https://www.thegatewaypundit.com/feed/

2026-07-05 22:04:39,791 - RSSFeedScraper - WARNING - Feed parsing issues: https://www.thegatewaypundit.com/feed/

2026-07-05 22:04:39,791 - RSSFeedScraper - INFO - Total RSS/HTML results: 0

2026-07-05 22:04:43,271 - httpx - INFO - HTTP Request: POST https://www.yiddish24.com/ajax/cat_pagination.php "HTTP/1.1 200 OK"

2026-07-05 22:04:43,475 - Yiddish24Scraper - INFO - Received response with 450722 characters

2026-07-05 22:04:43,479 - Yiddish24Scraper - INFO - Extracted HTML from JSON result field (296031 chars)

2026-07-05 22:04:43,483 - Yiddish24Scraper - INFO - Saved HTML debug file to: /mnt/data/openclaw-shared/home/shared/Social_Media_Monitor/debug/yiddish24_page1_response.html

2026-07-05 22:04:43,582 - Yiddish24Scraper - INFO - Found 100 potential article elements on page 1

2026-07-05 22:04:43,623 - Yiddish24Scraper - INFO - Page 1 extraction: 100 successful, 0 failed out of 100 elements

2026-07-05 22:04:43,623 - Yiddish24Scraper - INFO - Parsed 100 articles from HTML on page 1

2026-07-05 22:04:44,402 - Yiddish24Scraper - INFO - Page 1 summary: 10 new out of 100 total articles

2026-07-05 22:04:44,403 - Yiddish24Scraper - INFO - Page 1: 10/100 new items

2026-07-05 22:04:44,403 - Yiddish24Scraper - INFO - Low new item ratio (10.0%), stopping pagination

2026-07-05 22:04:44,403 - Yiddish24Scraper - INFO - Total Yiddish24 results found: 10

2026-07-05 22:04:51,500 - IveltScraper - INFO - Found 50 active topics on ivelt

2026-07-05 22:04:54,789 - IveltScraper - INFO - Filtering post containing exclude term: '*ביטע מאכט זיכער בלויז צו באריכטן קראנטע נייעס*'

2026-07-05 22:04:59,011 - IveltScraper - INFO - ivelt scrape complete. Checked 1 topics.

2026-07-05 22:04:59,011 - IveltScraper - INFO - All forums scrape complete. Found 3 new posts.

2026-07-05 22:04:59,011 - __main__ - INFO - Scraper rss completed: 0 results

2026-07-05 22:04:59,012 - __main__ - INFO - Scraper yiddish24 completed: 10 results

2026-07-05 22:04:59,012 - __main__ - INFO - Scraper ivelt completed: 3 results

2026-07-05 22:04:59,786 - social_media_monitor.email.sender - INFO - Email sent successfully at 2026-07-05 22:04:59.786221

2026-07-05 22:04:59,786 - __main__ - INFO - Email sent successfully

2026-07-05 22:04:59,786 - __main__ - INFO - Cache stats: 0 items in memory, 0 files on disk

2026-07-05 22:04:59,786 - __main__ - INFO - Monitor complete. Found 13 total results: 10 new yiddish24 results, 3 new ivelt results.

2026-07-05 22:04:59,787 - __main__ - INFO - === Social Media Monitor v2.0 Complete ===

2026-07-05 22:04:59,787 - social_media_monitor.database.connection - INFO - Closing all database connections

2026-07-05 22:04:59,817 - social_media_monitor.database.connection - INFO - Closed 5 database connections

2026-07-05 22:04:59,817 - __main__ - INFO - Closed all database connections

2026-07-05 22:04:59,818 - social_media_monitor.database.connection - INFO - Closing all database connections

2026-07-05 22:04:59,819 - social_media_monitor.database.connection - INFO - Closed 0 database connections

2026-07-05 22:30:00,622 - social_media_monitor.utils.logging_config - INFO - Posts log file: /mnt/data/openclaw-shared/home/shared/Social_Media_Monitor/logs/social_media_monitor_posts_2026-07-05.log

2026-07-05 22:30:00,622 - social_media_monitor.utils.logging_config - INFO - Downloads log file: /mnt/data/openclaw-shared/home/shared/Social_Media_Monitor/logs/social_media_monitor_downloads_2026-07-05.log

2026-07-05 22:30:00,622 - social_media_monitor.utils.logging_config - INFO - Logging configured with level INFO, log file: /mnt/data/openclaw-shared/home/shared/Social_Media_Monitor/logs/social_media_monitor_2026-07-05.log

2026-07-05 22:30:00,622 - __main__ - INFO - === Social Media Monitor v2.0 (Async) Started ===

2026-07-05 22:30:00,656 - social_media_monitor.config.settings - INFO - Configuration loaded from /home/openclaw/Social_Media_Monitor/config.yaml

2026-07-05 22:30:00,656 - __main__ - INFO - Configuration loaded and validated successfully

2026-07-05 22:30:00,658 - social_media_monitor.database.connection - INFO - Database connection pool initialized with 5 connections

2026-07-05 22:30:00,659 - social_media_monitor.database.connection - INFO - add_speaker_filters column already exists

2026-07-05 22:30:00,673 - social_media_monitor.database.connection - INFO - Database setup completed successfully

2026-07-05 22:30:00,673 - __main__ - INFO - Database initialized at: /mnt/data/openclaw-shared/home/shared/Social_Media_Monitor/social_media_monitor.db

2026-07-05 22:30:00,676 - social_media_monitor.scrapers.reddit_scraper - INFO - Reddit API initialized successfully

2026-07-05 22:30:00,676 - __main__ - INFO - GitHub scraper initialized and enabled

2026-07-05 22:30:00,676 - RSSFeedScraper - INFO - RSS scraper initialized: feeds=2, html_urls=0 (0 with custom selectors), include_terms=0, exclude_terms=0

2026-07-05 22:30:00,677 - __main__ - INFO - LinkedIn Feed scraper initialized and enabled

2026-07-05 22:30:00,679 - Yiddish24Scraper - INFO - File downloader initialized with 55 terms, prefix 'Y2', and file type organization enabled

2026-07-05 22:30:00,679 - Yiddish24Scraper - INFO - Yiddish24 scraper initialized: data_id='latest', page_limit=100, total_pages=10, debug_html=True, include_terms=55, exclude_terms=0

2026-07-05 22:30:00,682 - AhBlickLiveScraper - INFO - AhBlickLive scraper initialized: urls=1, max_items=100, include_terms=49, exclude_terms=0

2026-07-05 22:30:00,686 - TorahAnytimeScraper - INFO - File downloader initialized with 53 terms, prefix 'TA', and file type organization enabled

2026-07-05 22:30:00,686 - TorahAnytimeScraper - INFO - TorahAnytime scraper initialized: limit=400, offset=0, project_id=1, include_terms=53, exclude_terms=10

2026-07-05 22:30:00,688 - BatorahScraper - INFO - File downloader initialized with 19 terms, prefix 'BT', and file type organization enabled

2026-07-05 22:30:00,688 - BatorahScraper - INFO - Batorah scraper initialized: api_urls=2, page_size=100, include_terms=19, exclude_terms=1

2026-07-05 22:30:00,688 - KolHalashonScraper - INFO - Kol Halashon scraper initialized: base_url=https://www2.kolhalashon.com, max_shiurim=50, fetch_all_speakers=True, headless=False, exclude_filters=1, include_filters=0

2026-07-05 22:30:00,688 - __main__ - INFO - Running 2 scrapers concurrently...

2026-07-05 22:30:00,689 - GitHubScraper - INFO - Searching GitHub for terms: ['paperclip', '"Hermes Agent"', 'multica']

2026-07-05 22:30:00,779 - LinkedInFeedScraper - INFO - Fetching LinkedIn feed (25 posts)

2026-07-05 22:30:06,434 - LinkedInFeedScraper - ERROR - LinkedIn feed fetch failed: Unauthorized: LinkedIn session cookies are expired or invalid.

2026-07-05 22:30:06,443 - LinkedInFeedScraper - INFO - LinkedIn Feed scrape complete. Found 0 new posts.

2026-07-05 22:30:07,400 - httpx - INFO - HTTP Request: GET https://api.github.com/search/repositories?q=paperclip+in%3Aname%2Cdescription%2Creadme+stars%3A%3E150&sort=updated&order=desc&per_page=20 "HTTP/1.1 200 OK"

2026-07-05 22:30:07,488 - GitHubScraper - INFO - Found 242 GitHub repositories for term: paperclip

2026-07-05 22:30:08,543 - httpx - INFO - HTTP Request: GET https://api.github.com/search/repositories?q=%22Hermes+Agent%22+in%3Aname%2Cdescription%2Creadme+stars%3A%3E150&sort=updated&order=desc&per_page=20 "HTTP/1.1 200 OK"

2026-07-05 22:30:08,635 - GitHubScraper - INFO - Found 426 GitHub repositories for term: "Hermes Agent"

2026-07-05 22:30:09,505 - httpx - INFO - HTTP Request: GET https://api.github.com/search/repositories?q=multica+in%3Aname%2Cdescription%2Creadme+stars%3A%3E150&sort=updated&order=desc&per_page=20 "HTTP/1.1 200 OK"

2026-07-05 22:30:09,507 - GitHubScraper - INFO - Found 57 GitHub repositories for term: multica

2026-07-05 22:30:09,651 - GitHubScraper - INFO - Total GitHub results found: 0

2026-07-05 22:30:09,651 - __main__ - INFO - Scraper github completed: 0 results

2026-07-05 22:30:09,651 - __main__ - INFO - Scraper linkedin_feed completed: 0 results

2026-07-05 22:30:09,652 - __main__ - WARNING - LinkedIn Feed: 17 consecutive failures (fetch failed)

2026-07-05 22:30:09,652 - __main__ - INFO - No new results found across all enabled sources

2026-07-05 22:30:09,653 - __main__ - INFO - Cache stats: 0 items in memory, 0 files on disk

2026-07-05 22:30:09,653 - __main__ - INFO - Monitor complete. No new results found across all sources.

2026-07-05 22:30:09,653 - __main__ - INFO - === Social Media Monitor v2.0 Complete ===

2026-07-05 22:30:09,653 - social_media_monitor.database.connection - INFO - Closing all database connections

2026-07-05 22:30:09,669 - social_media_monitor.database.connection - INFO - Closed 5 database connections

2026-07-05 22:30:09,669 - __main__ - INFO - Closed all database connections

2026-07-05 22:30:09,671 - social_media_monitor.database.connection - INFO - Closing all database connections

2026-07-05 22:30:09,671 - social_media_monitor.database.connection - INFO - Closed 0 database connections

2026-07-05 23:04:39,296 - social_media_monitor.utils.logging_config - INFO - Posts log file: /mnt/data/openclaw-shared/home/shared/Social_Media_Monitor/logs/social_media_monitor_posts_2026-07-05.log

2026-07-05 23:04:39,297 - social_media_monitor.utils.logging_config - INFO - Downloads log file: /mnt/data/openclaw-shared/home/shared/Social_Media_Monitor/logs/social_media_monitor_downloads_2026-07-05.log

2026-07-05 23:04:39,297 - social_media_monitor.utils.logging_config - INFO - Logging configured with level INFO, log file: /mnt/data/openclaw-shared/home/shared/Social_Media_Monitor/logs/social_media_monitor_2026-07-05.log

2026-07-05 23:04:39,297 - __main__ - INFO - === Social Media Monitor v2.0 (Async) Started ===

2026-07-05 23:04:39,327 - social_media_monitor.config.settings - INFO - Configuration loaded from /home/openclaw/Social_Media_Monitor/config.yaml

2026-07-05 23:04:39,327 - __main__ - INFO - Configuration loaded and validated successfully

2026-07-05 23:04:39,329 - social_media_monitor.database.connection - INFO - Database connection pool initialized with 5 connections

2026-07-05 23:04:39,329 - social_media_monitor.database.connection - INFO - add_speaker_filters column already exists

2026-07-05 23:04:39,347 - social_media_monitor.database.connection - INFO - Database setup completed successfully

2026-07-05 23:04:39,347 - __main__ - INFO - Database initialized at: /mnt/data/openclaw-shared/home/shared/Social_Media_Monitor/social_media_monitor.db

2026-07-05 23:04:39,394 - social_media_monitor.scrapers.reddit_scraper - INFO - Reddit API initialized successfully

2026-07-05 23:04:39,396 - RSSFeedScraper - INFO - RSS scraper initialized: feeds=2, html_urls=0 (0 with custom selectors), include_terms=0, exclude_terms=0

2026-07-05 23:04:39,396 - __main__ - INFO - RSS scraper initialized and enabled

2026-07-05 23:04:39,404 - Yiddish24Scraper - INFO - File downloader initialized with 55 terms, prefix 'Y2', and file type organization enabled

2026-07-05 23:04:39,404 - Yiddish24Scraper - INFO - Yiddish24 scraper initialized: data_id='latest', page_limit=100, total_pages=10, debug_html=True, include_terms=55, exclude_terms=0

2026-07-05 23:04:39,404 - __main__ - INFO - Yiddish24 scraper initialized and enabled

2026-07-05 23:04:39,409 - AhBlickLiveScraper - INFO - AhBlickLive scraper initialized: urls=1, max_items=100, include_terms=49, exclude_terms=0

2026-07-05 23:04:39,417 - TorahAnytimeScraper - INFO - File downloader initialized with 53 terms, prefix 'TA', and file type organization enabled

2026-07-05 23:04:39,417 - TorahAnytimeScraper - INFO - TorahAnytime scraper initialized: limit=400, offset=0, project_id=1, include_terms=53, exclude_terms=10

2026-07-05 23:04:39,422 - BatorahScraper - INFO - File downloader initialized with 19 terms, prefix 'BT', and file type organization enabled

2026-07-05 23:04:39,422 - BatorahScraper - INFO - Batorah scraper initialized: api_urls=2, page_size=100, include_terms=19, exclude_terms=1

2026-07-05 23:04:39,422 - KolHalashonScraper - INFO - Kol Halashon scraper initialized: base_url=https://www2.kolhalashon.com, max_shiurim=50, fetch_all_speakers=True, headless=False, exclude_filters=1, include_filters=0

2026-07-05 23:04:39,422 - __main__ - INFO - Ivelt scraper initialized and enabled

2026-07-05 23:04:39,422 - __main__ - INFO - Running 3 scrapers concurrently...

2026-07-05 23:04:39,423 - Yiddish24Scraper - INFO - Starting Yiddish24 scraping (max 10 pages, limit 100 items)

2026-07-05 23:04:39,509 - Yiddish24Scraper - INFO - Fetching Yiddish24 page 1 with data_id='latest', page_limit=100

2026-07-05 23:04:39,513 - IveltScraper - INFO - Loaded 181 include, 0 exclude filters from DB topic_filters

2026-07-05 23:04:39,514 - IveltScraper - INFO - Fetching ivelt active topics from https://www.ivelt.com/forum/search.php?search_id=active_topics...

2026-07-05 23:04:39,516 - RSSFeedScraper - INFO - Fetching RSS feed: https://www.theyeshivaworld.com/feed

2026-07-05 23:04:39,747 - RSSFeedScraper - INFO - Fetching RSS feed: https://www.thegatewaypundit.com/feed/

2026-07-05 23:04:39,815 - RSSFeedScraper - WARNING - Feed parsing issues: https://www.thegatewaypundit.com/feed/

2026-07-05 23:04:39,817 - RSSFeedScraper - INFO - Total RSS/HTML results: 0

2026-07-05 23:04:43,064 - httpx - INFO - HTTP Request: POST https://www.yiddish24.com/ajax/cat_pagination.php "HTTP/1.1 200 OK"

2026-07-05 23:04:43,277 - Yiddish24Scraper - INFO - Received response with 440320 characters

2026-07-05 23:04:43,279 - Yiddish24Scraper - INFO - Extracted HTML from JSON result field (292015 chars)

2026-07-05 23:04:43,281 - Yiddish24Scraper - INFO - Saved HTML debug file to: /mnt/data/openclaw-shared/home/shared/Social_Media_Monitor/debug/yiddish24_page1_response.html

2026-07-05 23:04:43,372 - Yiddish24Scraper - INFO - Found 100 potential article elements on page 1

2026-07-05 23:04:43,411 - Yiddish24Scraper - INFO - Page 1 extraction: 100 successful, 0 failed out of 100 elements

2026-07-05 23:04:43,411 - Yiddish24Scraper - INFO - Parsed 100 articles from HTML on page 1

2026-07-05 23:04:44,376 - Yiddish24Scraper - INFO - Page 1 summary: 6 new out of 100 total articles

2026-07-05 23:04:44,377 - Yiddish24Scraper - INFO - Page 1: 6/100 new items

2026-07-05 23:04:44,377 - Yiddish24Scraper - INFO - Low new item ratio (6.0%), stopping pagination

2026-07-05 23:04:44,378 - Yiddish24Scraper - INFO - Total Yiddish24 results found: 6

2026-07-05 23:04:51,655 - IveltScraper - INFO - Found 50 active topics on ivelt

2026-07-05 23:05:15,735 - IveltScraper - INFO - ivelt scrape complete. Checked 3 topics.

2026-07-05 23:05:15,735 - IveltScraper - INFO - All forums scrape complete. Found 13 new posts.

2026-07-05 23:05:15,736 - __main__ - INFO - Scraper rss completed: 0 results

2026-07-05 23:05:15,736 - __main__ - INFO - Scraper yiddish24 completed: 6 results

2026-07-05 23:05:15,736 - __main__ - INFO - Scraper ivelt completed: 13 results

2026-07-05 23:05:16,671 - social_media_monitor.email.sender - INFO - Email sent successfully at 2026-07-05 23:05:16.671907

2026-07-05 23:05:16,672 - __main__ - INFO - Email sent successfully

2026-07-05 23:05:16,672 - __main__ - INFO - Cache stats: 0 items in memory, 0 files on disk

2026-07-05 23:05:16,672 - __main__ - INFO - Monitor complete. Found 19 total results: 6 new yiddish24 results, 13 new ivelt results.

2026-07-05 23:05:16,672 - __main__ - INFO - === Social Media Monitor v2.0 Complete ===

2026-07-05 23:05:16,672 - social_media_monitor.database.connection - INFO - Closing all database connections

2026-07-05 23:05:16,764 - social_media_monitor.database.connection - INFO - Closed 5 database connections

2026-07-05 23:05:16,764 - __main__ - INFO - Closed all database connections

2026-07-05 23:05:16,765 - social_media_monitor.database.connection - INFO - Closing all database connections

2026-07-05 23:05:16,766 - social_media_monitor.database.connection - INFO - Closed 0 database connections

2026-07-05 23:30:00,639 - social_media_monitor.utils.logging_config - INFO - Posts log file: /mnt/data/openclaw-shared/home/shared/Social_Media_Monitor/logs/social_media_monitor_posts_2026-07-05.log

2026-07-05 23:30:00,640 - social_media_monitor.utils.logging_config - INFO - Downloads log file: /mnt/data/openclaw-shared/home/shared/Social_Media_Monitor/logs/social_media_monitor_downloads_2026-07-05.log

2026-07-05 23:30:00,640 - social_media_monitor.utils.logging_config - INFO - Logging configured with level INFO, log file: /mnt/data/openclaw-shared/home/shared/Social_Media_Monitor/logs/social_media_monitor_2026-07-05.log

2026-07-05 23:30:00,640 - __main__ - INFO - === Social Media Monitor v2.0 (Async) Started ===

2026-07-05 23:30:00,670 - social_media_monitor.config.settings - INFO - Configuration loaded from /home/openclaw/Social_Media_Monitor/config.yaml

2026-07-05 23:30:00,670 - __main__ - INFO - Configuration loaded and validated successfully

2026-07-05 23:30:00,672 - social_media_monitor.database.connection - INFO - Database connection pool initialized with 5 connections

2026-07-05 23:30:00,673 - social_media_monitor.database.connection - INFO - add_speaker_filters column already exists

2026-07-05 23:30:00,698 - social_media_monitor.database.connection - INFO - Database setup completed successfully

2026-07-05 23:30:00,699 - __main__ - INFO - Database initialized at: /mnt/data/openclaw-shared/home/shared/Social_Media_Monitor/social_media_monitor.db

2026-07-05 23:30:00,701 - social_media_monitor.scrapers.reddit_scraper - INFO - Reddit API initialized successfully

2026-07-05 23:30:00,701 - __main__ - INFO - GitHub scraper initialized and enabled

2026-07-05 23:30:00,702 - RSSFeedScraper - INFO - RSS scraper initialized: feeds=2, html_urls=0 (0 with custom selectors), include_terms=0, exclude_terms=0

2026-07-05 23:30:00,702 - __main__ - INFO - LinkedIn Feed scraper initialized and enabled

2026-07-05 23:30:00,704 - Yiddish24Scraper - INFO - File downloader initialized with 55 terms, prefix 'Y2', and file type organization enabled

2026-07-05 23:30:00,704 - Yiddish24Scraper - INFO - Yiddish24 scraper initialized: data_id='latest', page_limit=100, total_pages=10, debug_html=True, include_terms=55, exclude_terms=0

2026-07-05 23:30:00,707 - AhBlickLiveScraper - INFO - AhBlickLive scraper initialized: urls=1, max_items=100, include_terms=49, exclude_terms=0

2026-07-05 23:30:00,712 - TorahAnytimeScraper - INFO - File downloader initialized with 53 terms, prefix 'TA', and file type organization enabled

2026-07-05 23:30:00,712 - TorahAnytimeScraper - INFO - TorahAnytime scraper initialized: limit=400, offset=0, project_id=1, include_terms=53, exclude_terms=10

2026-07-05 23:30:00,714 - BatorahScraper - INFO - File downloader initialized with 19 terms, prefix 'BT', and file type organization enabled

2026-07-05 23:30:00,714 - BatorahScraper - INFO - Batorah scraper initialized: api_urls=2, page_size=100, include_terms=19, exclude_terms=1

2026-07-05 23:30:00,714 - KolHalashonScraper - INFO - Kol Halashon scraper initialized: base_url=https://www2.kolhalashon.com, max_shiurim=50, fetch_all_speakers=True, headless=False, exclude_filters=1, include_filters=0

2026-07-05 23:30:00,714 - __main__ - INFO - Running 2 scrapers concurrently...

2026-07-05 23:30:00,714 - GitHubScraper - INFO - Searching GitHub for terms: ['paperclip', '"Hermes Agent"', 'multica']

2026-07-05 23:30:00,803 - LinkedInFeedScraper - INFO - Fetching LinkedIn feed (25 posts)

2026-07-05 23:30:06,446 - LinkedInFeedScraper - ERROR - LinkedIn feed fetch failed: Unauthorized: LinkedIn session cookies are expired or invalid.

2026-07-05 23:30:06,456 - LinkedInFeedScraper - INFO - LinkedIn Feed scrape complete. Found 0 new posts.

2026-07-05 23:30:07,532 - httpx - INFO - HTTP Request: GET https://api.github.com/search/repositories?q=paperclip+in%3Aname%2Cdescription%2Creadme+stars%3A%3E150&sort=updated&order=desc&per_page=20 "HTTP/1.1 200 OK"

2026-07-05 23:30:07,557 - GitHubScraper - INFO - Found 242 GitHub repositories for term: paperclip

2026-07-05 23:30:08,592 - httpx - INFO - HTTP Request: GET https://api.github.com/search/repositories?q=%22Hermes+Agent%22+in%3Aname%2Cdescription%2Creadme+stars%3A%3E150&sort=updated&order=desc&per_page=20 "HTTP/1.1 200 OK"

2026-07-05 23:30:08,596 - GitHubScraper - INFO - Found 426 GitHub repositories for term: "Hermes Agent"

2026-07-05 23:30:09,517 - httpx - INFO - HTTP Request: GET https://api.github.com/search/repositories?q=multica+in%3Aname%2Cdescription%2Creadme+stars%3A%3E150&sort=updated&order=desc&per_page=20 "HTTP/1.1 200 OK"

2026-07-05 23:30:09,519 - GitHubScraper - INFO - Found 57 GitHub repositories for term: multica

2026-07-05 23:30:09,643 - GitHubScraper - INFO - Total GitHub results found: 0

2026-07-05 23:30:09,643 - __main__ - INFO - Scraper github completed: 0 results

2026-07-05 23:30:09,643 - __main__ - INFO - Scraper linkedin_feed completed: 0 results

2026-07-05 23:30:09,643 - __main__ - WARNING - LinkedIn Feed: 18 consecutive failures (fetch failed)

2026-07-05 23:30:09,644 - __main__ - INFO - No new results found across all enabled sources

2026-07-05 23:30:09,644 - __main__ - INFO - Cache stats: 0 items in memory, 0 files on disk

2026-07-05 23:30:09,644 - __main__ - INFO - Monitor complete. No new results found across all sources.

2026-07-05 23:30:09,644 - __main__ - INFO - === Social Media Monitor v2.0 Complete ===

2026-07-05 23:30:09,644 - social_media_monitor.database.connection - INFO - Closing all database connections

2026-07-05 23:30:09,663 - social_media_monitor.database.connection - INFO - Closed 5 database connections

2026-07-05 23:30:09,663 - __main__ - INFO - Closed all database connections

2026-07-05 23:30:09,664 - social_media_monitor.database.connection - INFO - Closing all database connections

2026-07-05 23:30:09,664 - social_media_monitor.database.connection - INFO - Closed 0 database connections