← Logs

social_media_monitor_2026-07-06.log

2026-07-06 19:04:43,630 - Yiddish24Scraper - INFO - Page 1 extraction: 100 successful, 0 failed out of 100 elements

2026-07-06 19:04:43,630 - Yiddish24Scraper - INFO - Parsed 100 articles from HTML on page 1

2026-07-06 19:04:44,345 - Yiddish24Scraper - INFO - Page 1 summary: 2 new out of 100 total articles

2026-07-06 19:04:44,345 - Yiddish24Scraper - INFO - Page 1: 2/100 new items

2026-07-06 19:04:44,345 - Yiddish24Scraper - INFO - Low new item ratio (2.0%), stopping pagination

2026-07-06 19:04:44,346 - Yiddish24Scraper - INFO - Total Yiddish24 results found: 2

2026-07-06 19:04:52,817 - IveltScraper - INFO - Found 50 active topics on ivelt

2026-07-06 19:04:56,799 - IveltScraper - INFO - ivelt scrape complete. Checked 1 topics.

2026-07-06 19:04:56,799 - IveltScraper - INFO - All forums scrape complete. Found 9 new posts.

2026-07-06 19:04:56,799 - __main__ - INFO - Scraper rss completed: 0 results

2026-07-06 19:04:56,799 - __main__ - INFO - Scraper yiddish24 completed: 2 results

2026-07-06 19:04:56,799 - __main__ - INFO - Scraper ivelt completed: 9 results

2026-07-06 19:04:57,539 - social_media_monitor.email.sender - INFO - Email sent successfully at 2026-07-06 19:04:57.539596

2026-07-06 19:04:57,539 - __main__ - INFO - Email sent successfully

2026-07-06 19:04:57,540 - __main__ - INFO - Cache stats: 0 items in memory, 0 files on disk

2026-07-06 19:04:57,540 - __main__ - INFO - Monitor complete. Found 11 total results: 2 new yiddish24 results, 9 new ivelt results.

2026-07-06 19:04:57,540 - __main__ - INFO - === Social Media Monitor v2.0 Complete ===

2026-07-06 19:04:57,540 - social_media_monitor.database.connection - INFO - Closing all database connections

2026-07-06 19:04:57,590 - social_media_monitor.database.connection - INFO - Closed 5 database connections

2026-07-06 19:04:57,590 - __main__ - INFO - Closed all database connections

2026-07-06 19:04:57,592 - social_media_monitor.database.connection - INFO - Closing all database connections

2026-07-06 19:04:57,592 - social_media_monitor.database.connection - INFO - Closed 0 database connections

2026-07-06 19:30:00,652 - social_media_monitor.utils.logging_config - INFO - Posts log file: /mnt/data/openclaw-shared/home/shared/Social_Media_Monitor/logs/social_media_monitor_posts_2026-07-06.log

2026-07-06 19:30:00,653 - social_media_monitor.utils.logging_config - INFO - Downloads log file: /mnt/data/openclaw-shared/home/shared/Social_Media_Monitor/logs/social_media_monitor_downloads_2026-07-06.log

2026-07-06 19:30:00,653 - social_media_monitor.utils.logging_config - INFO - Logging configured with level INFO, log file: /mnt/data/openclaw-shared/home/shared/Social_Media_Monitor/logs/social_media_monitor_2026-07-06.log

2026-07-06 19:30:00,653 - __main__ - INFO - === Social Media Monitor v2.0 (Async) Started ===

2026-07-06 19:30:00,685 - social_media_monitor.config.settings - INFO - Configuration loaded from /home/openclaw/Social_Media_Monitor/config.yaml

2026-07-06 19:30:00,685 - __main__ - INFO - Configuration loaded and validated successfully

2026-07-06 19:30:00,687 - social_media_monitor.database.connection - INFO - Database connection pool initialized with 5 connections

2026-07-06 19:30:00,688 - social_media_monitor.database.connection - INFO - add_speaker_filters column already exists

2026-07-06 19:30:00,707 - social_media_monitor.database.connection - INFO - Database setup completed successfully

2026-07-06 19:30:00,707 - __main__ - INFO - Database initialized at: /mnt/data/openclaw-shared/home/shared/Social_Media_Monitor/social_media_monitor.db

2026-07-06 19:30:00,709 - social_media_monitor.scrapers.reddit_scraper - INFO - Reddit API initialized successfully

2026-07-06 19:30:00,710 - __main__ - INFO - GitHub scraper initialized and enabled

2026-07-06 19:30:00,710 - RSSFeedScraper - INFO - RSS scraper initialized: feeds=2, html_urls=0 (0 with custom selectors), include_terms=0, exclude_terms=0

2026-07-06 19:30:00,710 - __main__ - INFO - LinkedIn Feed scraper initialized and enabled

2026-07-06 19:30:00,713 - Yiddish24Scraper - INFO - File downloader initialized with 55 terms, prefix 'Y2', and file type organization enabled

2026-07-06 19:30:00,713 - Yiddish24Scraper - INFO - Yiddish24 scraper initialized: data_id='latest', page_limit=100, total_pages=10, debug_html=True, include_terms=55, exclude_terms=0

2026-07-06 19:30:00,716 - AhBlickLiveScraper - INFO - AhBlickLive scraper initialized: urls=1, max_items=100, include_terms=49, exclude_terms=0

2026-07-06 19:30:00,720 - TorahAnytimeScraper - INFO - File downloader initialized with 53 terms, prefix 'TA', and file type organization enabled

2026-07-06 19:30:00,721 - TorahAnytimeScraper - INFO - TorahAnytime scraper initialized: limit=400, offset=0, project_id=1, include_terms=53, exclude_terms=10

2026-07-06 19:30:00,723 - BatorahScraper - INFO - File downloader initialized with 19 terms, prefix 'BT', and file type organization enabled

2026-07-06 19:30:00,723 - BatorahScraper - INFO - Batorah scraper initialized: api_urls=2, page_size=100, include_terms=19, exclude_terms=1

2026-07-06 19:30:00,723 - KolHalashonScraper - INFO - Kol Halashon scraper initialized: base_url=https://www2.kolhalashon.com, max_shiurim=50, fetch_all_speakers=True, headless=False, exclude_filters=1, include_filters=0

2026-07-06 19:30:00,723 - __main__ - INFO - Running 2 scrapers concurrently...

2026-07-06 19:30:00,723 - GitHubScraper - INFO - Searching GitHub for terms: ['paperclip', '"Hermes Agent"', 'multica']

2026-07-06 19:30:00,816 - LinkedInFeedScraper - INFO - Fetching LinkedIn feed (25 posts)

2026-07-06 19:30:03,518 - LinkedInFeedScraper - ERROR - LinkedIn feed fetch failed: Unauthorized: LinkedIn session cookies are expired or invalid.

2026-07-06 19:30:03,525 - LinkedInFeedScraper - INFO - LinkedIn Feed scrape complete. Found 0 new posts.

2026-07-06 19:30:04,534 - httpx - INFO - HTTP Request: GET https://api.github.com/search/repositories?q=paperclip+in%3Aname%2Cdescription%2Creadme+stars%3A%3E150&sort=updated&order=desc&per_page=20 "HTTP/1.1 200 OK"

2026-07-06 19:30:04,623 - GitHubScraper - INFO - Found 242 GitHub repositories for term: paperclip

2026-07-06 19:30:05,657 - httpx - INFO - HTTP Request: GET https://api.github.com/search/repositories?q=%22Hermes+Agent%22+in%3Aname%2Cdescription%2Creadme+stars%3A%3E150&sort=updated&order=desc&per_page=20 "HTTP/1.1 200 OK"

2026-07-06 19:30:05,661 - GitHubScraper - INFO - Found 427 GitHub repositories for term: "Hermes Agent"

2026-07-06 19:30:06,677 - httpx - INFO - HTTP Request: GET https://api.github.com/search/repositories?q=multica+in%3Aname%2Cdescription%2Creadme+stars%3A%3E150&sort=updated&order=desc&per_page=20 "HTTP/1.1 200 OK"

2026-07-06 19:30:06,679 - GitHubScraper - INFO - Found 56 GitHub repositories for term: multica

2026-07-06 19:30:06,801 - GitHubScraper - INFO - Total GitHub results found: 0

2026-07-06 19:30:06,802 - __main__ - INFO - Scraper github completed: 0 results

2026-07-06 19:30:06,802 - __main__ - INFO - Scraper linkedin_feed completed: 0 results

2026-07-06 19:30:06,802 - __main__ - WARNING - LinkedIn Feed: 38 consecutive failures (fetch failed)

2026-07-06 19:30:06,803 - __main__ - INFO - No new results found across all enabled sources

2026-07-06 19:30:06,803 - __main__ - INFO - Cache stats: 0 items in memory, 0 files on disk

2026-07-06 19:30:06,803 - __main__ - INFO - Monitor complete. No new results found across all sources.

2026-07-06 19:30:06,803 - __main__ - INFO - === Social Media Monitor v2.0 Complete ===

2026-07-06 19:30:06,803 - social_media_monitor.database.connection - INFO - Closing all database connections

2026-07-06 19:30:06,821 - social_media_monitor.database.connection - INFO - Closed 5 database connections

2026-07-06 19:30:06,821 - __main__ - INFO - Closed all database connections

2026-07-06 19:30:06,822 - social_media_monitor.database.connection - INFO - Closing all database connections

2026-07-06 19:30:06,822 - social_media_monitor.database.connection - INFO - Closed 0 database connections

2026-07-06 20:04:39,246 - social_media_monitor.utils.logging_config - INFO - Posts log file: /mnt/data/openclaw-shared/home/shared/Social_Media_Monitor/logs/social_media_monitor_posts_2026-07-06.log

2026-07-06 20:04:39,246 - social_media_monitor.utils.logging_config - INFO - Downloads log file: /mnt/data/openclaw-shared/home/shared/Social_Media_Monitor/logs/social_media_monitor_downloads_2026-07-06.log

2026-07-06 20:04:39,246 - social_media_monitor.utils.logging_config - INFO - Logging configured with level INFO, log file: /mnt/data/openclaw-shared/home/shared/Social_Media_Monitor/logs/social_media_monitor_2026-07-06.log

2026-07-06 20:04:39,246 - __main__ - INFO - === Social Media Monitor v2.0 (Async) Started ===

2026-07-06 20:04:39,276 - social_media_monitor.config.settings - INFO - Configuration loaded from /home/openclaw/Social_Media_Monitor/config.yaml

2026-07-06 20:04:39,276 - __main__ - INFO - Configuration loaded and validated successfully

2026-07-06 20:04:39,278 - social_media_monitor.database.connection - INFO - Database connection pool initialized with 5 connections

2026-07-06 20:04:39,279 - social_media_monitor.database.connection - INFO - add_speaker_filters column already exists

2026-07-06 20:04:39,295 - social_media_monitor.database.connection - INFO - Database setup completed successfully

2026-07-06 20:04:39,295 - __main__ - INFO - Database initialized at: /mnt/data/openclaw-shared/home/shared/Social_Media_Monitor/social_media_monitor.db

2026-07-06 20:04:39,297 - social_media_monitor.scrapers.reddit_scraper - INFO - Reddit API initialized successfully

2026-07-06 20:04:39,298 - RSSFeedScraper - INFO - RSS scraper initialized: feeds=2, html_urls=0 (0 with custom selectors), include_terms=0, exclude_terms=0

2026-07-06 20:04:39,298 - __main__ - INFO - RSS scraper initialized and enabled

2026-07-06 20:04:39,300 - Yiddish24Scraper - INFO - File downloader initialized with 55 terms, prefix 'Y2', and file type organization enabled

2026-07-06 20:04:39,301 - Yiddish24Scraper - INFO - Yiddish24 scraper initialized: data_id='latest', page_limit=100, total_pages=10, debug_html=True, include_terms=55, exclude_terms=0

2026-07-06 20:04:39,301 - __main__ - INFO - Yiddish24 scraper initialized and enabled

2026-07-06 20:04:39,303 - AhBlickLiveScraper - INFO - AhBlickLive scraper initialized: urls=1, max_items=100, include_terms=49, exclude_terms=0

2026-07-06 20:04:39,308 - TorahAnytimeScraper - INFO - File downloader initialized with 53 terms, prefix 'TA', and file type organization enabled

2026-07-06 20:04:39,308 - TorahAnytimeScraper - INFO - TorahAnytime scraper initialized: limit=400, offset=0, project_id=1, include_terms=53, exclude_terms=10

2026-07-06 20:04:39,310 - BatorahScraper - INFO - File downloader initialized with 19 terms, prefix 'BT', and file type organization enabled

2026-07-06 20:04:39,310 - BatorahScraper - INFO - Batorah scraper initialized: api_urls=2, page_size=100, include_terms=19, exclude_terms=1

2026-07-06 20:04:39,310 - KolHalashonScraper - INFO - Kol Halashon scraper initialized: base_url=https://www2.kolhalashon.com, max_shiurim=50, fetch_all_speakers=True, headless=False, exclude_filters=1, include_filters=0

2026-07-06 20:04:39,310 - __main__ - INFO - Ivelt scraper initialized and enabled

2026-07-06 20:04:39,310 - __main__ - INFO - Running 3 scrapers concurrently...

2026-07-06 20:04:39,310 - Yiddish24Scraper - INFO - Starting Yiddish24 scraping (max 10 pages, limit 100 items)

2026-07-06 20:04:39,398 - Yiddish24Scraper - INFO - Fetching Yiddish24 page 1 with data_id='latest', page_limit=100

2026-07-06 20:04:39,402 - IveltScraper - INFO - Loaded 181 include, 0 exclude filters from DB topic_filters

2026-07-06 20:04:39,402 - IveltScraper - INFO - Fetching ivelt active topics from https://www.ivelt.com/forum/search.php?search_id=active_topics...

2026-07-06 20:04:39,403 - RSSFeedScraper - INFO - Fetching RSS feed: https://www.theyeshivaworld.com/feed

2026-07-06 20:04:39,634 - RSSFeedScraper - INFO - Fetching RSS feed: https://www.thegatewaypundit.com/feed/

2026-07-06 20:04:40,134 - RSSFeedScraper - INFO - Total RSS/HTML results: 16

2026-07-06 20:04:43,348 - httpx - INFO - HTTP Request: POST https://www.yiddish24.com/ajax/cat_pagination.php "HTTP/1.1 200 OK"

2026-07-06 20:04:43,354 - Yiddish24Scraper - INFO - Received response with 489184 characters

2026-07-06 20:04:43,358 - Yiddish24Scraper - INFO - Extracted HTML from JSON result field (303011 chars)

2026-07-06 20:04:43,361 - Yiddish24Scraper - INFO - Saved HTML debug file to: /mnt/data/openclaw-shared/home/shared/Social_Media_Monitor/debug/yiddish24_page1_response.html

2026-07-06 20:04:43,444 - Yiddish24Scraper - INFO - Found 100 potential article elements on page 1

2026-07-06 20:04:43,483 - Yiddish24Scraper - INFO - Page 1 extraction: 100 successful, 0 failed out of 100 elements

2026-07-06 20:04:43,484 - Yiddish24Scraper - INFO - Parsed 100 articles from HTML on page 1

2026-07-06 20:04:43,494 - Yiddish24Scraper - INFO - Term 'ענינים שונים' matched in DESCRIPTION: 'ענינים שונים - לעבן אויף קידוש השם'

2026-07-06 20:04:43,494 - Yiddish24Scraper - INFO - Downloading 1 media file(s) for article: הכל בכל

2026-07-06 20:04:44,004 - social_media_monitor.utils.file_downloader - INFO - Starting download from: https://cloudfront.yiddish24.com/________________________________260_260706035328.mp3

2026-07-06 20:04:45,497 - social_media_monitor.utils.file_downloader - INFO - Successfully downloaded: Y2_________________________________260_260706035328.mp3 (19.5 MB)

2026-07-06 20:04:45,498 - social_media_monitor.utils.file_downloader - INFO - Organized audio file to: /mnt/data/openclaw-shared/home/shared/Social_Media_Monitor/downloads/Audio (audio/mpeg)

2026-07-06 20:04:45,518 - social_media_monitor.utils.media_tagger - INFO - Tagged Y2_________________________________260_260706035328.mp3: title, artist, album, comment, source_url, genre, publisher

2026-07-06 20:04:45,518 - Yiddish24Scraper - INFO - Renamed downloaded file to include match term: Y2_________________________________260_260706035328_ענינים_שונים.mp3

2026-07-06 20:04:46,257 - Yiddish24Scraper - INFO - Page 1 summary: 12 new out of 100 total articles

2026-07-06 20:04:46,257 - Yiddish24Scraper - INFO - Page 1: 12/100 new items

2026-07-06 20:04:46,257 - Yiddish24Scraper - INFO - Low new item ratio (12.0%), stopping pagination

2026-07-06 20:04:46,258 - Yiddish24Scraper - INFO - Total Yiddish24 results found: 12

2026-07-06 20:04:51,569 - IveltScraper - INFO - Found 50 active topics on ivelt

2026-07-06 20:05:31,208 - IveltScraper - INFO - ivelt scrape complete. Checked 4 topics.

2026-07-06 20:05:31,208 - IveltScraper - INFO - All forums scrape complete. Found 21 new posts.

2026-07-06 20:05:31,208 - __main__ - INFO - Scraper rss completed: 16 results

2026-07-06 20:05:31,208 - __main__ - INFO - Scraper yiddish24 completed: 12 results

2026-07-06 20:05:31,208 - __main__ - INFO - Scraper ivelt completed: 21 results

2026-07-06 20:05:32,076 - social_media_monitor.email.sender - INFO - Email sent successfully at 2026-07-06 20:05:32.076787

2026-07-06 20:05:32,077 - __main__ - INFO - Email sent successfully

2026-07-06 20:05:32,077 - __main__ - INFO - Cache stats: 0 items in memory, 0 files on disk

2026-07-06 20:05:32,077 - __main__ - INFO - Monitor complete. Found 49 total results: 16 new rss results, 12 new yiddish24 results, 21 new ivelt results.

2026-07-06 20:05:32,077 - __main__ - INFO - === Social Media Monitor v2.0 Complete ===

2026-07-06 20:05:32,077 - social_media_monitor.database.connection - INFO - Closing all database connections

2026-07-06 20:05:32,297 - social_media_monitor.database.connection - INFO - Closed 5 database connections

2026-07-06 20:05:32,298 - __main__ - INFO - Closed all database connections

2026-07-06 20:05:32,299 - social_media_monitor.database.connection - INFO - Closing all database connections

2026-07-06 20:05:32,300 - social_media_monitor.database.connection - INFO - Closed 0 database connections

2026-07-06 20:30:00,640 - social_media_monitor.utils.logging_config - INFO - Posts log file: /mnt/data/openclaw-shared/home/shared/Social_Media_Monitor/logs/social_media_monitor_posts_2026-07-06.log

2026-07-06 20:30:00,641 - social_media_monitor.utils.logging_config - INFO - Downloads log file: /mnt/data/openclaw-shared/home/shared/Social_Media_Monitor/logs/social_media_monitor_downloads_2026-07-06.log

2026-07-06 20:30:00,641 - social_media_monitor.utils.logging_config - INFO - Logging configured with level INFO, log file: /mnt/data/openclaw-shared/home/shared/Social_Media_Monitor/logs/social_media_monitor_2026-07-06.log

2026-07-06 20:30:00,641 - __main__ - INFO - === Social Media Monitor v2.0 (Async) Started ===

2026-07-06 20:30:00,671 - social_media_monitor.config.settings - INFO - Configuration loaded from /home/openclaw/Social_Media_Monitor/config.yaml

2026-07-06 20:30:00,671 - __main__ - INFO - Configuration loaded and validated successfully

2026-07-06 20:30:00,673 - social_media_monitor.database.connection - INFO - Database connection pool initialized with 5 connections

2026-07-06 20:30:00,674 - social_media_monitor.database.connection - INFO - add_speaker_filters column already exists

2026-07-06 20:30:00,688 - social_media_monitor.database.connection - INFO - Database setup completed successfully

2026-07-06 20:30:00,688 - __main__ - INFO - Database initialized at: /mnt/data/openclaw-shared/home/shared/Social_Media_Monitor/social_media_monitor.db

2026-07-06 20:30:00,738 - social_media_monitor.scrapers.reddit_scraper - INFO - Reddit API initialized successfully

2026-07-06 20:30:00,738 - __main__ - INFO - GitHub scraper initialized and enabled

2026-07-06 20:30:00,739 - RSSFeedScraper - INFO - RSS scraper initialized: feeds=2, html_urls=0 (0 with custom selectors), include_terms=0, exclude_terms=0

2026-07-06 20:30:00,739 - __main__ - INFO - LinkedIn Feed scraper initialized and enabled

2026-07-06 20:30:00,743 - Yiddish24Scraper - INFO - File downloader initialized with 55 terms, prefix 'Y2', and file type organization enabled

2026-07-06 20:30:00,743 - Yiddish24Scraper - INFO - Yiddish24 scraper initialized: data_id='latest', page_limit=100, total_pages=10, debug_html=True, include_terms=55, exclude_terms=0

2026-07-06 20:30:00,746 - AhBlickLiveScraper - INFO - AhBlickLive scraper initialized: urls=1, max_items=100, include_terms=49, exclude_terms=0

2026-07-06 20:30:00,750 - TorahAnytimeScraper - INFO - File downloader initialized with 53 terms, prefix 'TA', and file type organization enabled

2026-07-06 20:30:00,750 - TorahAnytimeScraper - INFO - TorahAnytime scraper initialized: limit=400, offset=0, project_id=1, include_terms=53, exclude_terms=10

2026-07-06 20:30:00,753 - BatorahScraper - INFO - File downloader initialized with 19 terms, prefix 'BT', and file type organization enabled

2026-07-06 20:30:00,753 - BatorahScraper - INFO - Batorah scraper initialized: api_urls=2, page_size=100, include_terms=19, exclude_terms=1

2026-07-06 20:30:00,753 - KolHalashonScraper - INFO - Kol Halashon scraper initialized: base_url=https://www2.kolhalashon.com, max_shiurim=50, fetch_all_speakers=True, headless=False, exclude_filters=1, include_filters=0

2026-07-06 20:30:00,753 - __main__ - INFO - Running 2 scrapers concurrently...

2026-07-06 20:30:00,753 - GitHubScraper - INFO - Searching GitHub for terms: ['paperclip', '"Hermes Agent"', 'multica']

2026-07-06 20:30:00,842 - LinkedInFeedScraper - INFO - Fetching LinkedIn feed (25 posts)

2026-07-06 20:30:05,484 - LinkedInFeedScraper - ERROR - LinkedIn feed fetch failed: Unauthorized: LinkedIn session cookies are expired or invalid.

2026-07-06 20:30:05,491 - LinkedInFeedScraper - INFO - LinkedIn Feed scrape complete. Found 0 new posts.

2026-07-06 20:30:06,688 - httpx - INFO - HTTP Request: GET https://api.github.com/search/repositories?q=paperclip+in%3Aname%2Cdescription%2Creadme+stars%3A%3E150&sort=updated&order=desc&per_page=20 "HTTP/1.1 200 OK"

2026-07-06 20:30:06,714 - GitHubScraper - INFO - Found 242 GitHub repositories for term: paperclip

2026-07-06 20:30:07,928 - httpx - INFO - HTTP Request: GET https://api.github.com/search/repositories?q=%22Hermes+Agent%22+in%3Aname%2Cdescription%2Creadme+stars%3A%3E150&sort=updated&order=desc&per_page=20 "HTTP/1.1 200 OK"

2026-07-06 20:30:07,930 - GitHubScraper - INFO - Found 427 GitHub repositories for term: "Hermes Agent"

2026-07-06 20:30:09,138 - httpx - INFO - HTTP Request: GET https://api.github.com/search/repositories?q=multica+in%3Aname%2Cdescription%2Creadme+stars%3A%3E150&sort=updated&order=desc&per_page=20 "HTTP/1.1 200 OK"

2026-07-06 20:30:09,143 - GitHubScraper - INFO - Found 56 GitHub repositories for term: multica

2026-07-06 20:30:09,266 - GitHubScraper - INFO - Total GitHub results found: 0

2026-07-06 20:30:09,266 - __main__ - INFO - Scraper github completed: 0 results

2026-07-06 20:30:09,267 - __main__ - INFO - Scraper linkedin_feed completed: 0 results

2026-07-06 20:30:09,267 - __main__ - WARNING - LinkedIn Feed: 39 consecutive failures (fetch failed)

2026-07-06 20:30:09,267 - __main__ - INFO - No new results found across all enabled sources

2026-07-06 20:30:09,268 - __main__ - INFO - Cache stats: 0 items in memory, 0 files on disk

2026-07-06 20:30:09,268 - __main__ - INFO - Monitor complete. No new results found across all sources.

2026-07-06 20:30:09,268 - __main__ - INFO - === Social Media Monitor v2.0 Complete ===

2026-07-06 20:30:09,268 - social_media_monitor.database.connection - INFO - Closing all database connections

2026-07-06 20:30:09,283 - social_media_monitor.database.connection - INFO - Closed 5 database connections

2026-07-06 20:30:09,283 - __main__ - INFO - Closed all database connections

2026-07-06 20:30:09,285 - social_media_monitor.database.connection - INFO - Closing all database connections

2026-07-06 20:30:09,285 - social_media_monitor.database.connection - INFO - Closed 0 database connections

2026-07-06 21:04:39,266 - social_media_monitor.utils.logging_config - INFO - Posts log file: /mnt/data/openclaw-shared/home/shared/Social_Media_Monitor/logs/social_media_monitor_posts_2026-07-06.log

2026-07-06 21:04:39,266 - social_media_monitor.utils.logging_config - INFO - Downloads log file: /mnt/data/openclaw-shared/home/shared/Social_Media_Monitor/logs/social_media_monitor_downloads_2026-07-06.log

2026-07-06 21:04:39,266 - social_media_monitor.utils.logging_config - INFO - Logging configured with level INFO, log file: /mnt/data/openclaw-shared/home/shared/Social_Media_Monitor/logs/social_media_monitor_2026-07-06.log

2026-07-06 21:04:39,266 - __main__ - INFO - === Social Media Monitor v2.0 (Async) Started ===

2026-07-06 21:04:39,296 - social_media_monitor.config.settings - INFO - Configuration loaded from /home/openclaw/Social_Media_Monitor/config.yaml

2026-07-06 21:04:39,296 - __main__ - INFO - Configuration loaded and validated successfully

2026-07-06 21:04:39,298 - social_media_monitor.database.connection - INFO - Database connection pool initialized with 5 connections

2026-07-06 21:04:39,299 - social_media_monitor.database.connection - INFO - add_speaker_filters column already exists

2026-07-06 21:04:39,312 - social_media_monitor.database.connection - INFO - Database setup completed successfully

2026-07-06 21:04:39,312 - __main__ - INFO - Database initialized at: /mnt/data/openclaw-shared/home/shared/Social_Media_Monitor/social_media_monitor.db

2026-07-06 21:04:39,314 - social_media_monitor.scrapers.reddit_scraper - INFO - Reddit API initialized successfully

2026-07-06 21:04:39,315 - RSSFeedScraper - INFO - RSS scraper initialized: feeds=2, html_urls=0 (0 with custom selectors), include_terms=0, exclude_terms=0

2026-07-06 21:04:39,315 - __main__ - INFO - RSS scraper initialized and enabled

2026-07-06 21:04:39,317 - Yiddish24Scraper - INFO - File downloader initialized with 55 terms, prefix 'Y2', and file type organization enabled

2026-07-06 21:04:39,317 - Yiddish24Scraper - INFO - Yiddish24 scraper initialized: data_id='latest', page_limit=100, total_pages=10, debug_html=True, include_terms=55, exclude_terms=0

2026-07-06 21:04:39,317 - __main__ - INFO - Yiddish24 scraper initialized and enabled

2026-07-06 21:04:39,320 - AhBlickLiveScraper - INFO - AhBlickLive scraper initialized: urls=1, max_items=100, include_terms=49, exclude_terms=0

2026-07-06 21:04:39,324 - TorahAnytimeScraper - INFO - File downloader initialized with 53 terms, prefix 'TA', and file type organization enabled

2026-07-06 21:04:39,324 - TorahAnytimeScraper - INFO - TorahAnytime scraper initialized: limit=400, offset=0, project_id=1, include_terms=53, exclude_terms=10

2026-07-06 21:04:39,326 - BatorahScraper - INFO - File downloader initialized with 19 terms, prefix 'BT', and file type organization enabled

2026-07-06 21:04:39,327 - BatorahScraper - INFO - Batorah scraper initialized: api_urls=2, page_size=100, include_terms=19, exclude_terms=1

2026-07-06 21:04:39,327 - KolHalashonScraper - INFO - Kol Halashon scraper initialized: base_url=https://www2.kolhalashon.com, max_shiurim=50, fetch_all_speakers=True, headless=False, exclude_filters=1, include_filters=0

2026-07-06 21:04:39,327 - __main__ - INFO - Ivelt scraper initialized and enabled

2026-07-06 21:04:39,327 - __main__ - INFO - Running 3 scrapers concurrently...

2026-07-06 21:04:39,327 - Yiddish24Scraper - INFO - Starting Yiddish24 scraping (max 10 pages, limit 100 items)

2026-07-06 21:04:39,413 - Yiddish24Scraper - INFO - Fetching Yiddish24 page 1 with data_id='latest', page_limit=100

2026-07-06 21:04:39,417 - IveltScraper - INFO - Loaded 181 include, 0 exclude filters from DB topic_filters

2026-07-06 21:04:39,417 - IveltScraper - INFO - Fetching ivelt active topics from https://www.ivelt.com/forum/search.php?search_id=active_topics...

2026-07-06 21:04:39,418 - RSSFeedScraper - INFO - Fetching RSS feed: https://www.theyeshivaworld.com/feed

2026-07-06 21:04:39,656 - RSSFeedScraper - INFO - Fetching RSS feed: https://www.thegatewaypundit.com/feed/

2026-07-06 21:04:40,212 - RSSFeedScraper - INFO - Total RSS/HTML results: 3

2026-07-06 21:04:51,784 - IveltScraper - INFO - Found 50 active topics on ivelt

2026-07-06 21:04:54,035 - httpx - INFO - HTTP Request: POST https://www.yiddish24.com/ajax/cat_pagination.php "HTTP/1.1 200 OK"

2026-07-06 21:04:54,252 - Yiddish24Scraper - INFO - Received response with 457510 characters

2026-07-06 21:04:54,256 - Yiddish24Scraper - INFO - Extracted HTML from JSON result field (293774 chars)

2026-07-06 21:04:54,259 - Yiddish24Scraper - INFO - Saved HTML debug file to: /mnt/data/openclaw-shared/home/shared/Social_Media_Monitor/debug/yiddish24_page1_response.html

2026-07-06 21:04:54,346 - Yiddish24Scraper - INFO - Found 100 potential article elements on page 1

2026-07-06 21:04:54,394 - Yiddish24Scraper - INFO - Page 1 extraction: 100 successful, 0 failed out of 100 elements

2026-07-06 21:04:54,394 - Yiddish24Scraper - INFO - Parsed 100 articles from HTML on page 1

2026-07-06 21:04:55,119 - Yiddish24Scraper - INFO - Page 1 summary: 6 new out of 100 total articles

2026-07-06 21:04:55,120 - Yiddish24Scraper - INFO - Page 1: 6/100 new items

2026-07-06 21:04:55,120 - Yiddish24Scraper - INFO - Low new item ratio (6.0%), stopping pagination

2026-07-06 21:04:55,120 - Yiddish24Scraper - INFO - Total Yiddish24 results found: 6

2026-07-06 21:05:07,804 - IveltScraper - INFO - Filtering post containing exclude term: '*ביטע מאכט זיכער בלויז צו באריכטן קראנטע נייעס*'

2026-07-06 21:05:33,773 - IveltScraper - INFO - ivelt scrape complete. Checked 6 topics.

2026-07-06 21:05:33,773 - IveltScraper - INFO - All forums scrape complete. Found 29 new posts.

2026-07-06 21:05:33,773 - __main__ - INFO - Scraper rss completed: 3 results

2026-07-06 21:05:33,774 - __main__ - INFO - Scraper yiddish24 completed: 6 results

2026-07-06 21:05:33,774 - __main__ - INFO - Scraper ivelt completed: 29 results

2026-07-06 21:05:34,633 - social_media_monitor.email.sender - INFO - Email sent successfully at 2026-07-06 21:05:34.633023

2026-07-06 21:05:34,633 - __main__ - INFO - Email sent successfully

2026-07-06 21:05:34,633 - __main__ - INFO - Cache stats: 0 items in memory, 0 files on disk

2026-07-06 21:05:34,633 - __main__ - INFO - Monitor complete. Found 38 total results: 3 new rss results, 6 new yiddish24 results, 29 new ivelt results.

2026-07-06 21:05:34,633 - __main__ - INFO - === Social Media Monitor v2.0 Complete ===

2026-07-06 21:05:34,633 - social_media_monitor.database.connection - INFO - Closing all database connections

2026-07-06 21:05:34,701 - social_media_monitor.database.connection - INFO - Closed 5 database connections

2026-07-06 21:05:34,702 - __main__ - INFO - Closed all database connections

2026-07-06 21:05:34,703 - social_media_monitor.database.connection - INFO - Closing all database connections

2026-07-06 21:05:34,703 - social_media_monitor.database.connection - INFO - Closed 0 database connections

2026-07-06 21:30:00,632 - social_media_monitor.utils.logging_config - INFO - Posts log file: /mnt/data/openclaw-shared/home/shared/Social_Media_Monitor/logs/social_media_monitor_posts_2026-07-06.log

2026-07-06 21:30:00,632 - social_media_monitor.utils.logging_config - INFO - Downloads log file: /mnt/data/openclaw-shared/home/shared/Social_Media_Monitor/logs/social_media_monitor_downloads_2026-07-06.log

2026-07-06 21:30:00,632 - social_media_monitor.utils.logging_config - INFO - Logging configured with level INFO, log file: /mnt/data/openclaw-shared/home/shared/Social_Media_Monitor/logs/social_media_monitor_2026-07-06.log

2026-07-06 21:30:00,632 - __main__ - INFO - === Social Media Monitor v2.0 (Async) Started ===

2026-07-06 21:30:00,662 - social_media_monitor.config.settings - INFO - Configuration loaded from /home/openclaw/Social_Media_Monitor/config.yaml

2026-07-06 21:30:00,662 - __main__ - INFO - Configuration loaded and validated successfully

2026-07-06 21:30:00,664 - social_media_monitor.database.connection - INFO - Database connection pool initialized with 5 connections

2026-07-06 21:30:00,665 - social_media_monitor.database.connection - INFO - add_speaker_filters column already exists

2026-07-06 21:30:00,679 - social_media_monitor.database.connection - INFO - Database setup completed successfully

2026-07-06 21:30:00,679 - __main__ - INFO - Database initialized at: /mnt/data/openclaw-shared/home/shared/Social_Media_Monitor/social_media_monitor.db

2026-07-06 21:30:00,681 - social_media_monitor.scrapers.reddit_scraper - INFO - Reddit API initialized successfully

2026-07-06 21:30:00,681 - __main__ - INFO - GitHub scraper initialized and enabled

2026-07-06 21:30:00,682 - RSSFeedScraper - INFO - RSS scraper initialized: feeds=2, html_urls=0 (0 with custom selectors), include_terms=0, exclude_terms=0

2026-07-06 21:30:00,682 - __main__ - INFO - LinkedIn Feed scraper initialized and enabled

2026-07-06 21:30:00,684 - Yiddish24Scraper - INFO - File downloader initialized with 55 terms, prefix 'Y2', and file type organization enabled

2026-07-06 21:30:00,684 - Yiddish24Scraper - INFO - Yiddish24 scraper initialized: data_id='latest', page_limit=100, total_pages=10, debug_html=True, include_terms=55, exclude_terms=0

2026-07-06 21:30:00,687 - AhBlickLiveScraper - INFO - AhBlickLive scraper initialized: urls=1, max_items=100, include_terms=49, exclude_terms=0

2026-07-06 21:30:00,691 - TorahAnytimeScraper - INFO - File downloader initialized with 53 terms, prefix 'TA', and file type organization enabled

2026-07-06 21:30:00,691 - TorahAnytimeScraper - INFO - TorahAnytime scraper initialized: limit=400, offset=0, project_id=1, include_terms=53, exclude_terms=10

2026-07-06 21:30:00,694 - BatorahScraper - INFO - File downloader initialized with 19 terms, prefix 'BT', and file type organization enabled

2026-07-06 21:30:00,694 - BatorahScraper - INFO - Batorah scraper initialized: api_urls=2, page_size=100, include_terms=19, exclude_terms=1

2026-07-06 21:30:00,694 - KolHalashonScraper - INFO - Kol Halashon scraper initialized: base_url=https://www2.kolhalashon.com, max_shiurim=50, fetch_all_speakers=True, headless=False, exclude_filters=1, include_filters=0

2026-07-06 21:30:00,694 - __main__ - INFO - Running 2 scrapers concurrently...

2026-07-06 21:30:00,694 - GitHubScraper - INFO - Searching GitHub for terms: ['paperclip', '"Hermes Agent"', 'multica']

2026-07-06 21:30:00,789 - LinkedInFeedScraper - INFO - Fetching LinkedIn feed (25 posts)

2026-07-06 21:30:03,479 - LinkedInFeedScraper - ERROR - LinkedIn feed fetch failed: Unauthorized: LinkedIn session cookies are expired or invalid.

2026-07-06 21:30:03,487 - LinkedInFeedScraper - INFO - LinkedIn Feed scrape complete. Found 0 new posts.

2026-07-06 21:30:04,505 - httpx - INFO - HTTP Request: GET https://api.github.com/search/repositories?q=paperclip+in%3Aname%2Cdescription%2Creadme+stars%3A%3E150&sort=updated&order=desc&per_page=20 "HTTP/1.1 200 OK"

2026-07-06 21:30:04,601 - GitHubScraper - INFO - Found 242 GitHub repositories for term: paperclip

2026-07-06 21:30:05,846 - httpx - INFO - HTTP Request: GET https://api.github.com/search/repositories?q=%22Hermes+Agent%22+in%3Aname%2Cdescription%2Creadme+stars%3A%3E150&sort=updated&order=desc&per_page=20 "HTTP/1.1 200 OK"

2026-07-06 21:30:05,851 - GitHubScraper - INFO - Found 427 GitHub repositories for term: "Hermes Agent"

2026-07-06 21:30:06,864 - httpx - INFO - HTTP Request: GET https://api.github.com/search/repositories?q=multica+in%3Aname%2Cdescription%2Creadme+stars%3A%3E150&sort=updated&order=desc&per_page=20 "HTTP/1.1 200 OK"

2026-07-06 21:30:06,957 - GitHubScraper - INFO - Found 56 GitHub repositories for term: multica

2026-07-06 21:30:07,087 - GitHubScraper - INFO - Total GitHub results found: 0

2026-07-06 21:30:07,087 - __main__ - INFO - Scraper github completed: 0 results

2026-07-06 21:30:07,087 - __main__ - INFO - Scraper linkedin_feed completed: 0 results

2026-07-06 21:30:07,088 - __main__ - WARNING - LinkedIn Feed: 40 consecutive failures (fetch failed)

2026-07-06 21:30:07,088 - __main__ - INFO - No new results found across all enabled sources

2026-07-06 21:30:07,088 - __main__ - INFO - Cache stats: 0 items in memory, 0 files on disk

2026-07-06 21:30:07,088 - __main__ - INFO - Monitor complete. No new results found across all sources.

2026-07-06 21:30:07,088 - __main__ - INFO - === Social Media Monitor v2.0 Complete ===

2026-07-06 21:30:07,088 - social_media_monitor.database.connection - INFO - Closing all database connections

2026-07-06 21:30:07,107 - social_media_monitor.database.connection - INFO - Closed 5 database connections

2026-07-06 21:30:07,107 - __main__ - INFO - Closed all database connections

2026-07-06 21:30:07,109 - social_media_monitor.database.connection - INFO - Closing all database connections

2026-07-06 21:30:07,109 - social_media_monitor.database.connection - INFO - Closed 0 database connections

2026-07-06 22:04:39,291 - social_media_monitor.utils.logging_config - INFO - Posts log file: /mnt/data/openclaw-shared/home/shared/Social_Media_Monitor/logs/social_media_monitor_posts_2026-07-06.log

2026-07-06 22:04:39,291 - social_media_monitor.utils.logging_config - INFO - Downloads log file: /mnt/data/openclaw-shared/home/shared/Social_Media_Monitor/logs/social_media_monitor_downloads_2026-07-06.log

2026-07-06 22:04:39,291 - social_media_monitor.utils.logging_config - INFO - Logging configured with level INFO, log file: /mnt/data/openclaw-shared/home/shared/Social_Media_Monitor/logs/social_media_monitor_2026-07-06.log

2026-07-06 22:04:39,291 - __main__ - INFO - === Social Media Monitor v2.0 (Async) Started ===

2026-07-06 22:04:39,323 - social_media_monitor.config.settings - INFO - Configuration loaded from /home/openclaw/Social_Media_Monitor/config.yaml

2026-07-06 22:04:39,323 - __main__ - INFO - Configuration loaded and validated successfully

2026-07-06 22:04:39,326 - social_media_monitor.database.connection - INFO - Database connection pool initialized with 5 connections

2026-07-06 22:04:39,326 - social_media_monitor.database.connection - INFO - add_speaker_filters column already exists

2026-07-06 22:04:39,357 - social_media_monitor.database.connection - INFO - Database setup completed successfully

2026-07-06 22:04:39,358 - __main__ - INFO - Database initialized at: /mnt/data/openclaw-shared/home/shared/Social_Media_Monitor/social_media_monitor.db

2026-07-06 22:04:39,403 - social_media_monitor.scrapers.reddit_scraper - INFO - Reddit API initialized successfully

2026-07-06 22:04:39,404 - RSSFeedScraper - INFO - RSS scraper initialized: feeds=2, html_urls=0 (0 with custom selectors), include_terms=0, exclude_terms=0

2026-07-06 22:04:39,404 - __main__ - INFO - RSS scraper initialized and enabled

2026-07-06 22:04:39,408 - Yiddish24Scraper - INFO - File downloader initialized with 55 terms, prefix 'Y2', and file type organization enabled

2026-07-06 22:04:39,408 - Yiddish24Scraper - INFO - Yiddish24 scraper initialized: data_id='latest', page_limit=100, total_pages=10, debug_html=True, include_terms=55, exclude_terms=0

2026-07-06 22:04:39,408 - __main__ - INFO - Yiddish24 scraper initialized and enabled

2026-07-06 22:04:39,410 - AhBlickLiveScraper - INFO - AhBlickLive scraper initialized: urls=1, max_items=100, include_terms=49, exclude_terms=0

2026-07-06 22:04:39,415 - TorahAnytimeScraper - INFO - File downloader initialized with 53 terms, prefix 'TA', and file type organization enabled

2026-07-06 22:04:39,415 - TorahAnytimeScraper - INFO - TorahAnytime scraper initialized: limit=400, offset=0, project_id=1, include_terms=53, exclude_terms=10

2026-07-06 22:04:39,417 - BatorahScraper - INFO - File downloader initialized with 19 terms, prefix 'BT', and file type organization enabled

2026-07-06 22:04:39,417 - BatorahScraper - INFO - Batorah scraper initialized: api_urls=2, page_size=100, include_terms=19, exclude_terms=1

2026-07-06 22:04:39,417 - KolHalashonScraper - INFO - Kol Halashon scraper initialized: base_url=https://www2.kolhalashon.com, max_shiurim=50, fetch_all_speakers=True, headless=False, exclude_filters=1, include_filters=0

2026-07-06 22:04:39,417 - __main__ - INFO - Ivelt scraper initialized and enabled

2026-07-06 22:04:39,417 - __main__ - INFO - Running 3 scrapers concurrently...

2026-07-06 22:04:39,417 - Yiddish24Scraper - INFO - Starting Yiddish24 scraping (max 10 pages, limit 100 items)

2026-07-06 22:04:39,507 - Yiddish24Scraper - INFO - Fetching Yiddish24 page 1 with data_id='latest', page_limit=100

2026-07-06 22:04:39,511 - IveltScraper - INFO - Loaded 181 include, 0 exclude filters from DB topic_filters

2026-07-06 22:04:39,511 - IveltScraper - INFO - Fetching ivelt active topics from https://www.ivelt.com/forum/search.php?search_id=active_topics...

2026-07-06 22:04:39,514 - RSSFeedScraper - INFO - Fetching RSS feed: https://www.theyeshivaworld.com/feed

2026-07-06 22:04:39,776 - RSSFeedScraper - INFO - Fetching RSS feed: https://www.thegatewaypundit.com/feed/

2026-07-06 22:04:40,203 - RSSFeedScraper - INFO - Total RSS/HTML results: 3

2026-07-06 22:04:43,696 - httpx - INFO - HTTP Request: POST https://www.yiddish24.com/ajax/cat_pagination.php "HTTP/1.1 200 OK"

2026-07-06 22:04:43,910 - Yiddish24Scraper - INFO - Received response with 472037 characters

2026-07-06 22:04:43,914 - Yiddish24Scraper - INFO - Extracted HTML from JSON result field (299771 chars)

2026-07-06 22:04:43,918 - Yiddish24Scraper - INFO - Saved HTML debug file to: /mnt/data/openclaw-shared/home/shared/Social_Media_Monitor/debug/yiddish24_page1_response.html

2026-07-06 22:04:44,017 - Yiddish24Scraper - INFO - Found 100 potential article elements on page 1

2026-07-06 22:04:44,064 - Yiddish24Scraper - INFO - Page 1 extraction: 100 successful, 0 failed out of 100 elements

2026-07-06 22:04:44,064 - Yiddish24Scraper - INFO - Parsed 100 articles from HTML on page 1

2026-07-06 22:04:44,775 - Yiddish24Scraper - INFO - Page 1 summary: 10 new out of 100 total articles

2026-07-06 22:04:44,775 - Yiddish24Scraper - INFO - Page 1: 10/100 new items

2026-07-06 22:04:44,775 - Yiddish24Scraper - INFO - Low new item ratio (10.0%), stopping pagination

2026-07-06 22:04:44,776 - Yiddish24Scraper - INFO - Total Yiddish24 results found: 10

2026-07-06 22:04:51,620 - IveltScraper - INFO - Found 50 active topics on ivelt

2026-07-06 22:05:04,007 - IveltScraper - INFO - Filtering post containing exclude term: '*ביטע מאכט זיכער בלויז צו באריכטן קראנטע נייעס*'

2026-07-06 22:05:11,541 - IveltScraper - INFO - ivelt scrape complete. Checked 4 topics.

2026-07-06 22:05:11,541 - IveltScraper - INFO - All forums scrape complete. Found 14 new posts.

2026-07-06 22:05:11,542 - __main__ - INFO - Scraper rss completed: 3 results

2026-07-06 22:05:11,542 - __main__ - INFO - Scraper yiddish24 completed: 10 results

2026-07-06 22:05:11,542 - __main__ - INFO - Scraper ivelt completed: 14 results

2026-07-06 22:05:12,343 - social_media_monitor.email.sender - INFO - Email sent successfully at 2026-07-06 22:05:12.343372

2026-07-06 22:05:12,343 - __main__ - INFO - Email sent successfully

2026-07-06 22:05:12,344 - __main__ - INFO - Cache stats: 0 items in memory, 0 files on disk

2026-07-06 22:05:12,344 - __main__ - INFO - Monitor complete. Found 27 total results: 3 new rss results, 10 new yiddish24 results, 14 new ivelt results.

2026-07-06 22:05:12,344 - __main__ - INFO - === Social Media Monitor v2.0 Complete ===

2026-07-06 22:05:12,344 - social_media_monitor.database.connection - INFO - Closing all database connections

2026-07-06 22:05:12,421 - social_media_monitor.database.connection - INFO - Closed 5 database connections

2026-07-06 22:05:12,422 - __main__ - INFO - Closed all database connections

2026-07-06 22:05:12,424 - social_media_monitor.database.connection - INFO - Closing all database connections

2026-07-06 22:05:12,424 - social_media_monitor.database.connection - INFO - Closed 0 database connections

2026-07-06 22:30:00,659 - social_media_monitor.utils.logging_config - INFO - Posts log file: /mnt/data/openclaw-shared/home/shared/Social_Media_Monitor/logs/social_media_monitor_posts_2026-07-06.log

2026-07-06 22:30:00,659 - social_media_monitor.utils.logging_config - INFO - Downloads log file: /mnt/data/openclaw-shared/home/shared/Social_Media_Monitor/logs/social_media_monitor_downloads_2026-07-06.log

2026-07-06 22:30:00,660 - social_media_monitor.utils.logging_config - INFO - Logging configured with level INFO, log file: /mnt/data/openclaw-shared/home/shared/Social_Media_Monitor/logs/social_media_monitor_2026-07-06.log

2026-07-06 22:30:00,660 - __main__ - INFO - === Social Media Monitor v2.0 (Async) Started ===

2026-07-06 22:30:00,690 - social_media_monitor.config.settings - INFO - Configuration loaded from /home/openclaw/Social_Media_Monitor/config.yaml

2026-07-06 22:30:00,691 - __main__ - INFO - Configuration loaded and validated successfully

2026-07-06 22:30:00,693 - social_media_monitor.database.connection - INFO - Database connection pool initialized with 5 connections

2026-07-06 22:30:00,693 - social_media_monitor.database.connection - INFO - add_speaker_filters column already exists

2026-07-06 22:30:00,714 - social_media_monitor.database.connection - INFO - Database setup completed successfully

2026-07-06 22:30:00,714 - __main__ - INFO - Database initialized at: /mnt/data/openclaw-shared/home/shared/Social_Media_Monitor/social_media_monitor.db

2026-07-06 22:30:00,717 - social_media_monitor.scrapers.reddit_scraper - INFO - Reddit API initialized successfully

2026-07-06 22:30:00,717 - __main__ - INFO - GitHub scraper initialized and enabled

2026-07-06 22:30:00,718 - RSSFeedScraper - INFO - RSS scraper initialized: feeds=2, html_urls=0 (0 with custom selectors), include_terms=0, exclude_terms=0

2026-07-06 22:30:00,718 - __main__ - INFO - LinkedIn Feed scraper initialized and enabled

2026-07-06 22:30:00,721 - Yiddish24Scraper - INFO - File downloader initialized with 55 terms, prefix 'Y2', and file type organization enabled

2026-07-06 22:30:00,721 - Yiddish24Scraper - INFO - Yiddish24 scraper initialized: data_id='latest', page_limit=100, total_pages=10, debug_html=True, include_terms=55, exclude_terms=0

2026-07-06 22:30:00,724 - AhBlickLiveScraper - INFO - AhBlickLive scraper initialized: urls=1, max_items=100, include_terms=49, exclude_terms=0

2026-07-06 22:30:00,728 - TorahAnytimeScraper - INFO - File downloader initialized with 53 terms, prefix 'TA', and file type organization enabled

2026-07-06 22:30:00,728 - TorahAnytimeScraper - INFO - TorahAnytime scraper initialized: limit=400, offset=0, project_id=1, include_terms=53, exclude_terms=10

2026-07-06 22:30:00,730 - BatorahScraper - INFO - File downloader initialized with 19 terms, prefix 'BT', and file type organization enabled

2026-07-06 22:30:00,730 - BatorahScraper - INFO - Batorah scraper initialized: api_urls=2, page_size=100, include_terms=19, exclude_terms=1

2026-07-06 22:30:00,731 - KolHalashonScraper - INFO - Kol Halashon scraper initialized: base_url=https://www2.kolhalashon.com, max_shiurim=50, fetch_all_speakers=True, headless=False, exclude_filters=1, include_filters=0

2026-07-06 22:30:00,731 - __main__ - INFO - Running 2 scrapers concurrently...

2026-07-06 22:30:00,731 - GitHubScraper - INFO - Searching GitHub for terms: ['paperclip', '"Hermes Agent"', 'multica']

2026-07-06 22:30:00,824 - LinkedInFeedScraper - INFO - Fetching LinkedIn feed (25 posts)

2026-07-06 22:30:04,514 - LinkedInFeedScraper - ERROR - LinkedIn feed fetch failed: Unauthorized: LinkedIn session cookies are expired or invalid.

2026-07-06 22:30:04,523 - LinkedInFeedScraper - INFO - LinkedIn Feed scrape complete. Found 0 new posts.

2026-07-06 22:30:05,595 - httpx - INFO - HTTP Request: GET https://api.github.com/search/repositories?q=paperclip+in%3Aname%2Cdescription%2Creadme+stars%3A%3E150&sort=updated&order=desc&per_page=20 "HTTP/1.1 200 OK"

2026-07-06 22:30:05,621 - GitHubScraper - INFO - Found 242 GitHub repositories for term: paperclip

2026-07-06 22:30:06,716 - httpx - INFO - HTTP Request: GET https://api.github.com/search/repositories?q=%22Hermes+Agent%22+in%3Aname%2Cdescription%2Creadme+stars%3A%3E150&sort=updated&order=desc&per_page=20 "HTTP/1.1 200 OK"

2026-07-06 22:30:06,809 - GitHubScraper - INFO - Found 427 GitHub repositories for term: "Hermes Agent"

2026-07-06 22:30:06,918 - GitHubScraper - INFO - Added new GitHub repository: ODS

2026-07-06 22:30:07,782 - httpx - INFO - HTTP Request: GET https://api.github.com/search/repositories?q=multica+in%3Aname%2Cdescription%2Creadme+stars%3A%3E150&sort=updated&order=desc&per_page=20 "HTTP/1.1 200 OK"

2026-07-06 22:30:07,787 - GitHubScraper - INFO - Found 56 GitHub repositories for term: multica

2026-07-06 22:30:07,914 - GitHubScraper - INFO - Total GitHub results found: 1

2026-07-06 22:30:07,914 - __main__ - INFO - Scraper github completed: 1 results

2026-07-06 22:30:07,915 - __main__ - INFO - Scraper linkedin_feed completed: 0 results

2026-07-06 22:30:07,915 - __main__ - WARNING - LinkedIn Feed: 41 consecutive failures (fetch failed)

2026-07-06 22:30:08,663 - social_media_monitor.email.sender - INFO - Email sent successfully at 2026-07-06 22:30:08.663422

2026-07-06 22:30:08,663 - __main__ - INFO - Email sent successfully

2026-07-06 22:30:08,664 - __main__ - INFO - Cache stats: 0 items in memory, 0 files on disk

2026-07-06 22:30:08,664 - __main__ - INFO - Monitor complete. Found 1 total results: 1 new github results.

2026-07-06 22:30:08,664 - __main__ - INFO - === Social Media Monitor v2.0 Complete ===

2026-07-06 22:30:08,664 - social_media_monitor.database.connection - INFO - Closing all database connections

2026-07-06 22:30:08,685 - social_media_monitor.database.connection - INFO - Closed 5 database connections

2026-07-06 22:30:08,685 - __main__ - INFO - Closed all database connections

2026-07-06 22:30:08,687 - social_media_monitor.database.connection - INFO - Closing all database connections

2026-07-06 22:30:08,687 - social_media_monitor.database.connection - INFO - Closed 0 database connections

2026-07-06 23:04:39,277 - social_media_monitor.utils.logging_config - INFO - Posts log file: /mnt/data/openclaw-shared/home/shared/Social_Media_Monitor/logs/social_media_monitor_posts_2026-07-06.log

2026-07-06 23:04:39,277 - social_media_monitor.utils.logging_config - INFO - Downloads log file: /mnt/data/openclaw-shared/home/shared/Social_Media_Monitor/logs/social_media_monitor_downloads_2026-07-06.log

2026-07-06 23:04:39,277 - social_media_monitor.utils.logging_config - INFO - Logging configured with level INFO, log file: /mnt/data/openclaw-shared/home/shared/Social_Media_Monitor/logs/social_media_monitor_2026-07-06.log

2026-07-06 23:04:39,277 - __main__ - INFO - === Social Media Monitor v2.0 (Async) Started ===

2026-07-06 23:04:39,308 - social_media_monitor.config.settings - INFO - Configuration loaded from /home/openclaw/Social_Media_Monitor/config.yaml

2026-07-06 23:04:39,308 - __main__ - INFO - Configuration loaded and validated successfully

2026-07-06 23:04:39,310 - social_media_monitor.database.connection - INFO - Database connection pool initialized with 5 connections

2026-07-06 23:04:39,311 - social_media_monitor.database.connection - INFO - add_speaker_filters column already exists

2026-07-06 23:04:39,326 - social_media_monitor.database.connection - INFO - Database setup completed successfully

2026-07-06 23:04:39,326 - __main__ - INFO - Database initialized at: /mnt/data/openclaw-shared/home/shared/Social_Media_Monitor/social_media_monitor.db

2026-07-06 23:04:39,328 - social_media_monitor.scrapers.reddit_scraper - INFO - Reddit API initialized successfully

2026-07-06 23:04:39,329 - RSSFeedScraper - INFO - RSS scraper initialized: feeds=2, html_urls=0 (0 with custom selectors), include_terms=0, exclude_terms=0

2026-07-06 23:04:39,329 - __main__ - INFO - RSS scraper initialized and enabled

2026-07-06 23:04:39,332 - Yiddish24Scraper - INFO - File downloader initialized with 55 terms, prefix 'Y2', and file type organization enabled

2026-07-06 23:04:39,332 - Yiddish24Scraper - INFO - Yiddish24 scraper initialized: data_id='latest', page_limit=100, total_pages=10, debug_html=True, include_terms=55, exclude_terms=0

2026-07-06 23:04:39,332 - __main__ - INFO - Yiddish24 scraper initialized and enabled

2026-07-06 23:04:39,335 - AhBlickLiveScraper - INFO - AhBlickLive scraper initialized: urls=1, max_items=100, include_terms=49, exclude_terms=0

2026-07-06 23:04:39,339 - TorahAnytimeScraper - INFO - File downloader initialized with 53 terms, prefix 'TA', and file type organization enabled

2026-07-06 23:04:39,339 - TorahAnytimeScraper - INFO - TorahAnytime scraper initialized: limit=400, offset=0, project_id=1, include_terms=53, exclude_terms=10

2026-07-06 23:04:39,342 - BatorahScraper - INFO - File downloader initialized with 19 terms, prefix 'BT', and file type organization enabled

2026-07-06 23:04:39,342 - BatorahScraper - INFO - Batorah scraper initialized: api_urls=2, page_size=100, include_terms=19, exclude_terms=1

2026-07-06 23:04:39,342 - KolHalashonScraper - INFO - Kol Halashon scraper initialized: base_url=https://www2.kolhalashon.com, max_shiurim=50, fetch_all_speakers=True, headless=False, exclude_filters=1, include_filters=0

2026-07-06 23:04:39,342 - __main__ - INFO - Ivelt scraper initialized and enabled

2026-07-06 23:04:39,342 - __main__ - INFO - Running 3 scrapers concurrently...

2026-07-06 23:04:39,342 - Yiddish24Scraper - INFO - Starting Yiddish24 scraping (max 10 pages, limit 100 items)

2026-07-06 23:04:39,435 - Yiddish24Scraper - INFO - Fetching Yiddish24 page 1 with data_id='latest', page_limit=100

2026-07-06 23:04:39,440 - IveltScraper - INFO - Loaded 181 include, 0 exclude filters from DB topic_filters

2026-07-06 23:04:39,440 - IveltScraper - INFO - Fetching ivelt active topics from https://www.ivelt.com/forum/search.php?search_id=active_topics...

2026-07-06 23:04:39,441 - RSSFeedScraper - INFO - Fetching RSS feed: https://www.theyeshivaworld.com/feed

2026-07-06 23:04:39,743 - RSSFeedScraper - INFO - Fetching RSS feed: https://www.thegatewaypundit.com/feed/

2026-07-06 23:04:40,255 - RSSFeedScraper - INFO - Total RSS/HTML results: 3

2026-07-06 23:04:46,565 - httpx - INFO - HTTP Request: POST https://www.yiddish24.com/ajax/cat_pagination.php "HTTP/1.1 200 OK"

2026-07-06 23:04:46,787 - Yiddish24Scraper - INFO - Received response with 464937 characters

2026-07-06 23:04:46,791 - Yiddish24Scraper - INFO - Extracted HTML from JSON result field (296265 chars)

2026-07-06 23:04:46,794 - Yiddish24Scraper - INFO - Saved HTML debug file to: /mnt/data/openclaw-shared/home/shared/Social_Media_Monitor/debug/yiddish24_page1_response.html

2026-07-06 23:04:46,875 - Yiddish24Scraper - INFO - Found 100 potential article elements on page 1

2026-07-06 23:04:46,917 - Yiddish24Scraper - INFO - Page 1 extraction: 100 successful, 0 failed out of 100 elements

2026-07-06 23:04:46,917 - Yiddish24Scraper - INFO - Parsed 100 articles from HTML on page 1

2026-07-06 23:04:47,854 - Yiddish24Scraper - INFO - Page 1 summary: 3 new out of 100 total articles

2026-07-06 23:04:47,855 - Yiddish24Scraper - INFO - Page 1: 3/100 new items

2026-07-06 23:04:47,855 - Yiddish24Scraper - INFO - Low new item ratio (3.0%), stopping pagination

2026-07-06 23:04:47,855 - Yiddish24Scraper - INFO - Total Yiddish24 results found: 3

2026-07-06 23:04:51,623 - IveltScraper - INFO - Found 50 active topics on ivelt

2026-07-06 23:05:19,208 - IveltScraper - INFO - ivelt scrape complete. Checked 4 topics.

2026-07-06 23:05:19,209 - IveltScraper - INFO - All forums scrape complete. Found 23 new posts.

2026-07-06 23:05:19,209 - __main__ - INFO - Scraper rss completed: 3 results

2026-07-06 23:05:19,209 - __main__ - INFO - Scraper yiddish24 completed: 3 results

2026-07-06 23:05:19,209 - __main__ - INFO - Scraper ivelt completed: 23 results

2026-07-06 23:05:20,346 - social_media_monitor.email.sender - INFO - Email sent successfully at 2026-07-06 23:05:20.346082

2026-07-06 23:05:20,346 - __main__ - INFO - Email sent successfully

2026-07-06 23:05:20,346 - __main__ - INFO - Cache stats: 0 items in memory, 0 files on disk

2026-07-06 23:05:20,346 - __main__ - INFO - Monitor complete. Found 29 total results: 3 new rss results, 3 new yiddish24 results, 23 new ivelt results.

2026-07-06 23:05:20,347 - __main__ - INFO - === Social Media Monitor v2.0 Complete ===

2026-07-06 23:05:20,347 - social_media_monitor.database.connection - INFO - Closing all database connections

2026-07-06 23:05:20,469 - social_media_monitor.database.connection - INFO - Closed 5 database connections

2026-07-06 23:05:20,469 - __main__ - INFO - Closed all database connections

2026-07-06 23:05:20,471 - social_media_monitor.database.connection - INFO - Closing all database connections

2026-07-06 23:05:20,471 - social_media_monitor.database.connection - INFO - Closed 0 database connections

2026-07-06 23:30:00,900 - social_media_monitor.utils.logging_config - INFO - Posts log file: /mnt/data/openclaw-shared/home/shared/Social_Media_Monitor/logs/social_media_monitor_posts_2026-07-06.log

2026-07-06 23:30:00,902 - social_media_monitor.utils.logging_config - INFO - Downloads log file: /mnt/data/openclaw-shared/home/shared/Social_Media_Monitor/logs/social_media_monitor_downloads_2026-07-06.log

2026-07-06 23:30:00,902 - social_media_monitor.utils.logging_config - INFO - Logging configured with level INFO, log file: /mnt/data/openclaw-shared/home/shared/Social_Media_Monitor/logs/social_media_monitor_2026-07-06.log

2026-07-06 23:30:00,902 - __main__ - INFO - === Social Media Monitor v2.0 (Async) Started ===

2026-07-06 23:30:00,932 - social_media_monitor.config.settings - INFO - Configuration loaded from /home/openclaw/Social_Media_Monitor/config.yaml

2026-07-06 23:30:00,933 - __main__ - INFO - Configuration loaded and validated successfully

2026-07-06 23:30:00,939 - social_media_monitor.database.connection - INFO - Database connection pool initialized with 5 connections

2026-07-06 23:30:00,944 - social_media_monitor.database.connection - INFO - add_speaker_filters column already exists

2026-07-06 23:30:00,958 - social_media_monitor.database.connection - INFO - Database setup completed successfully

2026-07-06 23:30:00,958 - __main__ - INFO - Database initialized at: /mnt/data/openclaw-shared/home/shared/Social_Media_Monitor/social_media_monitor.db

2026-07-06 23:30:01,003 - social_media_monitor.scrapers.reddit_scraper - INFO - Reddit API initialized successfully

2026-07-06 23:30:01,004 - __main__ - INFO - GitHub scraper initialized and enabled

2026-07-06 23:30:01,010 - RSSFeedScraper - INFO - RSS scraper initialized: feeds=2, html_urls=0 (0 with custom selectors), include_terms=0, exclude_terms=0

2026-07-06 23:30:01,010 - __main__ - INFO - LinkedIn Feed scraper initialized and enabled

2026-07-06 23:30:01,016 - Yiddish24Scraper - INFO - File downloader initialized with 55 terms, prefix 'Y2', and file type organization enabled

2026-07-06 23:30:01,016 - Yiddish24Scraper - INFO - Yiddish24 scraper initialized: data_id='latest', page_limit=100, total_pages=10, debug_html=True, include_terms=55, exclude_terms=0

2026-07-06 23:30:01,018 - AhBlickLiveScraper - INFO - AhBlickLive scraper initialized: urls=1, max_items=100, include_terms=49, exclude_terms=0

2026-07-06 23:30:01,022 - TorahAnytimeScraper - INFO - File downloader initialized with 53 terms, prefix 'TA', and file type organization enabled

2026-07-06 23:30:01,023 - TorahAnytimeScraper - INFO - TorahAnytime scraper initialized: limit=400, offset=0, project_id=1, include_terms=53, exclude_terms=10

2026-07-06 23:30:01,025 - BatorahScraper - INFO - File downloader initialized with 19 terms, prefix 'BT', and file type organization enabled

2026-07-06 23:30:01,025 - BatorahScraper - INFO - Batorah scraper initialized: api_urls=2, page_size=100, include_terms=19, exclude_terms=1

2026-07-06 23:30:01,025 - KolHalashonScraper - INFO - Kol Halashon scraper initialized: base_url=https://www2.kolhalashon.com, max_shiurim=50, fetch_all_speakers=True, headless=False, exclude_filters=1, include_filters=0

2026-07-06 23:30:01,025 - __main__ - INFO - Running 2 scrapers concurrently...

2026-07-06 23:30:01,025 - GitHubScraper - INFO - Searching GitHub for terms: ['paperclip', '"Hermes Agent"', 'multica']

2026-07-06 23:30:01,158 - LinkedInFeedScraper - INFO - Fetching LinkedIn feed (25 posts)

2026-07-06 23:30:06,793 - LinkedInFeedScraper - ERROR - LinkedIn feed fetch failed: Unauthorized: LinkedIn session cookies are expired or invalid.

2026-07-06 23:30:06,801 - LinkedInFeedScraper - INFO - LinkedIn Feed scrape complete. Found 0 new posts.

2026-07-06 23:30:07,722 - httpx - INFO - HTTP Request: GET https://api.github.com/search/repositories?q=paperclip+in%3Aname%2Cdescription%2Creadme+stars%3A%3E150&sort=updated&order=desc&per_page=20 "HTTP/1.1 200 OK"

2026-07-06 23:30:07,747 - GitHubScraper - INFO - Found 242 GitHub repositories for term: paperclip

2026-07-06 23:30:08,709 - httpx - INFO - HTTP Request: GET https://api.github.com/search/repositories?q=%22Hermes+Agent%22+in%3Aname%2Cdescription%2Creadme+stars%3A%3E150&sort=updated&order=desc&per_page=20 "HTTP/1.1 200 OK"

2026-07-06 23:30:08,713 - GitHubScraper - INFO - Found 427 GitHub repositories for term: "Hermes Agent"

2026-07-06 23:30:09,689 - httpx - INFO - HTTP Request: GET https://api.github.com/search/repositories?q=multica+in%3Aname%2Cdescription%2Creadme+stars%3A%3E150&sort=updated&order=desc&per_page=20 "HTTP/1.1 200 OK"

2026-07-06 23:30:09,692 - GitHubScraper - INFO - Found 56 GitHub repositories for term: multica

2026-07-06 23:30:09,827 - GitHubScraper - INFO - Total GitHub results found: 0

2026-07-06 23:30:09,827 - __main__ - INFO - Scraper github completed: 0 results

2026-07-06 23:30:09,827 - __main__ - INFO - Scraper linkedin_feed completed: 0 results

2026-07-06 23:30:09,829 - __main__ - WARNING - LinkedIn Feed: 42 consecutive failures (fetch failed)

2026-07-06 23:30:09,829 - __main__ - INFO - No new results found across all enabled sources

2026-07-06 23:30:09,830 - __main__ - INFO - Cache stats: 0 items in memory, 0 files on disk

2026-07-06 23:30:09,830 - __main__ - INFO - Monitor complete. No new results found across all sources.

2026-07-06 23:30:09,830 - __main__ - INFO - === Social Media Monitor v2.0 Complete ===

2026-07-06 23:30:09,830 - social_media_monitor.database.connection - INFO - Closing all database connections

2026-07-06 23:30:09,842 - social_media_monitor.database.connection - INFO - Closed 5 database connections

2026-07-06 23:30:09,842 - __main__ - INFO - Closed all database connections

2026-07-06 23:30:09,844 - social_media_monitor.database.connection - INFO - Closing all database connections

2026-07-06 23:30:09,844 - social_media_monitor.database.connection - INFO - Closed 0 database connections