โ† Logs

social_media_monitor_2026-06-28.log

2026-06-28 20:30:02,065 - social_media_monitor.utils.logging_config - INFO - Logging configured with level INFO, log file: /mnt/data/openclaw-shared/home/shared/Social_Media_Monitor/logs/social_media_monitor_2026-06-28.log
2026-06-28 20:30:02,065 - __main__ - INFO - === Social Media Monitor v2.0 (Async) Started ===
2026-06-28 20:30:02,092 - social_media_monitor.config.settings - INFO - Configuration loaded from /home/openclaw/Social_Media_Monitor/config.yaml
2026-06-28 20:30:02,092 - social_media_monitor.config.settings - WARNING - Twitter monitoring enabled but credentials validation not implemented
2026-06-28 20:30:02,092 - __main__ - INFO - Configuration loaded and validated successfully
2026-06-28 20:30:02,095 - social_media_monitor.database.connection - INFO - Database connection pool initialized with 5 connections
2026-06-28 20:30:02,095 - social_media_monitor.database.connection - INFO - add_speaker_filters column already exists
2026-06-28 20:30:02,110 - social_media_monitor.database.connection - INFO - Database setup completed successfully
2026-06-28 20:30:02,110 - __main__ - INFO - Database initialized at: /mnt/data/openclaw-shared/home/shared/Social_Media_Monitor/social_media_monitor.db
2026-06-28 20:30:02,113 - social_media_monitor.scrapers.reddit_scraper - INFO - Reddit API initialized successfully
2026-06-28 20:30:02,113 - __main__ - INFO - GitHub scraper initialized and enabled
2026-06-28 20:30:02,113 - __main__ - INFO - GitHub feed scraper initialized and enabled
2026-06-28 20:30:02,113 - RSSFeedScraper - INFO - RSS scraper initialized: feeds=2, html_urls=0 (0 with custom selectors), include_terms=0, exclude_terms=0
2026-06-28 20:30:02,114 - __main__ - INFO - LinkedIn Feed scraper initialized and enabled
2026-06-28 20:30:02,116 - Yiddish24Scraper - INFO - File downloader initialized with 55 terms, prefix 'Y2', and file type organization enabled
2026-06-28 20:30:02,116 - Yiddish24Scraper - INFO - Yiddish24 scraper initialized: data_id='latest', page_limit=100, total_pages=10, debug_html=True, include_terms=55, exclude_terms=0
2026-06-28 20:30:02,119 - AhBlickLiveScraper - INFO - AhBlickLive scraper initialized: urls=1, max_items=100, include_terms=49, exclude_terms=0
2026-06-28 20:30:02,123 - TorahAnytimeScraper - INFO - File downloader initialized with 53 terms, prefix 'TA', and file type organization enabled
2026-06-28 20:30:02,123 - TorahAnytimeScraper - INFO - TorahAnytime scraper initialized: limit=400, offset=0, project_id=1, include_terms=53, exclude_terms=10
2026-06-28 20:30:02,126 - BatorahScraper - INFO - File downloader initialized with 19 terms, prefix 'BT', and file type organization enabled
2026-06-28 20:30:02,126 - BatorahScraper - INFO - Batorah scraper initialized: api_urls=2, page_size=100, include_terms=19, exclude_terms=1
2026-06-28 20:30:02,126 - KolHalashonScraper - INFO - Kol Halashon scraper initialized: base_url=https://www2.kolhalashon.com, max_shiurim=50, fetch_all_speakers=True, headless=False, exclude_filters=1, include_filters=0
2026-06-28 20:30:02,126 - __main__ - INFO - Running 3 scrapers concurrently...
2026-06-28 20:30:02,126 - GitHubScraper - INFO - Searching GitHub for terms: ['paperclip', '"Hermes Agent"', 'multica']
2026-06-28 20:30:02,230 - LinkedInFeedScraper - INFO - Fetching LinkedIn feed (25 posts)
2026-06-28 20:31:02,424 - LinkedInFeedScraper - ERROR - LinkedIn cookie auto-grab from Chrome (port 9224) failed: Message: session not created: cannot connect to chrome at 127.0.0.1:9224
from chrome not reachable; For documentation on this error, please visit: https://www.selenium.dev/documentation/webdriver/troubleshooting/errors#sessionnotcreatedexception
Stacktrace:
#0 0x5e4bf8e743da <unknown>
#1 0x5e4bf8855ef0 <unknown>
#2 0x5e4bf884176c <unknown>
#3 0x5e4bf8899b3e <unknown>
#4 0x5e4bf888eca1 <unknown>
#5 0x5e4bf88ded96 <unknown>
#6 0x5e4bf88de47c <unknown>
#7 0x5e4bf889de4f <unknown>
#8 0x5e4bf889ec31 <unknown>
#9 0x5e4bf8e3a9f7 <unknown>
#10 0x5e4bf8e39203 <unknown>
#11 0x5e4bf8e23f16 <unknown>
#12 0x5e4bf8e39d9a <unknown>
#13 0x5e4bf8e0beb0 <unknown>
#14 0x5e4bf8e60d58 <unknown>
#15 0x5e4bf8e60ef5 <unknown>
#16 0x5e4bf8e72f5e <unknown>
#17 0x77d6baa9caa4 <unknown>
#18 0x77d6bab29c6c <unknown>
2026-06-28 20:31:02,424 - LinkedInFeedScraper - ERROR - LinkedIn cookies not configured (li_at / JSESSIONID)
2026-06-28 20:31:02,433 - LinkedInFeedScraper - INFO - LinkedIn Feed scrape complete. Found 0 new posts.
2026-06-28 20:31:03,225 - httpx - INFO - HTTP Request: GET https://api.github.com/users/abeperl/received_events?per_page=100&page=1 "HTTP/1.1 200 OK"
2026-06-28 20:31:03,881 - httpx - INFO - HTTP Request: GET https://api.github.com/search/repositories?q=paperclip+in%3Aname%2Cdescription%2Creadme+stars%3A%3E150&sort=updated&order=desc&per_page=20 "HTTP/1.1 200 OK"
2026-06-28 20:31:03,885 - GitHubScraper - INFO - Found 242 GitHub repositories for term: paperclip
2026-06-28 20:31:04,770 - httpx - INFO - HTTP Request: GET https://api.github.com/users/abeperl/received_events?per_page=100&page=2 "HTTP/1.1 200 OK"
2026-06-28 20:31:05,126 - GitHubScraper - INFO - GitHub feed: 156 fetched across pages, 1 new events for abeperl
2026-06-28 20:31:05,127 - httpx - INFO - HTTP Request: GET https://api.github.com/search/repositories?q=%22Hermes+Agent%22+in%3Aname%2Cdescription%2Creadme+stars%3A%3E150&sort=updated&order=desc&per_page=20 "HTTP/1.1 200 OK"
2026-06-28 20:31:05,131 - GitHubScraper - INFO - Found 407 GitHub repositories for term: "Hermes Agent"
2026-06-28 20:31:06,131 - httpx - INFO - HTTP Request: GET https://api.github.com/search/repositories?q=multica+in%3Aname%2Cdescription%2Creadme+stars%3A%3E150&sort=updated&order=desc&per_page=20 "HTTP/1.1 200 OK"
2026-06-28 20:31:06,135 - GitHubScraper - INFO - Found 54 GitHub repositories for term: multica
2026-06-28 20:31:06,265 - GitHubScraper - INFO - Total GitHub results found: 0
2026-06-28 20:31:06,266 - __main__ - INFO - Scraper github completed: 0 results
2026-06-28 20:31:06,266 - __main__ - INFO - Scraper github_feed completed: 1 results
2026-06-28 20:31:06,266 - __main__ - INFO - Scraper linkedin_feed completed: 0 results
2026-06-28 20:31:06,266 - __main__ - WARNING - LinkedIn Feed: 30 consecutive failures (0 results)
2026-06-28 20:31:07,035 - social_media_monitor.email.sender - INFO - Email sent successfully at 2026-06-28 20:31:07.035559
2026-06-28 20:31:07,035 - __main__ - INFO - Email sent successfully
2026-06-28 20:31:07,036 - __main__ - INFO - Cache stats: 0 items in memory, 0 files on disk
2026-06-28 20:31:07,036 - __main__ - INFO - Monitor complete. Found 1 total results: 1 new github feed results.
2026-06-28 20:31:07,036 - __main__ - INFO - === Social Media Monitor v2.0 Complete ===
2026-06-28 20:31:07,036 - social_media_monitor.database.connection - INFO - Closing all database connections
2026-06-28 20:31:07,081 - social_media_monitor.database.connection - INFO - Closed 5 database connections
2026-06-28 20:31:07,082 - __main__ - INFO - Closed all database connections
2026-06-28 20:31:07,083 - social_media_monitor.database.connection - INFO - Closing all database connections
2026-06-28 20:31:07,083 - social_media_monitor.database.connection - INFO - Closed 0 database connections
2026-06-28 21:00:02,628 - social_media_monitor.utils.logging_config - INFO - Posts log file: /mnt/data/openclaw-shared/home/shared/Social_Media_Monitor/logs/social_media_monitor_posts_2026-06-28.log
2026-06-28 21:00:02,628 - social_media_monitor.utils.logging_config - INFO - Downloads log file: /mnt/data/openclaw-shared/home/shared/Social_Media_Monitor/logs/social_media_monitor_downloads_2026-06-28.log
2026-06-28 21:00:02,628 - social_media_monitor.utils.logging_config - INFO - Logging configured with level INFO, log file: /mnt/data/openclaw-shared/home/shared/Social_Media_Monitor/logs/social_media_monitor_2026-06-28.log
2026-06-28 21:00:02,628 - __main__ - INFO - === Social Media Monitor v2.0 (Async) Started ===
2026-06-28 21:00:02,654 - social_media_monitor.config.settings - INFO - Configuration loaded from /home/openclaw/Social_Media_Monitor/config.yaml
2026-06-28 21:00:02,654 - __main__ - INFO - Configuration loaded and validated successfully
2026-06-28 21:00:02,657 - social_media_monitor.database.connection - INFO - Database connection pool initialized with 5 connections
2026-06-28 21:00:02,657 - social_media_monitor.database.connection - INFO - add_speaker_filters column already exists
2026-06-28 21:00:02,678 - social_media_monitor.database.connection - INFO - Database setup completed successfully
2026-06-28 21:00:02,679 - __main__ - INFO - Database initialized at: /mnt/data/openclaw-shared/home/shared/Social_Media_Monitor/social_media_monitor.db
2026-06-28 21:00:02,723 - social_media_monitor.scrapers.reddit_scraper - INFO - Reddit API initialized successfully
2026-06-28 21:00:02,724 - RSSFeedScraper - INFO - RSS scraper initialized: feeds=2, html_urls=0 (0 with custom selectors), include_terms=0, exclude_terms=0
2026-06-28 21:00:02,724 - __main__ - INFO - RSS scraper initialized and enabled
2026-06-28 21:00:02,728 - Yiddish24Scraper - INFO - File downloader initialized with 55 terms, prefix 'Y2', and file type organization enabled
2026-06-28 21:00:02,728 - Yiddish24Scraper - INFO - Yiddish24 scraper initialized: data_id='latest', page_limit=100, total_pages=10, debug_html=True, include_terms=55, exclude_terms=0
2026-06-28 21:00:02,728 - __main__ - INFO - Yiddish24 scraper initialized and enabled
2026-06-28 21:00:02,730 - AhBlickLiveScraper - INFO - AhBlickLive scraper initialized: urls=1, max_items=100, include_terms=49, exclude_terms=0
2026-06-28 21:00:02,734 - TorahAnytimeScraper - INFO - File downloader initialized with 53 terms, prefix 'TA', and file type organization enabled
2026-06-28 21:00:02,734 - TorahAnytimeScraper - INFO - TorahAnytime scraper initialized: limit=400, offset=0, project_id=1, include_terms=53, exclude_terms=10
2026-06-28 21:00:02,737 - BatorahScraper - INFO - File downloader initialized with 19 terms, prefix 'BT', and file type organization enabled
2026-06-28 21:00:02,737 - BatorahScraper - INFO - Batorah scraper initialized: api_urls=2, page_size=100, include_terms=19, exclude_terms=1
2026-06-28 21:00:02,737 - KolHalashonScraper - INFO - Kol Halashon scraper initialized: base_url=https://www2.kolhalashon.com, max_shiurim=50, fetch_all_speakers=True, headless=False, exclude_filters=1, include_filters=0
2026-06-28 21:00:02,737 - __main__ - INFO - Ivelt scraper initialized and enabled
2026-06-28 21:00:02,737 - __main__ - INFO - Running 3 scrapers concurrently...
2026-06-28 21:00:02,737 - Yiddish24Scraper - INFO - Starting Yiddish24 scraping (max 10 pages, limit 100 items)
2026-06-28 21:00:02,824 - Yiddish24Scraper - INFO - Fetching Yiddish24 page 1 with data_id='latest', page_limit=100
2026-06-28 21:00:02,829 - IveltScraper - INFO - Loaded 181 include, 0 exclude filters from DB topic_filters
2026-06-28 21:00:02,829 - IveltScraper - INFO - Fetching ivelt active topics from https://www.ivelt.com/forum/search.php?search_id=active_topics...
2026-06-28 21:00:02,830 - RSSFeedScraper - INFO - Fetching RSS feed: https://www.theyeshivaworld.com/feed
2026-06-28 21:00:03,394 - RSSFeedScraper - INFO - Fetching RSS feed: https://www.thegatewaypundit.com/feed/
2026-06-28 21:00:03,898 - RSSFeedScraper - INFO - Total RSS/HTML results: 3
2026-06-28 21:00:07,261 - httpx - INFO - HTTP Request: POST https://www.yiddish24.com/ajax/cat_pagination.php "HTTP/1.1 200 OK"
2026-06-28 21:00:07,469 - Yiddish24Scraper - INFO - Received response with 432998 characters
2026-06-28 21:00:07,472 - Yiddish24Scraper - INFO - Extracted HTML from JSON result field (291970 chars)
2026-06-28 21:00:07,475 - Yiddish24Scraper - INFO - Saved HTML debug file to: /mnt/data/openclaw-shared/home/shared/Social_Media_Monitor/debug/yiddish24_page1_response.html
2026-06-28 21:00:07,576 - Yiddish24Scraper - INFO - Found 100 potential article elements on page 1
2026-06-28 21:00:07,616 - Yiddish24Scraper - INFO - Page 1 extraction: 100 successful, 0 failed out of 100 elements
2026-06-28 21:00:07,616 - Yiddish24Scraper - INFO - Parsed 100 articles from HTML on page 1
2026-06-28 21:00:08,303 - Yiddish24Scraper - INFO - Page 1 summary: 6 new out of 100 total articles
2026-06-28 21:00:08,303 - Yiddish24Scraper - INFO - Page 1: 6/100 new items
2026-06-28 21:00:08,303 - Yiddish24Scraper - INFO - Low new item ratio (6.0%), stopping pagination
2026-06-28 21:00:08,303 - Yiddish24Scraper - INFO - Total Yiddish24 results found: 6
2026-06-28 21:00:14,827 - IveltScraper - INFO - Found 50 active topics on ivelt
2026-06-28 21:00:17,948 - IveltScraper - INFO - Filtering post containing exclude term: '*ื‘ื™ื˜ืข ืžืื›ื˜ ื–ื™ื›ืขืจ ื‘ืœื•ื™ื– ืฆื• ื‘ืืจื™ื›ื˜ืŸ ืงืจืื ื˜ืข ื ื™ื™ืขืก*'
2026-06-28 21:00:41,375 - IveltScraper - INFO - ivelt scrape complete. Checked 6 topics.
2026-06-28 21:00:41,375 - IveltScraper - INFO - All forums scrape complete. Found 39 new posts.
2026-06-28 21:00:41,375 - __main__ - INFO - Scraper rss completed: 3 results
2026-06-28 21:00:41,375 - __main__ - INFO - Scraper yiddish24 completed: 6 results
2026-06-28 21:00:41,375 - __main__ - INFO - Scraper ivelt completed: 39 results
2026-06-28 21:00:42,433 - social_media_monitor.email.sender - INFO - Email sent successfully at 2026-06-28 21:00:42.433413
2026-06-28 21:00:42,433 - __main__ - INFO - Email sent successfully
2026-06-28 21:00:42,434 - __main__ - INFO - Cache stats: 0 items in memory, 0 files on disk
2026-06-28 21:00:42,434 - __main__ - INFO - Monitor complete. Found 48 total results: 3 new rss results, 6 new yiddish24 results, 39 new ivelt results.
2026-06-28 21:00:42,434 - __main__ - INFO - === Social Media Monitor v2.0 Complete ===
2026-06-28 21:00:42,434 - social_media_monitor.database.connection - INFO - Closing all database connections
2026-06-28 21:00:42,511 - social_media_monitor.database.connection - INFO - Closed 5 database connections
2026-06-28 21:00:42,511 - __main__ - INFO - Closed all database connections
2026-06-28 21:00:42,513 - social_media_monitor.database.connection - INFO - Closing all database connections
2026-06-28 21:00:42,513 - social_media_monitor.database.connection - INFO - Closed 0 database connections
2026-06-28 21:30:02,050 - social_media_monitor.utils.logging_config - INFO - Posts log file: /mnt/data/openclaw-shared/home/shared/Social_Media_Monitor/logs/social_media_monitor_posts_2026-06-28.log
2026-06-28 21:30:02,051 - social_media_monitor.utils.logging_config - INFO - Downloads log file: /mnt/data/openclaw-shared/home/shared/Social_Media_Monitor/logs/social_media_monitor_downloads_2026-06-28.log
2026-06-28 21:30:02,051 - social_media_monitor.utils.logging_config - INFO - Logging configured with level INFO, log file: /mnt/data/openclaw-shared/home/shared/Social_Media_Monitor/logs/social_media_monitor_2026-06-28.log
2026-06-28 21:30:02,051 - __main__ - INFO - === Social Media Monitor v2.0 (Async) Started ===
2026-06-28 21:30:02,078 - social_media_monitor.config.settings - INFO - Configuration loaded from /home/openclaw/Social_Media_Monitor/config.yaml
2026-06-28 21:30:02,078 - social_media_monitor.config.settings - WARNING - Twitter monitoring enabled but credentials validation not implemented
2026-06-28 21:30:02,078 - __main__ - INFO - Configuration loaded and validated successfully
2026-06-28 21:30:02,083 - social_media_monitor.database.connection - INFO - Database connection pool initialized with 5 connections
2026-06-28 21:30:02,083 - social_media_monitor.database.connection - INFO - add_speaker_filters column already exists
2026-06-28 21:30:02,095 - social_media_monitor.database.connection - INFO - Database setup completed successfully
2026-06-28 21:30:02,096 - __main__ - INFO - Database initialized at: /mnt/data/openclaw-shared/home/shared/Social_Media_Monitor/social_media_monitor.db
2026-06-28 21:30:02,099 - social_media_monitor.scrapers.reddit_scraper - INFO - Reddit API initialized successfully
2026-06-28 21:30:02,099 - __main__ - INFO - GitHub scraper initialized and enabled
2026-06-28 21:30:02,099 - __main__ - INFO - GitHub feed scraper initialized and enabled
2026-06-28 21:30:02,100 - RSSFeedScraper - INFO - RSS scraper initialized: feeds=2, html_urls=0 (0 with custom selectors), include_terms=0, exclude_terms=0
2026-06-28 21:30:02,100 - __main__ - INFO - LinkedIn Feed scraper initialized and enabled
2026-06-28 21:30:02,102 - Yiddish24Scraper - INFO - File downloader initialized with 55 terms, prefix 'Y2', and file type organization enabled
2026-06-28 21:30:02,102 - Yiddish24Scraper - INFO - Yiddish24 scraper initialized: data_id='latest', page_limit=100, total_pages=10, debug_html=True, include_terms=55, exclude_terms=0
2026-06-28 21:30:02,105 - AhBlickLiveScraper - INFO - AhBlickLive scraper initialized: urls=1, max_items=100, include_terms=49, exclude_terms=0
2026-06-28 21:30:02,110 - TorahAnytimeScraper - INFO - File downloader initialized with 53 terms, prefix 'TA', and file type organization enabled
2026-06-28 21:30:02,110 - TorahAnytimeScraper - INFO - TorahAnytime scraper initialized: limit=400, offset=0, project_id=1, include_terms=53, exclude_terms=10
2026-06-28 21:30:02,112 - BatorahScraper - INFO - File downloader initialized with 19 terms, prefix 'BT', and file type organization enabled
2026-06-28 21:30:02,112 - BatorahScraper - INFO - Batorah scraper initialized: api_urls=2, page_size=100, include_terms=19, exclude_terms=1
2026-06-28 21:30:02,112 - KolHalashonScraper - INFO - Kol Halashon scraper initialized: base_url=https://www2.kolhalashon.com, max_shiurim=50, fetch_all_speakers=True, headless=False, exclude_filters=1, include_filters=0
2026-06-28 21:30:02,112 - __main__ - INFO - Running 3 scrapers concurrently...
2026-06-28 21:30:02,112 - GitHubScraper - INFO - Searching GitHub for terms: ['paperclip', '"Hermes Agent"', 'multica']
2026-06-28 21:30:02,213 - LinkedInFeedScraper - INFO - Fetching LinkedIn feed (25 posts)
2026-06-28 21:30:02,503 - LinkedInFeedScraper - ERROR - LinkedIn cookies not configured (li_at / JSESSIONID)
2026-06-28 21:30:02,510 - LinkedInFeedScraper - INFO - LinkedIn Feed scrape complete. Found 0 new posts.
2026-06-28 21:30:03,234 - httpx - INFO - HTTP Request: GET https://api.github.com/users/abeperl/received_events?per_page=100&page=1 "HTTP/1.1 200 OK"
2026-06-28 21:30:04,042 - httpx - INFO - HTTP Request: GET https://api.github.com/search/repositories?q=paperclip+in%3Aname%2Cdescription%2Creadme+stars%3A%3E150&sort=updated&order=desc&per_page=20 "HTTP/1.1 200 OK"
2026-06-28 21:30:04,048 - GitHubScraper - INFO - Found 242 GitHub repositories for term: paperclip
2026-06-28 21:30:04,923 - httpx - INFO - HTTP Request: GET https://api.github.com/search/repositories?q=%22Hermes+Agent%22+in%3Aname%2Cdescription%2Creadme+stars%3A%3E150&sort=updated&order=desc&per_page=20 "HTTP/1.1 200 OK"
2026-06-28 21:30:04,938 - httpx - INFO - HTTP Request: GET https://api.github.com/users/abeperl/received_events?per_page=100&page=2 "HTTP/1.1 200 OK"
2026-06-28 21:30:05,469 - GitHubScraper - INFO - GitHub feed: 156 fetched across pages, 0 new events for abeperl
2026-06-28 21:30:05,473 - GitHubScraper - INFO - Found 407 GitHub repositories for term: "Hermes Agent"
2026-06-28 21:30:08,845 - httpx - INFO - HTTP Request: GET https://api.github.com/search/repositories?q=multica+in%3Aname%2Cdescription%2Creadme+stars%3A%3E150&sort=updated&order=desc&per_page=20 "HTTP/1.1 200 OK"
2026-06-28 21:30:08,849 - GitHubScraper - INFO - Found 54 GitHub repositories for term: multica
2026-06-28 21:30:09,011 - GitHubScraper - INFO - Total GitHub results found: 0
2026-06-28 21:30:09,011 - __main__ - INFO - Scraper github completed: 0 results
2026-06-28 21:30:09,012 - __main__ - INFO - Scraper github_feed completed: 0 results
2026-06-28 21:30:09,012 - __main__ - INFO - Scraper linkedin_feed completed: 0 results
2026-06-28 21:30:09,015 - __main__ - WARNING - LinkedIn Feed: 31 consecutive failures (0 results)
2026-06-28 21:30:09,015 - __main__ - INFO - No new results found across all enabled sources
2026-06-28 21:30:09,015 - __main__ - INFO - Cache stats: 0 items in memory, 0 files on disk
2026-06-28 21:30:09,015 - __main__ - INFO - Monitor complete. No new results found across all sources.
2026-06-28 21:30:09,015 - __main__ - INFO - === Social Media Monitor v2.0 Complete ===
2026-06-28 21:30:09,015 - social_media_monitor.database.connection - INFO - Closing all database connections
2026-06-28 21:30:09,058 - social_media_monitor.database.connection - INFO - Closed 5 database connections
2026-06-28 21:30:09,058 - __main__ - INFO - Closed all database connections
2026-06-28 21:30:09,060 - social_media_monitor.database.connection - INFO - Closing all database connections
2026-06-28 21:30:09,060 - social_media_monitor.database.connection - INFO - Closed 0 database connections
2026-06-28 22:00:02,193 - social_media_monitor.utils.logging_config - INFO - Posts log file: /mnt/data/openclaw-shared/home/shared/Social_Media_Monitor/logs/social_media_monitor_posts_2026-06-28.log
2026-06-28 22:00:02,193 - social_media_monitor.utils.logging_config - INFO - Downloads log file: /mnt/data/openclaw-shared/home/shared/Social_Media_Monitor/logs/social_media_monitor_downloads_2026-06-28.log
2026-06-28 22:00:02,193 - social_media_monitor.utils.logging_config - INFO - Logging configured with level INFO, log file: /mnt/data/openclaw-shared/home/shared/Social_Media_Monitor/logs/social_media_monitor_2026-06-28.log
2026-06-28 22:00:02,193 - __main__ - INFO - === Social Media Monitor v2.0 (Async) Started ===
2026-06-28 22:00:02,223 - social_media_monitor.config.settings - INFO - Configuration loaded from /home/openclaw/Social_Media_Monitor/config.yaml
2026-06-28 22:00:02,223 - __main__ - INFO - Configuration loaded and validated successfully
2026-06-28 22:00:02,230 - social_media_monitor.database.connection - INFO - Database connection pool initialized with 5 connections
2026-06-28 22:00:02,231 - social_media_monitor.database.connection - INFO - add_speaker_filters column already exists
2026-06-28 22:00:02,251 - social_media_monitor.database.connection - INFO - Database setup completed successfully
2026-06-28 22:00:02,251 - __main__ - INFO - Database initialized at: /mnt/data/openclaw-shared/home/shared/Social_Media_Monitor/social_media_monitor.db
2026-06-28 22:00:02,253 - social_media_monitor.scrapers.reddit_scraper - INFO - Reddit API initialized successfully
2026-06-28 22:00:02,254 - RSSFeedScraper - INFO - RSS scraper initialized: feeds=2, html_urls=0 (0 with custom selectors), include_terms=0, exclude_terms=0
2026-06-28 22:00:02,254 - __main__ - INFO - RSS scraper initialized and enabled
2026-06-28 22:00:02,257 - Yiddish24Scraper - INFO - File downloader initialized with 55 terms, prefix 'Y2', and file type organization enabled
2026-06-28 22:00:02,257 - Yiddish24Scraper - INFO - Yiddish24 scraper initialized: data_id='latest', page_limit=100, total_pages=10, debug_html=True, include_terms=55, exclude_terms=0
2026-06-28 22:00:02,257 - __main__ - INFO - Yiddish24 scraper initialized and enabled
2026-06-28 22:00:02,260 - AhBlickLiveScraper - INFO - AhBlickLive scraper initialized: urls=1, max_items=100, include_terms=49, exclude_terms=0
2026-06-28 22:00:02,264 - TorahAnytimeScraper - INFO - File downloader initialized with 53 terms, prefix 'TA', and file type organization enabled
2026-06-28 22:00:02,264 - TorahAnytimeScraper - INFO - TorahAnytime scraper initialized: limit=400, offset=0, project_id=1, include_terms=53, exclude_terms=10
2026-06-28 22:00:02,266 - BatorahScraper - INFO - File downloader initialized with 19 terms, prefix 'BT', and file type organization enabled
2026-06-28 22:00:02,266 - BatorahScraper - INFO - Batorah scraper initialized: api_urls=2, page_size=100, include_terms=19, exclude_terms=1
2026-06-28 22:00:02,266 - KolHalashonScraper - INFO - Kol Halashon scraper initialized: base_url=https://www2.kolhalashon.com, max_shiurim=50, fetch_all_speakers=True, headless=False, exclude_filters=1, include_filters=0
2026-06-28 22:00:02,266 - __main__ - INFO - Ivelt scraper initialized and enabled
2026-06-28 22:00:02,266 - __main__ - INFO - Running 3 scrapers concurrently...
2026-06-28 22:00:02,267 - Yiddish24Scraper - INFO - Starting Yiddish24 scraping (max 10 pages, limit 100 items)
2026-06-28 22:00:02,354 - Yiddish24Scraper - INFO - Fetching Yiddish24 page 1 with data_id='latest', page_limit=100
2026-06-28 22:00:02,358 - IveltScraper - INFO - Loaded 181 include, 0 exclude filters from DB topic_filters
2026-06-28 22:00:02,359 - IveltScraper - INFO - Fetching ivelt active topics from https://www.ivelt.com/forum/search.php?search_id=active_topics...
2026-06-28 22:00:02,360 - RSSFeedScraper - INFO - Fetching RSS feed: https://www.theyeshivaworld.com/feed
2026-06-28 22:00:02,692 - RSSFeedScraper - INFO - Fetching RSS feed: https://www.thegatewaypundit.com/feed/
2026-06-28 22:00:03,121 - RSSFeedScraper - INFO - Total RSS/HTML results: 3
2026-06-28 22:00:16,028 - IveltScraper - INFO - Found 50 active topics on ivelt
2026-06-28 22:00:33,182 - Yiddish24Scraper - ERROR - Error fetching Yiddish24 page 1:
2026-06-28 22:00:33,195 - Yiddish24Scraper - ERROR - Traceback (most recent call last):
File "/home/openclaw/.local/lib/python3.12/site-packages/httpx/_transports/default.py", line 101, in map_httpcore_exceptions
yield
File "/home/openclaw/.local/lib/python3.12/site-packages/httpx/_transports/default.py", line 394, in handle_async_request
resp = await self._pool.handle_async_request(req)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/openclaw/.local/lib/python3.12/site-packages/httpcore/_async/connection_pool.py", line 256, in handle_async_request
raise exc from None
File "/home/openclaw/.local/lib/python3.12/site-packages/httpcore/_async/connection_pool.py", line 236, in handle_async_request
response = await connection.handle_async_request(
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
pool_request.request
^^^^^^^^^^^^^^^^^^^^
)
^
File "/home/openclaw/.local/lib/python3.12/site-packages/httpcore/_async/connection.py", line 103, in handle_async_request
return await self._connection.handle_async_request(request)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/openclaw/.local/lib/python3.12/site-packages/httpcore/_async/http11.py", line 136, in handle_async_request
raise exc
File "/home/openclaw/.local/lib/python3.12/site-packages/httpcore/_async/http11.py", line 106, in handle_async_request
) = await self._receive_response_headers(**kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/openclaw/.local/lib/python3.12/site-packages/httpcore/_async/http11.py", line 177, in _receive_response_headers
event = await self._receive_event(timeout=timeout)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/openclaw/.local/lib/python3.12/site-packages/httpcore/_async/http11.py", line 217, in _receive_event
data = await self._network_stream.read(
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
self.READ_NUM_BYTES, timeout=timeout
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
)
^
File "/home/openclaw/.local/lib/python3.12/site-packages/httpcore/_backends/anyio.py", line 32, in read
with map_exceptions(exc_map):
~~~~~~~~~~~~~~^^^^^^^^^
File "/home/linuxbrew/.linuxbrew/opt/python@3.14/lib/python3.14/contextlib.py", line 162, in __exit__
self.gen.throw(value)
~~~~~~~~~~~~~~^^^^^^^
File "/home/openclaw/.local/lib/python3.12/site-packages/httpcore/_exceptions.py", line 14, in map_exceptions
raise to_exc(exc) from exc
httpcore.ReadTimeout
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
File "/home/openclaw/Social_Media_Monitor/src/social_media_monitor/scrapers/yiddish24_scraper.py", line 207, in _fetch_page
response = await self.make_request(
^^^^^^^^^^^^^^^^^^^^^^^^
...<5 lines>...
)
^
File "/home/openclaw/Social_Media_Monitor/src/social_media_monitor/scrapers/base_scraper.py", line 231, in make_request
response = await session.post(url, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/openclaw/.local/lib/python3.12/site-packages/httpx/_client.py", line 1859, in post
return await self.request(
^^^^^^^^^^^^^^^^^^^
...<13 lines>...
)
^
File "/home/openclaw/.local/lib/python3.12/site-packages/httpx/_client.py", line 1540, in request
return await self.send(request, auth=auth, follow_redirects=follow_redirects)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/openclaw/.local/lib/python3.12/site-packages/httpx/_client.py", line 1629, in send
response = await self._send_handling_auth(
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
...<4 lines>...
)
^
File "/home/openclaw/.local/lib/python3.12/site-packages/httpx/_client.py", line 1657, in _send_handling_auth
response = await self._send_handling_redirects(
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
...<3 lines>...
)
^
File "/home/openclaw/.local/lib/python3.12/site-packages/httpx/_client.py", line 1694, in _send_handling_redirects
response = await self._send_single_request(request)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/openclaw/.local/lib/python3.12/site-packages/httpx/_client.py", line 1730, in _send_single_request
response = await transport.handle_async_request(request)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/openclaw/.local/lib/python3.12/site-packages/httpx/_transports/default.py", line 393, in handle_async_request
with map_httpcore_exceptions():
~~~~~~~~~~~~~~~~~~~~~~~^^
File "/home/linuxbrew/.linuxbrew/opt/python@3.14/lib/python3.14/contextlib.py", line 162, in __exit__
self.gen.throw(value)
~~~~~~~~~~~~~~^^^^^^^
File "/home/openclaw/.local/lib/python3.12/site-packages/httpx/_transports/default.py", line 118, in map_httpcore_exceptions
raise mapped_exc(message) from exc
httpx.ReadTimeout
2026-06-28 22:00:33,195 - Yiddish24Scraper - INFO - Page 1: 0/0 new items
2026-06-28 22:00:33,196 - Yiddish24Scraper - INFO - No more results found at page 1, stopping pagination
2026-06-28 22:00:33,196 - Yiddish24Scraper - INFO - Total Yiddish24 results found: 0
2026-06-28 22:00:38,190 - IveltScraper - INFO - Filtering post containing exclude term: '*ื‘ื™ื˜ืข ืžืื›ื˜ ื–ื™ื›ืขืจ ื‘ืœื•ื™ื– ืฆื• ื‘ืืจื™ื›ื˜ืŸ ืงืจืื ื˜ืข ื ื™ื™ืขืก*'
2026-06-28 22:00:54,435 - IveltScraper - INFO - ivelt scrape complete. Checked 7 topics.
2026-06-28 22:00:54,436 - IveltScraper - INFO - All forums scrape complete. Found 39 new posts.
2026-06-28 22:00:54,436 - __main__ - INFO - Scraper rss completed: 3 results
2026-06-28 22:00:54,436 - __main__ - INFO - Scraper yiddish24 completed: 0 results
2026-06-28 22:00:54,436 - __main__ - INFO - Scraper ivelt completed: 39 results
2026-06-28 22:00:55,326 - social_media_monitor.email.sender - INFO - Email sent successfully at 2026-06-28 22:00:55.326135
2026-06-28 22:00:55,326 - __main__ - INFO - Email sent successfully
2026-06-28 22:00:55,326 - __main__ - INFO - Cache stats: 0 items in memory, 0 files on disk
2026-06-28 22:00:55,326 - __main__ - INFO - Monitor complete. Found 42 total results: 3 new rss results, 39 new ivelt results.
2026-06-28 22:00:55,327 - __main__ - INFO - === Social Media Monitor v2.0 Complete ===
2026-06-28 22:00:55,327 - social_media_monitor.database.connection - INFO - Closing all database connections
2026-06-28 22:00:55,416 - social_media_monitor.database.connection - INFO - Closed 5 database connections
2026-06-28 22:00:55,416 - __main__ - INFO - Closed all database connections
2026-06-28 22:00:55,418 - social_media_monitor.database.connection - INFO - Closing all database connections
2026-06-28 22:00:55,419 - social_media_monitor.database.connection - INFO - Closed 0 database connections
2026-06-28 22:30:02,560 - social_media_monitor.utils.logging_config - INFO - Posts log file: /mnt/data/openclaw-shared/home/shared/Social_Media_Monitor/logs/social_media_monitor_posts_2026-06-28.log
2026-06-28 22:30:02,564 - social_media_monitor.utils.logging_config - INFO - Downloads log file: /mnt/data/openclaw-shared/home/shared/Social_Media_Monitor/logs/social_media_monitor_downloads_2026-06-28.log
2026-06-28 22:30:02,564 - social_media_monitor.utils.logging_config - INFO - Logging configured with level INFO, log file: /mnt/data/openclaw-shared/home/shared/Social_Media_Monitor/logs/social_media_monitor_2026-06-28.log
2026-06-28 22:30:02,564 - __main__ - INFO - === Social Media Monitor v2.0 (Async) Started ===
2026-06-28 22:30:02,614 - social_media_monitor.config.settings - INFO - Configuration loaded from /home/openclaw/Social_Media_Monitor/config.yaml
2026-06-28 22:30:02,614 - social_media_monitor.config.settings - WARNING - Twitter monitoring enabled but credentials validation not implemented
2026-06-28 22:30:02,614 - __main__ - INFO - Configuration loaded and validated successfully
2026-06-28 22:30:02,624 - social_media_monitor.database.connection - INFO - Database connection pool initialized with 5 connections
2026-06-28 22:30:02,630 - social_media_monitor.database.connection - INFO - add_speaker_filters column already exists
2026-06-28 22:30:02,643 - social_media_monitor.database.connection - INFO - Database setup completed successfully
2026-06-28 22:30:02,643 - __main__ - INFO - Database initialized at: /mnt/data/openclaw-shared/home/shared/Social_Media_Monitor/social_media_monitor.db
2026-06-28 22:30:02,699 - social_media_monitor.scrapers.reddit_scraper - INFO - Reddit API initialized successfully
2026-06-28 22:30:02,699 - __main__ - INFO - GitHub scraper initialized and enabled
2026-06-28 22:30:02,700 - __main__ - INFO - GitHub feed scraper initialized and enabled
2026-06-28 22:30:02,705 - RSSFeedScraper - INFO - RSS scraper initialized: feeds=2, html_urls=0 (0 with custom selectors), include_terms=0, exclude_terms=0
2026-06-28 22:30:02,705 - __main__ - INFO - LinkedIn Feed scraper initialized and enabled
2026-06-28 22:30:02,713 - Yiddish24Scraper - INFO - File downloader initialized with 55 terms, prefix 'Y2', and file type organization enabled
2026-06-28 22:30:02,713 - Yiddish24Scraper - INFO - Yiddish24 scraper initialized: data_id='latest', page_limit=100, total_pages=10, debug_html=True, include_terms=55, exclude_terms=0
2026-06-28 22:30:02,717 - AhBlickLiveScraper - INFO - AhBlickLive scraper initialized: urls=1, max_items=100, include_terms=49, exclude_terms=0
2026-06-28 22:30:02,724 - TorahAnytimeScraper - INFO - File downloader initialized with 53 terms, prefix 'TA', and file type organization enabled
2026-06-28 22:30:02,724 - TorahAnytimeScraper - INFO - TorahAnytime scraper initialized: limit=400, offset=0, project_id=1, include_terms=53, exclude_terms=10
2026-06-28 22:30:02,726 - BatorahScraper - INFO - File downloader initialized with 19 terms, prefix 'BT', and file type organization enabled
2026-06-28 22:30:02,726 - BatorahScraper - INFO - Batorah scraper initialized: api_urls=2, page_size=100, include_terms=19, exclude_terms=1
2026-06-28 22:30:02,726 - KolHalashonScraper - INFO - Kol Halashon scraper initialized: base_url=https://www2.kolhalashon.com, max_shiurim=50, fetch_all_speakers=True, headless=False, exclude_filters=1, include_filters=0
2026-06-28 22:30:02,726 - __main__ - INFO - Running 3 scrapers concurrently...
2026-06-28 22:30:02,727 - GitHubScraper - INFO - Searching GitHub for terms: ['paperclip', '"Hermes Agent"', 'multica']
2026-06-28 22:30:02,873 - LinkedInFeedScraper - INFO - Fetching LinkedIn feed (25 posts)
2026-06-28 22:30:03,268 - LinkedInFeedScraper - ERROR - LinkedIn cookies not configured (li_at / JSESSIONID)
2026-06-28 22:30:03,345 - LinkedInFeedScraper - INFO - LinkedIn Feed scrape complete. Found 0 new posts.
2026-06-28 22:30:04,089 - httpx - INFO - HTTP Request: GET https://api.github.com/users/abeperl/received_events?per_page=100&page=1 "HTTP/1.1 200 OK"
2026-06-28 22:30:05,011 - httpx - INFO - HTTP Request: GET https://api.github.com/search/repositories?q=paperclip+in%3Aname%2Cdescription%2Creadme+stars%3A%3E150&sort=updated&order=desc&per_page=20 "HTTP/1.1 200 OK"
2026-06-28 22:30:05,016 - GitHubScraper - INFO - Found 242 GitHub repositories for term: paperclip
2026-06-28 22:30:05,824 - httpx - INFO - HTTP Request: GET https://api.github.com/users/abeperl/received_events?per_page=100&page=2 "HTTP/1.1 200 OK"
2026-06-28 22:30:06,306 - GitHubScraper - INFO - GitHub feed: 156 fetched across pages, 0 new events for abeperl
2026-06-28 22:30:06,307 - httpx - INFO - HTTP Request: GET https://api.github.com/search/repositories?q=%22Hermes+Agent%22+in%3Aname%2Cdescription%2Creadme+stars%3A%3E150&sort=updated&order=desc&per_page=20 "HTTP/1.1 200 OK"
2026-06-28 22:30:06,309 - GitHubScraper - INFO - Found 407 GitHub repositories for term: "Hermes Agent"
2026-06-28 22:30:07,329 - httpx - INFO - HTTP Request: GET https://api.github.com/search/repositories?q=multica+in%3Aname%2Cdescription%2Creadme+stars%3A%3E150&sort=updated&order=desc&per_page=20 "HTTP/1.1 200 OK"
2026-06-28 22:30:07,333 - GitHubScraper - INFO - Found 54 GitHub repositories for term: multica
2026-06-28 22:30:07,503 - GitHubScraper - INFO - Total GitHub results found: 0
2026-06-28 22:30:07,503 - __main__ - INFO - Scraper github completed: 0 results
2026-06-28 22:30:07,503 - __main__ - INFO - Scraper github_feed completed: 0 results
2026-06-28 22:30:07,503 - __main__ - INFO - Scraper linkedin_feed completed: 0 results
2026-06-28 22:30:07,505 - __main__ - WARNING - LinkedIn Feed: 32 consecutive failures (0 results)
2026-06-28 22:30:07,505 - __main__ - INFO - No new results found across all enabled sources
2026-06-28 22:30:07,506 - __main__ - INFO - Cache stats: 0 items in memory, 0 files on disk
2026-06-28 22:30:07,506 - __main__ - INFO - Monitor complete. No new results found across all sources.
2026-06-28 22:30:07,506 - __main__ - INFO - === Social Media Monitor v2.0 Complete ===
2026-06-28 22:30:07,506 - social_media_monitor.database.connection - INFO - Closing all database connections
2026-06-28 22:30:07,545 - social_media_monitor.database.connection - INFO - Closed 5 database connections
2026-06-28 22:30:07,546 - __main__ - INFO - Closed all database connections
2026-06-28 22:30:07,548 - social_media_monitor.database.connection - INFO - Closing all database connections
2026-06-28 22:30:07,548 - social_media_monitor.database.connection - INFO - Closed 0 database connections
2026-06-28 23:00:01,745 - social_media_monitor.utils.logging_config - INFO - Posts log file: /mnt/data/openclaw-shared/home/shared/Social_Media_Monitor/logs/social_media_monitor_posts_2026-06-28.log
2026-06-28 23:00:01,745 - social_media_monitor.utils.logging_config - INFO - Downloads log file: /mnt/data/openclaw-shared/home/shared/Social_Media_Monitor/logs/social_media_monitor_downloads_2026-06-28.log
2026-06-28 23:00:01,745 - social_media_monitor.utils.logging_config - INFO - Logging configured with level INFO, log file: /mnt/data/openclaw-shared/home/shared/Social_Media_Monitor/logs/social_media_monitor_2026-06-28.log
2026-06-28 23:00:01,746 - __main__ - INFO - === Social Media Monitor v2.0 (Async) Started ===
2026-06-28 23:00:01,775 - social_media_monitor.config.settings - INFO - Configuration loaded from /home/openclaw/Social_Media_Monitor/config.yaml
2026-06-28 23:00:01,775 - __main__ - INFO - Configuration loaded and validated successfully
2026-06-28 23:00:01,777 - social_media_monitor.database.connection - INFO - Database connection pool initialized with 5 connections
2026-06-28 23:00:01,778 - social_media_monitor.database.connection - INFO - add_speaker_filters column already exists
2026-06-28 23:00:01,803 - social_media_monitor.database.connection - INFO - Database setup completed successfully
2026-06-28 23:00:01,803 - __main__ - INFO - Database initialized at: /mnt/data/openclaw-shared/home/shared/Social_Media_Monitor/social_media_monitor.db
2026-06-28 23:00:01,806 - social_media_monitor.scrapers.reddit_scraper - INFO - Reddit API initialized successfully
2026-06-28 23:00:01,806 - RSSFeedScraper - INFO - RSS scraper initialized: feeds=2, html_urls=0 (0 with custom selectors), include_terms=0, exclude_terms=0
2026-06-28 23:00:01,807 - __main__ - INFO - RSS scraper initialized and enabled
2026-06-28 23:00:01,809 - Yiddish24Scraper - INFO - File downloader initialized with 55 terms, prefix 'Y2', and file type organization enabled
2026-06-28 23:00:01,809 - Yiddish24Scraper - INFO - Yiddish24 scraper initialized: data_id='latest', page_limit=100, total_pages=10, debug_html=True, include_terms=55, exclude_terms=0
2026-06-28 23:00:01,809 - __main__ - INFO - Yiddish24 scraper initialized and enabled
2026-06-28 23:00:01,813 - AhBlickLiveScraper - INFO - AhBlickLive scraper initialized: urls=1, max_items=100, include_terms=49, exclude_terms=0
2026-06-28 23:00:01,818 - TorahAnytimeScraper - INFO - File downloader initialized with 53 terms, prefix 'TA', and file type organization enabled
2026-06-28 23:00:01,818 - TorahAnytimeScraper - INFO - TorahAnytime scraper initialized: limit=400, offset=0, project_id=1, include_terms=53, exclude_terms=10
2026-06-28 23:00:01,820 - BatorahScraper - INFO - File downloader initialized with 19 terms, prefix 'BT', and file type organization enabled
2026-06-28 23:00:01,820 - BatorahScraper - INFO - Batorah scraper initialized: api_urls=2, page_size=100, include_terms=19, exclude_terms=1
2026-06-28 23:00:01,820 - KolHalashonScraper - INFO - Kol Halashon scraper initialized: base_url=https://www2.kolhalashon.com, max_shiurim=50, fetch_all_speakers=True, headless=False, exclude_filters=1, include_filters=0
2026-06-28 23:00:01,820 - __main__ - INFO - Ivelt scraper initialized and enabled
2026-06-28 23:00:01,820 - __main__ - INFO - Running 3 scrapers concurrently...
2026-06-28 23:00:01,821 - Yiddish24Scraper - INFO - Starting Yiddish24 scraping (max 10 pages, limit 100 items)
2026-06-28 23:00:01,916 - Yiddish24Scraper - INFO - Fetching Yiddish24 page 1 with data_id='latest', page_limit=100
2026-06-28 23:00:01,926 - IveltScraper - INFO - Loaded 181 include, 0 exclude filters from DB topic_filters
2026-06-28 23:00:01,926 - IveltScraper - INFO - Fetching ivelt active topics from https://www.ivelt.com/forum/search.php?search_id=active_topics...
2026-06-28 23:00:01,928 - RSSFeedScraper - INFO - Fetching RSS feed: https://www.theyeshivaworld.com/feed
2026-06-28 23:00:02,351 - RSSFeedScraper - INFO - Fetching RSS feed: https://www.thegatewaypundit.com/feed/
2026-06-28 23:00:02,899 - RSSFeedScraper - INFO - Total RSS/HTML results: 3
2026-06-28 23:00:06,598 - httpx - INFO - HTTP Request: POST https://www.yiddish24.com/ajax/cat_pagination.php "HTTP/1.1 200 OK"
2026-06-28 23:00:06,825 - Yiddish24Scraper - INFO - Received response with 445326 characters
2026-06-28 23:00:06,828 - Yiddish24Scraper - INFO - Extracted HTML from JSON result field (299444 chars)
2026-06-28 23:00:06,832 - Yiddish24Scraper - INFO - Saved HTML debug file to: /mnt/data/openclaw-shared/home/shared/Social_Media_Monitor/debug/yiddish24_page1_response.html
2026-06-28 23:00:06,927 - Yiddish24Scraper - INFO - Found 100 potential article elements on page 1
2026-06-28 23:00:06,969 - Yiddish24Scraper - INFO - Page 1 extraction: 100 successful, 0 failed out of 100 elements
2026-06-28 23:00:06,970 - Yiddish24Scraper - INFO - Parsed 100 articles from HTML on page 1
2026-06-28 23:00:06,991 - Yiddish24Scraper - INFO - Term 'ืคื•ืŸ ื“ื™ ืžื’ื™ื“'ืก ื˜ื™ืฉืœ' matched in DESCRIPTION: 'ืคื•ืŸ ื“ื™ ืžื’ื™ื“'ืก ื˜ื™ืฉืœ - ืžื™ื™ื ืข ืฉื•ื•ืขืจื™ื’ืงื™ื™ื˜ืŸ ืžืื›ืŸ ืžื™ืš ืืกืืš ืฉื˜ืขืจืงืขืจ'
2026-06-28 23:00:06,991 - Yiddish24Scraper - INFO - Downloading 1 media file(s) for article: ื”ื›ืœ ื‘ื›ืœ
2026-06-28 23:00:07,638 - social_media_monitor.utils.file_downloader - INFO - Starting download from: https://cloudfront.yiddish24.com/__________________________________________788387574.mp3
2026-06-28 23:00:08,785 - social_media_monitor.utils.file_downloader - INFO - Successfully downloaded: Y2___________________________________________788387574.mp3 (6.7 MB)
2026-06-28 23:00:08,787 - social_media_monitor.utils.file_downloader - INFO - Organized audio file to: /mnt/data/openclaw-shared/home/shared/Social_Media_Monitor/downloads/Audio (audio/mpeg)
2026-06-28 23:00:08,802 - social_media_monitor.utils.media_tagger - INFO - Tagged Y2___________________________________________788387574.mp3: title, artist, album, comment, source_url, genre, publisher
2026-06-28 23:00:08,802 - Yiddish24Scraper - INFO - Renamed downloaded file to include match term: Y2___________________________________________788387574_ืคื•ืŸ_ื“ื™_ืžื’ื™ื“_ืก_ื˜ื™ืฉืœ.mp3
2026-06-28 23:00:09,815 - Yiddish24Scraper - INFO - Page 1 summary: 16 new out of 100 total articles
2026-06-28 23:00:09,816 - Yiddish24Scraper - INFO - Page 1: 16/100 new items
2026-06-28 23:00:09,816 - Yiddish24Scraper - INFO - Low new item ratio (16.0%), stopping pagination
2026-06-28 23:00:09,816 - Yiddish24Scraper - INFO - Total Yiddish24 results found: 16
2026-06-28 23:00:14,226 - IveltScraper - INFO - Found 50 active topics on ivelt
2026-06-28 23:00:44,254 - IveltScraper - INFO - Filtering post containing exclude term: '*ื‘ื™ื˜ืข ืžืื›ื˜ ื–ื™ื›ืขืจ ื‘ืœื•ื™ื– ืฆื• ื‘ืืจื™ื›ื˜ืŸ ืงืจืื ื˜ืข ื ื™ื™ืขืก*'
2026-06-28 23:00:48,410 - IveltScraper - INFO - ivelt scrape complete. Checked 6 topics.
2026-06-28 23:00:48,410 - IveltScraper - INFO - All forums scrape complete. Found 48 new posts.
2026-06-28 23:00:48,410 - __main__ - INFO - Scraper rss completed: 3 results
2026-06-28 23:00:48,411 - __main__ - INFO - Scraper yiddish24 completed: 16 results
2026-06-28 23:00:48,411 - __main__ - INFO - Scraper ivelt completed: 48 results
2026-06-28 23:00:49,239 - social_media_monitor.email.sender - INFO - Email sent successfully at 2026-06-28 23:00:49.238975
2026-06-28 23:00:49,239 - __main__ - INFO - Email sent successfully
2026-06-28 23:00:49,239 - __main__ - INFO - Cache stats: 0 items in memory, 0 files on disk
2026-06-28 23:00:49,239 - __main__ - INFO - Monitor complete. Found 67 total results: 3 new rss results, 16 new yiddish24 results, 48 new ivelt results.
2026-06-28 23:00:49,239 - __main__ - INFO - === Social Media Monitor v2.0 Complete ===
2026-06-28 23:00:49,239 - social_media_monitor.database.connection - INFO - Closing all database connections
2026-06-28 23:00:49,370 - social_media_monitor.database.connection - INFO - Closed 5 database connections
2026-06-28 23:00:49,370 - __main__ - INFO - Closed all database connections
2026-06-28 23:00:49,372 - social_media_monitor.database.connection - INFO - Closing all database connections
2026-06-28 23:00:49,372 - social_media_monitor.database.connection - INFO - Closed 0 database connections
2026-06-28 23:30:02,005 - social_media_monitor.utils.logging_config - INFO - Posts log file: /mnt/data/openclaw-shared/home/shared/Social_Media_Monitor/logs/social_media_monitor_posts_2026-06-28.log
2026-06-28 23:30:02,005 - social_media_monitor.utils.logging_config - INFO - Downloads log file: /mnt/data/openclaw-shared/home/shared/Social_Media_Monitor/logs/social_media_monitor_downloads_2026-06-28.log
2026-06-28 23:30:02,005 - social_media_monitor.utils.logging_config - INFO - Logging configured with level INFO, log file: /mnt/data/openclaw-shared/home/shared/Social_Media_Monitor/logs/social_media_monitor_2026-06-28.log
2026-06-28 23:30:02,005 - __main__ - INFO - === Social Media Monitor v2.0 (Async) Started ===
2026-06-28 23:30:02,062 - social_media_monitor.config.settings - INFO - Configuration loaded from /home/openclaw/Social_Media_Monitor/config.yaml
2026-06-28 23:30:02,062 - social_media_monitor.config.settings - WARNING - Twitter monitoring enabled but credentials validation not implemented
2026-06-28 23:30:02,062 - __main__ - INFO - Configuration loaded and validated successfully
2026-06-28 23:30:02,064 - social_media_monitor.database.connection - INFO - Database connection pool initialized with 5 connections
2026-06-28 23:30:02,070 - social_media_monitor.database.connection - INFO - add_speaker_filters column already exists
2026-06-28 23:30:02,084 - social_media_monitor.database.connection - INFO - Database setup completed successfully
2026-06-28 23:30:02,084 - __main__ - INFO - Database initialized at: /mnt/data/openclaw-shared/home/shared/Social_Media_Monitor/social_media_monitor.db
2026-06-28 23:30:02,090 - social_media_monitor.scrapers.reddit_scraper - INFO - Reddit API initialized successfully
2026-06-28 23:30:02,090 - __main__ - INFO - GitHub scraper initialized and enabled
2026-06-28 23:30:02,090 - __main__ - INFO - GitHub feed scraper initialized and enabled
2026-06-28 23:30:02,091 - RSSFeedScraper - INFO - RSS scraper initialized: feeds=2, html_urls=0 (0 with custom selectors), include_terms=0, exclude_terms=0
2026-06-28 23:30:02,092 - __main__ - INFO - LinkedIn Feed scraper initialized and enabled
2026-06-28 23:30:02,098 - Yiddish24Scraper - INFO - File downloader initialized with 55 terms, prefix 'Y2', and file type organization enabled
2026-06-28 23:30:02,098 - Yiddish24Scraper - INFO - Yiddish24 scraper initialized: data_id='latest', page_limit=100, total_pages=10, debug_html=True, include_terms=55, exclude_terms=0
2026-06-28 23:30:02,105 - AhBlickLiveScraper - INFO - AhBlickLive scraper initialized: urls=1, max_items=100, include_terms=49, exclude_terms=0
2026-06-28 23:30:02,116 - TorahAnytimeScraper - INFO - File downloader initialized with 53 terms, prefix 'TA', and file type organization enabled
2026-06-28 23:30:02,116 - TorahAnytimeScraper - INFO - TorahAnytime scraper initialized: limit=400, offset=0, project_id=1, include_terms=53, exclude_terms=10
2026-06-28 23:30:02,121 - BatorahScraper - INFO - File downloader initialized with 19 terms, prefix 'BT', and file type organization enabled
2026-06-28 23:30:02,121 - BatorahScraper - INFO - Batorah scraper initialized: api_urls=2, page_size=100, include_terms=19, exclude_terms=1
2026-06-28 23:30:02,122 - KolHalashonScraper - INFO - Kol Halashon scraper initialized: base_url=https://www2.kolhalashon.com, max_shiurim=50, fetch_all_speakers=True, headless=False, exclude_filters=1, include_filters=0
2026-06-28 23:30:02,122 - __main__ - INFO - Running 3 scrapers concurrently...
2026-06-28 23:30:02,122 - GitHubScraper - INFO - Searching GitHub for terms: ['paperclip', '"Hermes Agent"', 'multica']
2026-06-28 23:30:02,222 - LinkedInFeedScraper - INFO - Fetching LinkedIn feed (25 posts)
2026-06-28 23:30:02,372 - LinkedInFeedScraper - ERROR - LinkedIn cookies not configured (li_at / JSESSIONID)
2026-06-28 23:30:02,381 - LinkedInFeedScraper - INFO - LinkedIn Feed scrape complete. Found 0 new posts.
2026-06-28 23:30:03,072 - httpx - INFO - HTTP Request: GET https://api.github.com/users/abeperl/received_events?per_page=100&page=1 "HTTP/1.1 200 OK"
2026-06-28 23:30:03,903 - httpx - INFO - HTTP Request: GET https://api.github.com/search/repositories?q=paperclip+in%3Aname%2Cdescription%2Creadme+stars%3A%3E150&sort=updated&order=desc&per_page=20 "HTTP/1.1 200 OK"
2026-06-28 23:30:03,910 - GitHubScraper - INFO - Found 242 GitHub repositories for term: paperclip
2026-06-28 23:30:04,799 - httpx - INFO - HTTP Request: GET https://api.github.com/users/abeperl/received_events?per_page=100&page=2 "HTTP/1.1 200 OK"
2026-06-28 23:30:05,292 - GitHubScraper - INFO - GitHub feed: 157 fetched across pages, 1 new events for abeperl
2026-06-28 23:30:05,294 - httpx - INFO - HTTP Request: GET https://api.github.com/search/repositories?q=%22Hermes+Agent%22+in%3Aname%2Cdescription%2Creadme+stars%3A%3E150&sort=updated&order=desc&per_page=20 "HTTP/1.1 200 OK"
2026-06-28 23:30:05,297 - GitHubScraper - INFO - Found 407 GitHub repositories for term: "Hermes Agent"
2026-06-28 23:30:06,323 - httpx - INFO - HTTP Request: GET https://api.github.com/search/repositories?q=multica+in%3Aname%2Cdescription%2Creadme+stars%3A%3E150&sort=updated&order=desc&per_page=20 "HTTP/1.1 200 OK"
2026-06-28 23:30:06,326 - GitHubScraper - INFO - Found 54 GitHub repositories for term: multica
2026-06-28 23:30:06,479 - GitHubScraper - INFO - Total GitHub results found: 0
2026-06-28 23:30:06,479 - __main__ - INFO - Scraper github completed: 0 results
2026-06-28 23:30:06,480 - __main__ - INFO - Scraper github_feed completed: 1 results
2026-06-28 23:30:06,480 - __main__ - INFO - Scraper linkedin_feed completed: 0 results
2026-06-28 23:30:06,481 - __main__ - WARNING - LinkedIn Feed: 33 consecutive failures (0 results)
2026-06-28 23:30:07,158 - social_media_monitor.email.sender - INFO - Email sent successfully at 2026-06-28 23:30:07.158160
2026-06-28 23:30:07,158 - __main__ - INFO - Email sent successfully
2026-06-28 23:30:07,158 - __main__ - INFO - Cache stats: 0 items in memory, 0 files on disk
2026-06-28 23:30:07,158 - __main__ - INFO - Monitor complete. Found 1 total results: 1 new github feed results.
2026-06-28 23:30:07,159 - __main__ - INFO - === Social Media Monitor v2.0 Complete ===
2026-06-28 23:30:07,159 - social_media_monitor.database.connection - INFO - Closing all database connections
2026-06-28 23:30:07,203 - social_media_monitor.database.connection - INFO - Closed 5 database connections
2026-06-28 23:30:07,204 - __main__ - INFO - Closed all database connections
2026-06-28 23:30:07,205 - social_media_monitor.database.connection - INFO - Closing all database connections
2026-06-28 23:30:07,206 - social_media_monitor.database.connection - INFO - Closed 0 database connections