← Logs

social_media_monitor_2026-06-20.log

2026-06-20 20:35:18,989 - social_media_monitor.database.connection - INFO - Closed 0 database connections
2026-06-20 21:00:01,630 - social_media_monitor.utils.logging_config - INFO - Posts log file: /mnt/data/openclaw-shared/home/shared/Social_Media_Monitor/logs/social_media_monitor_posts_2026-06-20.log
2026-06-20 21:00:01,631 - social_media_monitor.utils.logging_config - INFO - Downloads log file: /mnt/data/openclaw-shared/home/shared/Social_Media_Monitor/logs/social_media_monitor_downloads_2026-06-20.log
2026-06-20 21:00:01,631 - social_media_monitor.utils.logging_config - INFO - Logging configured with level INFO, log file: /mnt/data/openclaw-shared/home/shared/Social_Media_Monitor/logs/social_media_monitor_2026-06-20.log
2026-06-20 21:00:01,631 - __main__ - INFO - === Social Media Monitor v2.0 (Async) Started ===
2026-06-20 21:00:01,658 - social_media_monitor.config.settings - INFO - Configuration loaded from /home/openclaw/Social_Media_Monitor/config.yaml
2026-06-20 21:00:01,658 - __main__ - INFO - Configuration loaded and validated successfully
2026-06-20 21:00:01,660 - social_media_monitor.database.connection - INFO - Database connection pool initialized with 5 connections
2026-06-20 21:00:01,661 - social_media_monitor.database.connection - INFO - add_speaker_filters column already exists
2026-06-20 21:00:01,703 - social_media_monitor.database.connection - INFO - Database setup completed successfully
2026-06-20 21:00:01,704 - __main__ - INFO - Database initialized at: /mnt/data/openclaw-shared/home/shared/Social_Media_Monitor/social_media_monitor.db
2026-06-20 21:00:01,791 - social_media_monitor.scrapers.reddit_scraper - INFO - Reddit API initialized successfully
2026-06-20 21:00:01,793 - RSSFeedScraper - INFO - RSS scraper initialized: feeds=2, html_urls=0 (0 with custom selectors), include_terms=0, exclude_terms=0
2026-06-20 21:00:01,793 - __main__ - INFO - RSS scraper initialized and enabled
2026-06-20 21:00:01,800 - Yiddish24Scraper - INFO - File downloader initialized with 55 terms, prefix 'Y2', and file type organization enabled
2026-06-20 21:00:01,800 - Yiddish24Scraper - INFO - Yiddish24 scraper initialized: data_id='latest', page_limit=100, total_pages=10, debug_html=True, include_terms=55, exclude_terms=0
2026-06-20 21:00:01,800 - __main__ - INFO - Yiddish24 scraper initialized and enabled
2026-06-20 21:00:01,805 - AhBlickLiveScraper - INFO - AhBlickLive scraper initialized: urls=1, max_items=100, include_terms=49, exclude_terms=0
2026-06-20 21:00:01,811 - TorahAnytimeScraper - INFO - File downloader initialized with 53 terms, prefix 'TA', and file type organization enabled
2026-06-20 21:00:01,812 - TorahAnytimeScraper - INFO - TorahAnytime scraper initialized: limit=400, offset=0, project_id=1, include_terms=53, exclude_terms=10
2026-06-20 21:00:01,814 - BatorahScraper - INFO - File downloader initialized with 19 terms, prefix 'BT', and file type organization enabled
2026-06-20 21:00:01,814 - BatorahScraper - INFO - Batorah scraper initialized: api_urls=2, page_size=100, include_terms=19, exclude_terms=1
2026-06-20 21:00:01,814 - KolHalashonScraper - INFO - Kol Halashon scraper initialized: base_url=https://www2.kolhalashon.com, max_shiurim=50, fetch_all_speakers=True, headless=False, exclude_filters=1, include_filters=0
2026-06-20 21:00:01,814 - __main__ - INFO - Ivelt scraper initialized and enabled
2026-06-20 21:00:01,814 - __main__ - INFO - Running 3 scrapers concurrently...
2026-06-20 21:00:01,814 - Yiddish24Scraper - INFO - Starting Yiddish24 scraping (max 10 pages, limit 100 items)
2026-06-20 21:00:01,908 - Yiddish24Scraper - INFO - Fetching Yiddish24 page 1 with data_id='latest', page_limit=100
2026-06-20 21:00:01,912 - IveltScraper - INFO - Loaded 181 include, 0 exclude filters from DB topic_filters
2026-06-20 21:00:01,912 - IveltScraper - INFO - Fetching ivelt.com active topics...
2026-06-20 21:00:01,915 - RSSFeedScraper - INFO - Fetching RSS feed: https://www.theyeshivaworld.com/feed
2026-06-20 21:00:02,345 - RSSFeedScraper - INFO - Fetching RSS feed: https://www.thegatewaypundit.com/feed/
2026-06-20 21:00:02,736 - RSSFeedScraper - INFO - Total RSS/HTML results: 3
2026-06-20 21:00:03,122 - httpx - INFO - HTTP Request: POST https://www.yiddish24.com/ajax/cat_pagination.php "HTTP/1.1 200 OK"
2026-06-20 21:00:03,352 - Yiddish24Scraper - INFO - Received response with 455942 characters
2026-06-20 21:00:03,358 - Yiddish24Scraper - INFO - Extracted HTML from JSON result field (299851 chars)
2026-06-20 21:00:03,361 - Yiddish24Scraper - INFO - Saved HTML debug file to: /mnt/data/openclaw-shared/home/shared/Social_Media_Monitor/debug/yiddish24_page1_response.html
2026-06-20 21:00:03,434 - Yiddish24Scraper - INFO - Found 100 potential article elements on page 1
2026-06-20 21:00:03,475 - Yiddish24Scraper - INFO - Page 1 extraction: 100 successful, 0 failed out of 100 elements
2026-06-20 21:00:03,476 - Yiddish24Scraper - INFO - Parsed 100 articles from HTML on page 1
2026-06-20 21:00:04,176 - Yiddish24Scraper - INFO - Page 1 summary: 0 new out of 100 total articles
2026-06-20 21:00:04,177 - Yiddish24Scraper - INFO - Page 1: 0/100 new items
2026-06-20 21:00:04,177 - Yiddish24Scraper - INFO - Low new item ratio (0.0%), stopping pagination
2026-06-20 21:00:04,177 - Yiddish24Scraper - INFO - Total Yiddish24 results found: 0
2026-06-20 21:00:13,695 - IveltScraper - INFO - Found 50 active topics on ivelt.com
2026-06-20 21:00:28,291 - IveltScraper - INFO - Ivelt scrape complete. Checked 4 topics, found 2 new posts.
2026-06-20 21:00:28,291 - __main__ - INFO - Scraper rss completed: 3 results
2026-06-20 21:00:28,292 - __main__ - INFO - Scraper yiddish24 completed: 0 results
2026-06-20 21:00:28,292 - __main__ - INFO - Scraper ivelt completed: 2 results
2026-06-20 21:00:29,161 - social_media_monitor.email.sender - INFO - Email sent successfully at 2026-06-20 21:00:29.161836
2026-06-20 21:00:29,162 - __main__ - INFO - Email sent successfully
2026-06-20 21:00:29,162 - __main__ - INFO - Cache stats: 0 items in memory, 0 files on disk
2026-06-20 21:00:29,162 - __main__ - INFO - Monitor complete. Found 5 total results: 3 new rss results, 2 new ivelt results.
2026-06-20 21:00:29,162 - __main__ - INFO - === Social Media Monitor v2.0 Complete ===
2026-06-20 21:00:29,162 - social_media_monitor.database.connection - INFO - Closing all database connections
2026-06-20 21:00:29,219 - social_media_monitor.database.connection - INFO - Closed 5 database connections
2026-06-20 21:00:29,220 - __main__ - INFO - Closed all database connections
2026-06-20 21:00:29,222 - social_media_monitor.database.connection - INFO - Closing all database connections
2026-06-20 21:00:29,222 - social_media_monitor.database.connection - INFO - Closed 0 database connections
2026-06-20 21:30:02,266 - social_media_monitor.utils.logging_config - INFO - Posts log file: /mnt/data/openclaw-shared/home/shared/Social_Media_Monitor/logs/social_media_monitor_posts_2026-06-20.log
2026-06-20 21:30:02,266 - social_media_monitor.utils.logging_config - INFO - Downloads log file: /mnt/data/openclaw-shared/home/shared/Social_Media_Monitor/logs/social_media_monitor_downloads_2026-06-20.log
2026-06-20 21:30:02,267 - social_media_monitor.utils.logging_config - INFO - Logging configured with level INFO, log file: /mnt/data/openclaw-shared/home/shared/Social_Media_Monitor/logs/social_media_monitor_2026-06-20.log
2026-06-20 21:30:02,267 - __main__ - INFO - === Social Media Monitor v2.0 (Async) Started ===
2026-06-20 21:30:02,294 - social_media_monitor.config.settings - INFO - Configuration loaded from /home/openclaw/Social_Media_Monitor/config.yaml
2026-06-20 21:30:02,294 - social_media_monitor.config.settings - WARNING - Twitter monitoring enabled but credentials validation not implemented
2026-06-20 21:30:02,294 - __main__ - INFO - Configuration loaded and validated successfully
2026-06-20 21:30:02,297 - social_media_monitor.database.connection - INFO - Database connection pool initialized with 5 connections
2026-06-20 21:30:02,297 - social_media_monitor.database.connection - INFO - add_speaker_filters column already exists
2026-06-20 21:30:02,310 - social_media_monitor.database.connection - INFO - Database setup completed successfully
2026-06-20 21:30:02,311 - __main__ - INFO - Database initialized at: /mnt/data/openclaw-shared/home/shared/Social_Media_Monitor/social_media_monitor.db
2026-06-20 21:30:02,312 - social_media_monitor.scrapers.reddit_scraper - INFO - Reddit API initialized successfully
2026-06-20 21:30:02,312 - __main__ - INFO - Reddit scraper initialized and enabled
2026-06-20 21:30:02,312 - __main__ - INFO - GitHub scraper initialized and enabled
2026-06-20 21:30:02,312 - __main__ - INFO - Twitter scraper initialized and enabled
2026-06-20 21:30:02,313 - RSSFeedScraper - INFO - RSS scraper initialized: feeds=2, html_urls=0 (0 with custom selectors), include_terms=0, exclude_terms=0
2026-06-20 21:30:02,313 - __main__ - INFO - LinkedIn scraper initialized and enabled
2026-06-20 21:30:02,313 - __main__ - INFO - LinkedIn Feed scraper initialized and enabled
2026-06-20 21:30:02,315 - Yiddish24Scraper - INFO - File downloader initialized with 55 terms, prefix 'Y2', and file type organization enabled
2026-06-20 21:30:02,316 - Yiddish24Scraper - INFO - Yiddish24 scraper initialized: data_id='latest', page_limit=100, total_pages=10, debug_html=True, include_terms=55, exclude_terms=0
2026-06-20 21:30:02,318 - AhBlickLiveScraper - INFO - AhBlickLive scraper initialized: urls=1, max_items=100, include_terms=49, exclude_terms=0
2026-06-20 21:30:02,323 - TorahAnytimeScraper - INFO - File downloader initialized with 53 terms, prefix 'TA', and file type organization enabled
2026-06-20 21:30:02,323 - TorahAnytimeScraper - INFO - TorahAnytime scraper initialized: limit=400, offset=0, project_id=1, include_terms=53, exclude_terms=10
2026-06-20 21:30:02,325 - BatorahScraper - INFO - File downloader initialized with 19 terms, prefix 'BT', and file type organization enabled
2026-06-20 21:30:02,325 - BatorahScraper - INFO - Batorah scraper initialized: api_urls=2, page_size=100, include_terms=19, exclude_terms=1
2026-06-20 21:30:02,325 - KolHalashonScraper - INFO - Kol Halashon scraper initialized: base_url=https://www2.kolhalashon.com, max_shiurim=50, fetch_all_speakers=True, headless=False, exclude_filters=1, include_filters=0
2026-06-20 21:30:02,326 - __main__ - INFO - Running 5 scrapers concurrently...
2026-06-20 21:30:02,326 - social_media_monitor.scrapers.reddit_scraper - INFO - Reddit search monitoring enabled. Running search with terms: ['paperclip', '"Hermes Agent"', 'multica']
2026-06-20 21:30:02,326 - social_media_monitor.scrapers.reddit_scraper - INFO - Searching Reddit RSS for terms: ['"Hermes Agent"']
2026-06-20 21:30:03,211 - social_media_monitor.scrapers.reddit_scraper - INFO - Reddit RSS: 25 entries for term '"Hermes Agent"'
2026-06-20 21:30:03,259 - social_media_monitor.scrapers.reddit_scraper - INFO - New Reddit post: Claude Cowork 3P Gateway returned no usable models. Add entries under
2026-06-20 21:30:04,527 - social_media_monitor.scrapers.reddit_scraper - INFO - Total Reddit RSS results found: 1
2026-06-20 21:30:04,528 - social_media_monitor.scrapers.reddit_scraper - INFO - Searching 'paperclip' restricted to subreddits: ['PaperClip_AI', 'PaperclipAI']
2026-06-20 21:30:04,528 - social_media_monitor.scrapers.reddit_scraper - INFO - Searching Reddit RSS for terms: ['paperclip']
2026-06-20 21:30:04,528 - social_media_monitor.scrapers.reddit_scraper - INFO - Restricting Reddit search to subreddits: r/PaperClip_AI+PaperclipAI
2026-06-20 21:30:04,677 - social_media_monitor.scrapers.reddit_scraper - INFO - Reddit RSS: 0 entries for term 'paperclip'
2026-06-20 21:30:05,677 - social_media_monitor.scrapers.reddit_scraper - INFO - Total Reddit RSS results found: 0
2026-06-20 21:30:05,677 - social_media_monitor.scrapers.reddit_scraper - INFO - Searching 'multica' restricted to subreddits: ['hermesagent', 'HermesAI']
2026-06-20 21:30:05,677 - social_media_monitor.scrapers.reddit_scraper - INFO - Searching Reddit RSS for terms: ['multica']
2026-06-20 21:30:05,678 - social_media_monitor.scrapers.reddit_scraper - INFO - Restricting Reddit search to subreddits: r/hermesagent+HermesAI
2026-06-20 21:30:05,828 - social_media_monitor.scrapers.reddit_scraper - INFO - Reddit RSS: 0 entries for term 'multica'
2026-06-20 21:30:06,828 - social_media_monitor.scrapers.reddit_scraper - INFO - Total Reddit RSS results found: 0
2026-06-20 21:30:06,829 - social_media_monitor.scrapers.reddit_scraper - INFO - Reddit subreddit feed monitoring enabled.
2026-06-20 21:30:06,829 - social_media_monitor.scrapers.reddit_scraper - INFO - Monitoring feed 'hermes_agent': r/hermesagent+HermesAI+AI_Agents+aiagents+AgentsOfAI+LocalLLaMA+AutoGPT (limit=25)
2026-06-20 21:30:06,829 - social_media_monitor.scrapers.reddit_scraper - INFO - Searching feed 'hermes_agent' for: hermes agent via RSS
2026-06-20 21:30:07,977 - social_media_monitor.scrapers.reddit_scraper - INFO - Searching feed 'hermes_agent' for: hermes AI via RSS
2026-06-20 21:30:09,126 - social_media_monitor.scrapers.reddit_scraper - INFO - Searching feed 'hermes_agent' for: nous research hermes via RSS
2026-06-20 21:30:10,275 - social_media_monitor.scrapers.reddit_scraper - INFO - Searching feed 'hermes_agent' for: multica via RSS
2026-06-20 21:30:11,426 - social_media_monitor.scrapers.reddit_scraper - INFO - Monitoring feed 'paperclip_ai': r/PaperClip_AI+PaperclipAI+pAIperclip (limit=25)
2026-06-20 21:30:11,426 - social_media_monitor.scrapers.reddit_scraper - INFO - Searching feed 'paperclip_ai' for: paperclip ai via RSS
2026-06-20 21:30:12,575 - social_media_monitor.scrapers.reddit_scraper - INFO - Searching feed 'paperclip_ai' for: paperclip agent via RSS
2026-06-20 21:30:13,724 - social_media_monitor.scrapers.reddit_scraper - INFO - Searching feed 'paperclip_ai' for: multica via RSS
2026-06-20 21:30:14,876 - social_media_monitor.scrapers.reddit_scraper - INFO - Monitoring feed 'claude_agents': r/vibecoding (limit=25)
2026-06-20 21:30:14,876 - social_media_monitor.scrapers.reddit_scraper - INFO - Searching feed 'claude_agents' for: multica via RSS
2026-06-20 21:30:16,026 - social_media_monitor.scrapers.reddit_scraper - INFO - Total subreddit feed results: 0
2026-06-20 21:30:16,027 - social_media_monitor.scrapers.reddit_scraper - INFO - Reddit top feed monitoring enabled (t=day).
2026-06-20 21:30:16,027 - social_media_monitor.scrapers.reddit_scraper - INFO - Fetching top/day posts from feed 'hermes_agent': ['hermesagent', 'HermesAI', 'AI_Agents', 'aiagents', 'AgentsOfAI', 'LocalLLaMA', 'AutoGPT']
2026-06-20 21:30:46,480 - social_media_monitor.scrapers.reddit_scraper - WARNING - Could not fetch top RSS for r/hermesagent after retries
2026-06-20 21:31:02,013 - social_media_monitor.scrapers.reddit_scraper - INFO - Reddit top RSS: 0 entries for r/HermesAI
2026-06-20 21:31:35,451 - social_media_monitor.scrapers.reddit_scraper - WARNING - Could not fetch top RSS for r/AI_Agents after retries
2026-06-20 21:32:05,884 - social_media_monitor.scrapers.reddit_scraper - WARNING - Could not fetch top RSS for r/aiagents after retries
2026-06-20 21:32:06,417 - social_media_monitor.scrapers.reddit_scraper - INFO - Reddit top RSS: 14 entries for r/AgentsOfAI
2026-06-20 21:32:06,515 - social_media_monitor.scrapers.reddit_scraper - INFO - New top post from feed 'hermes_agent' r/AgentsOfAI: Multi agent turn taking falls apart the second you add real-time voice
2026-06-20 21:32:40,006 - social_media_monitor.scrapers.reddit_scraper - WARNING - Could not fetch top RSS for r/LocalLLaMA after retries
2026-06-20 21:33:10,457 - social_media_monitor.scrapers.reddit_scraper - WARNING - Could not fetch top RSS for r/AutoGPT after retries
2026-06-20 21:33:10,457 - social_media_monitor.scrapers.reddit_scraper - INFO - Fetching top/day posts from feed 'paperclip_ai': ['PaperClip_AI', 'PaperclipAI', 'pAIperclip']
2026-06-20 21:33:10,701 - social_media_monitor.scrapers.reddit_scraper - INFO - Reddit top RSS: 0 entries for r/PaperClip_AI
2026-06-20 21:33:44,148 - social_media_monitor.scrapers.reddit_scraper - WARNING - Could not fetch top RSS for r/PaperclipAI after retries
2026-06-20 21:33:44,399 - social_media_monitor.scrapers.reddit_scraper - INFO - Reddit top RSS: 0 entries for r/pAIperclip
2026-06-20 21:33:47,400 - social_media_monitor.scrapers.reddit_scraper - INFO - Fetching top/day posts from feed 'claude_agents': ['vibecoding']
2026-06-20 21:34:03,426 - social_media_monitor.scrapers.reddit_scraper - INFO - Reddit top RSS: 25 entries for r/vibecoding
2026-06-20 21:34:03,451 - social_media_monitor.scrapers.reddit_scraper - INFO - New top post from feed 'claude_agents' r/vibecoding: How Tech Companies Treated Developers Before Vibe Coding
2026-06-20 21:34:06,642 - social_media_monitor.scrapers.reddit_scraper - INFO - Total top feed results: 2
2026-06-20 21:34:06,645 - GitHubScraper - INFO - Searching GitHub for terms: ['paperclip', '"Hermes Agent"', 'multica']
2026-06-20 21:34:06,770 - TwitterScraper - INFO - Searching Twitter/X (Google News RSS) for: ['paperclip', '"Hermes Agent"', 'multica']
2026-06-20 21:34:07,193 - TwitterScraper - INFO - Twitter/X RSS: 100 entries for 'paperclip'
2026-06-20 21:34:21,895 - TwitterScraper - INFO - Twitter/X RSS: 100 entries for '"Hermes Agent"'
2026-06-20 21:34:36,541 - TwitterScraper - INFO - Twitter/X RSS: 100 entries for 'multica'
2026-06-20 21:34:50,621 - TwitterScraper - INFO - Twitter scrape complete. Found 0 new results.
2026-06-20 21:34:50,622 - LinkedInScraper - INFO - Searching LinkedIn (Google News RSS) for: ['paperclip', '"Hermes Agent"', 'multica']
2026-06-20 21:34:51,059 - LinkedInScraper - INFO - LinkedIn RSS: 100 entries for 'paperclip'
2026-06-20 21:34:51,117 - LinkedInScraper - INFO - New LinkedIn post: Everyone's collecting AI tools. Almost nobody has a stack. A tool does
2026-06-20 21:34:52,138 - LinkedInScraper - INFO - LinkedIn RSS: 100 entries for '"Hermes Agent"'
2026-06-20 21:34:52,420 - LinkedInScraper - INFO - New LinkedIn post: Teach an agent a workflow once. Have it remember after every rebuild.
2026-06-20 21:34:53,272 - LinkedInScraper - INFO - LinkedIn RSS: 100 entries for 'multica'
2026-06-20 21:34:53,876 - LinkedInScraper - INFO - New LinkedIn post: They say AI won’t replace your job (yet). But someone who knows how to
2026-06-20 21:34:53,979 - LinkedInScraper - INFO - LinkedIn scrape complete. Found 3 new results.
2026-06-20 21:34:53,979 - LinkedInFeedScraper - INFO - Fetching LinkedIn feed (25 posts)
2026-06-20 21:34:54,232 - LinkedInFeedScraper - INFO - Successfully auto-grabbed fresh LinkedIn cookies and User-Agent from running Chrome!
2026-06-20 21:35:01,634 - LinkedInFeedScraper - INFO - LinkedIn Feed scrape complete. Found 0 new posts.
2026-06-20 21:35:02,569 - httpx - INFO - HTTP Request: GET https://api.github.com/search/repositories?q=paperclip+in%3Aname%2Cdescription%2Creadme+stars%3A%3E150&sort=updated&order=desc&per_page=20 "HTTP/1.1 200 OK"
2026-06-20 21:35:02,596 - GitHubScraper - INFO - Found 241 GitHub repositories for term: paperclip
2026-06-20 21:35:03,621 - httpx - INFO - HTTP Request: GET https://api.github.com/search/repositories?q=%22Hermes+Agent%22+in%3Aname%2Cdescription%2Creadme+stars%3A%3E150&sort=updated&order=desc&per_page=20 "HTTP/1.1 200 OK"
2026-06-20 21:35:03,625 - GitHubScraper - INFO - Found 384 GitHub repositories for term: "Hermes Agent"
2026-06-20 21:35:04,721 - httpx - INFO - HTTP Request: GET https://api.github.com/search/repositories?q=multica+in%3Aname%2Cdescription%2Creadme+stars%3A%3E150&sort=updated&order=desc&per_page=20 "HTTP/1.1 200 OK"
2026-06-20 21:35:04,809 - GitHubScraper - INFO - Found 52 GitHub repositories for term: multica
2026-06-20 21:35:04,947 - GitHubScraper - INFO - Total GitHub results found: 0
2026-06-20 21:35:04,948 - __main__ - INFO - Scraper reddit completed: 3 results
2026-06-20 21:35:04,948 - __main__ - INFO - Scraper github completed: 0 results
2026-06-20 21:35:04,948 - __main__ - INFO - Scraper twitter completed: 0 results
2026-06-20 21:35:04,948 - __main__ - INFO - Scraper linkedin completed: 3 results
2026-06-20 21:35:04,948 - __main__ - INFO - Scraper linkedin_feed completed: 0 results
2026-06-20 21:35:05,919 - social_media_monitor.email.sender - INFO - Email sent successfully at 2026-06-20 21:35:05.919487
2026-06-20 21:35:05,919 - __main__ - INFO - Email sent successfully
2026-06-20 21:35:05,920 - __main__ - INFO - Cache stats: 0 items in memory, 0 files on disk
2026-06-20 21:35:05,920 - __main__ - INFO - Monitor complete. Found 6 total results: 3 new reddit results, 3 new linkedin results.
2026-06-20 21:35:05,920 - __main__ - INFO - === Social Media Monitor v2.0 Complete ===
2026-06-20 21:35:05,920 - social_media_monitor.database.connection - INFO - Closing all database connections
2026-06-20 21:35:06,004 - social_media_monitor.database.connection - INFO - Closed 5 database connections
2026-06-20 21:35:06,004 - __main__ - INFO - Closed all database connections
2026-06-20 21:35:06,007 - social_media_monitor.database.connection - INFO - Closing all database connections
2026-06-20 21:35:06,008 - social_media_monitor.database.connection - INFO - Closed 0 database connections
2026-06-20 22:00:02,522 - social_media_monitor.utils.logging_config - INFO - Posts log file: /mnt/data/openclaw-shared/home/shared/Social_Media_Monitor/logs/social_media_monitor_posts_2026-06-20.log
2026-06-20 22:00:02,523 - social_media_monitor.utils.logging_config - INFO - Downloads log file: /mnt/data/openclaw-shared/home/shared/Social_Media_Monitor/logs/social_media_monitor_downloads_2026-06-20.log
2026-06-20 22:00:02,523 - social_media_monitor.utils.logging_config - INFO - Logging configured with level INFO, log file: /mnt/data/openclaw-shared/home/shared/Social_Media_Monitor/logs/social_media_monitor_2026-06-20.log
2026-06-20 22:00:02,523 - __main__ - INFO - === Social Media Monitor v2.0 (Async) Started ===
2026-06-20 22:00:02,549 - social_media_monitor.config.settings - INFO - Configuration loaded from /home/openclaw/Social_Media_Monitor/config.yaml
2026-06-20 22:00:02,550 - __main__ - INFO - Configuration loaded and validated successfully
2026-06-20 22:00:02,552 - social_media_monitor.database.connection - INFO - Database connection pool initialized with 5 connections
2026-06-20 22:00:02,556 - social_media_monitor.database.connection - INFO - add_speaker_filters column already exists
2026-06-20 22:00:02,581 - social_media_monitor.database.connection - INFO - Database setup completed successfully
2026-06-20 22:00:02,581 - __main__ - INFO - Database initialized at: /mnt/data/openclaw-shared/home/shared/Social_Media_Monitor/social_media_monitor.db
2026-06-20 22:00:02,626 - social_media_monitor.scrapers.reddit_scraper - INFO - Reddit API initialized successfully
2026-06-20 22:00:02,627 - RSSFeedScraper - INFO - RSS scraper initialized: feeds=2, html_urls=0 (0 with custom selectors), include_terms=0, exclude_terms=0
2026-06-20 22:00:02,627 - __main__ - INFO - RSS scraper initialized and enabled
2026-06-20 22:00:02,630 - Yiddish24Scraper - INFO - File downloader initialized with 55 terms, prefix 'Y2', and file type organization enabled
2026-06-20 22:00:02,630 - Yiddish24Scraper - INFO - Yiddish24 scraper initialized: data_id='latest', page_limit=100, total_pages=10, debug_html=True, include_terms=55, exclude_terms=0
2026-06-20 22:00:02,630 - __main__ - INFO - Yiddish24 scraper initialized and enabled
2026-06-20 22:00:02,635 - AhBlickLiveScraper - INFO - AhBlickLive scraper initialized: urls=1, max_items=100, include_terms=49, exclude_terms=0
2026-06-20 22:00:02,643 - TorahAnytimeScraper - INFO - File downloader initialized with 53 terms, prefix 'TA', and file type organization enabled
2026-06-20 22:00:02,643 - TorahAnytimeScraper - INFO - TorahAnytime scraper initialized: limit=400, offset=0, project_id=1, include_terms=53, exclude_terms=10
2026-06-20 22:00:02,645 - BatorahScraper - INFO - File downloader initialized with 19 terms, prefix 'BT', and file type organization enabled
2026-06-20 22:00:02,645 - BatorahScraper - INFO - Batorah scraper initialized: api_urls=2, page_size=100, include_terms=19, exclude_terms=1
2026-06-20 22:00:02,645 - KolHalashonScraper - INFO - Kol Halashon scraper initialized: base_url=https://www2.kolhalashon.com, max_shiurim=50, fetch_all_speakers=True, headless=False, exclude_filters=1, include_filters=0
2026-06-20 22:00:02,645 - __main__ - INFO - Ivelt scraper initialized and enabled
2026-06-20 22:00:02,645 - __main__ - INFO - Running 3 scrapers concurrently...
2026-06-20 22:00:02,646 - Yiddish24Scraper - INFO - Starting Yiddish24 scraping (max 10 pages, limit 100 items)
2026-06-20 22:00:02,750 - Yiddish24Scraper - INFO - Fetching Yiddish24 page 1 with data_id='latest', page_limit=100
2026-06-20 22:00:02,756 - IveltScraper - INFO - Loaded 181 include, 0 exclude filters from DB topic_filters
2026-06-20 22:00:02,756 - IveltScraper - INFO - Fetching ivelt.com active topics...
2026-06-20 22:00:02,758 - RSSFeedScraper - INFO - Fetching RSS feed: https://www.theyeshivaworld.com/feed
2026-06-20 22:00:02,956 - RSSFeedScraper - INFO - Fetching RSS feed: https://www.thegatewaypundit.com/feed/
2026-06-20 22:00:03,353 - RSSFeedScraper - INFO - Total RSS/HTML results: 3
2026-06-20 22:00:03,706 - httpx - INFO - HTTP Request: POST https://www.yiddish24.com/ajax/cat_pagination.php "HTTP/1.1 200 OK"
2026-06-20 22:00:03,913 - Yiddish24Scraper - INFO - Received response with 455942 characters
2026-06-20 22:00:03,914 - Yiddish24Scraper - INFO - Extracted HTML from JSON result field (299851 chars)
2026-06-20 22:00:03,915 - Yiddish24Scraper - INFO - Saved HTML debug file to: /mnt/data/openclaw-shared/home/shared/Social_Media_Monitor/debug/yiddish24_page1_response.html
2026-06-20 22:00:03,992 - Yiddish24Scraper - INFO - Found 100 potential article elements on page 1
2026-06-20 22:00:04,035 - Yiddish24Scraper - INFO - Page 1 extraction: 100 successful, 0 failed out of 100 elements
2026-06-20 22:00:04,035 - Yiddish24Scraper - INFO - Parsed 100 articles from HTML on page 1
2026-06-20 22:00:04,660 - Yiddish24Scraper - INFO - Page 1 summary: 0 new out of 100 total articles
2026-06-20 22:00:04,661 - Yiddish24Scraper - INFO - Page 1: 0/100 new items
2026-06-20 22:00:04,661 - Yiddish24Scraper - INFO - Low new item ratio (0.0%), stopping pagination
2026-06-20 22:00:04,661 - Yiddish24Scraper - INFO - Total Yiddish24 results found: 0
2026-06-20 22:00:14,218 - IveltScraper - INFO - Found 50 active topics on ivelt.com
2026-06-20 22:00:28,729 - IveltScraper - INFO - Ivelt scrape complete. Checked 4 topics, found 2 new posts.
2026-06-20 22:00:28,729 - __main__ - INFO - Scraper rss completed: 3 results
2026-06-20 22:00:28,730 - __main__ - INFO - Scraper yiddish24 completed: 0 results
2026-06-20 22:00:28,730 - __main__ - INFO - Scraper ivelt completed: 2 results
2026-06-20 22:00:29,519 - social_media_monitor.email.sender - INFO - Email sent successfully at 2026-06-20 22:00:29.519267
2026-06-20 22:00:29,519 - __main__ - INFO - Email sent successfully
2026-06-20 22:00:29,519 - __main__ - INFO - Cache stats: 0 items in memory, 0 files on disk
2026-06-20 22:00:29,520 - __main__ - INFO - Monitor complete. Found 5 total results: 3 new rss results, 2 new ivelt results.
2026-06-20 22:00:29,520 - __main__ - INFO - === Social Media Monitor v2.0 Complete ===
2026-06-20 22:00:29,520 - social_media_monitor.database.connection - INFO - Closing all database connections
2026-06-20 22:00:29,570 - social_media_monitor.database.connection - INFO - Closed 5 database connections
2026-06-20 22:00:29,571 - __main__ - INFO - Closed all database connections
2026-06-20 22:00:29,572 - social_media_monitor.database.connection - INFO - Closing all database connections
2026-06-20 22:00:29,572 - social_media_monitor.database.connection - INFO - Closed 0 database connections
2026-06-20 22:30:01,787 - social_media_monitor.utils.logging_config - INFO - Posts log file: /mnt/data/openclaw-shared/home/shared/Social_Media_Monitor/logs/social_media_monitor_posts_2026-06-20.log
2026-06-20 22:30:01,787 - social_media_monitor.utils.logging_config - INFO - Downloads log file: /mnt/data/openclaw-shared/home/shared/Social_Media_Monitor/logs/social_media_monitor_downloads_2026-06-20.log
2026-06-20 22:30:01,787 - social_media_monitor.utils.logging_config - INFO - Logging configured with level INFO, log file: /mnt/data/openclaw-shared/home/shared/Social_Media_Monitor/logs/social_media_monitor_2026-06-20.log
2026-06-20 22:30:01,787 - __main__ - INFO - === Social Media Monitor v2.0 (Async) Started ===
2026-06-20 22:30:01,814 - social_media_monitor.config.settings - INFO - Configuration loaded from /home/openclaw/Social_Media_Monitor/config.yaml
2026-06-20 22:30:01,816 - social_media_monitor.config.settings - WARNING - Twitter monitoring enabled but credentials validation not implemented
2026-06-20 22:30:01,816 - __main__ - INFO - Configuration loaded and validated successfully
2026-06-20 22:30:01,818 - social_media_monitor.database.connection - INFO - Database connection pool initialized with 5 connections
2026-06-20 22:30:01,818 - social_media_monitor.database.connection - INFO - add_speaker_filters column already exists
2026-06-20 22:30:01,840 - social_media_monitor.database.connection - INFO - Database setup completed successfully
2026-06-20 22:30:01,840 - __main__ - INFO - Database initialized at: /mnt/data/openclaw-shared/home/shared/Social_Media_Monitor/social_media_monitor.db
2026-06-20 22:30:01,841 - social_media_monitor.scrapers.reddit_scraper - INFO - Reddit API initialized successfully
2026-06-20 22:30:01,841 - __main__ - INFO - Reddit scraper initialized and enabled
2026-06-20 22:30:01,841 - __main__ - INFO - GitHub scraper initialized and enabled
2026-06-20 22:30:01,841 - __main__ - INFO - Twitter scraper initialized and enabled
2026-06-20 22:30:01,842 - RSSFeedScraper - INFO - RSS scraper initialized: feeds=2, html_urls=0 (0 with custom selectors), include_terms=0, exclude_terms=0
2026-06-20 22:30:01,842 - __main__ - INFO - LinkedIn scraper initialized and enabled
2026-06-20 22:30:01,842 - __main__ - INFO - LinkedIn Feed scraper initialized and enabled
2026-06-20 22:30:01,844 - Yiddish24Scraper - INFO - File downloader initialized with 55 terms, prefix 'Y2', and file type organization enabled
2026-06-20 22:30:01,844 - Yiddish24Scraper - INFO - Yiddish24 scraper initialized: data_id='latest', page_limit=100, total_pages=10, debug_html=True, include_terms=55, exclude_terms=0
2026-06-20 22:30:01,847 - AhBlickLiveScraper - INFO - AhBlickLive scraper initialized: urls=1, max_items=100, include_terms=49, exclude_terms=0
2026-06-20 22:30:01,852 - TorahAnytimeScraper - INFO - File downloader initialized with 53 terms, prefix 'TA', and file type organization enabled
2026-06-20 22:30:01,852 - TorahAnytimeScraper - INFO - TorahAnytime scraper initialized: limit=400, offset=0, project_id=1, include_terms=53, exclude_terms=10
2026-06-20 22:30:01,854 - BatorahScraper - INFO - File downloader initialized with 19 terms, prefix 'BT', and file type organization enabled
2026-06-20 22:30:01,854 - BatorahScraper - INFO - Batorah scraper initialized: api_urls=2, page_size=100, include_terms=19, exclude_terms=1
2026-06-20 22:30:01,854 - KolHalashonScraper - INFO - Kol Halashon scraper initialized: base_url=https://www2.kolhalashon.com, max_shiurim=50, fetch_all_speakers=True, headless=False, exclude_filters=1, include_filters=0
2026-06-20 22:30:01,854 - __main__ - INFO - Running 5 scrapers concurrently...
2026-06-20 22:30:01,854 - social_media_monitor.scrapers.reddit_scraper - INFO - Reddit search monitoring enabled. Running search with terms: ['paperclip', '"Hermes Agent"', 'multica']
2026-06-20 22:30:01,854 - social_media_monitor.scrapers.reddit_scraper - INFO - Searching Reddit RSS for terms: ['"Hermes Agent"']
2026-06-20 22:30:02,731 - social_media_monitor.scrapers.reddit_scraper - INFO - Reddit RSS: 25 entries for term '"Hermes Agent"'
2026-06-20 22:30:04,064 - social_media_monitor.scrapers.reddit_scraper - INFO - Total Reddit RSS results found: 0
2026-06-20 22:30:04,064 - social_media_monitor.scrapers.reddit_scraper - INFO - Searching 'paperclip' restricted to subreddits: ['PaperClip_AI', 'PaperclipAI']
2026-06-20 22:30:04,064 - social_media_monitor.scrapers.reddit_scraper - INFO - Searching Reddit RSS for terms: ['paperclip']
2026-06-20 22:30:04,064 - social_media_monitor.scrapers.reddit_scraper - INFO - Restricting Reddit search to subreddits: r/PaperClip_AI+PaperclipAI
2026-06-20 22:30:04,212 - social_media_monitor.scrapers.reddit_scraper - INFO - Reddit RSS: 0 entries for term 'paperclip'
2026-06-20 22:30:05,213 - social_media_monitor.scrapers.reddit_scraper - INFO - Total Reddit RSS results found: 0
2026-06-20 22:30:05,213 - social_media_monitor.scrapers.reddit_scraper - INFO - Searching 'multica' restricted to subreddits: ['hermesagent', 'HermesAI']
2026-06-20 22:30:05,213 - social_media_monitor.scrapers.reddit_scraper - INFO - Searching Reddit RSS for terms: ['multica']
2026-06-20 22:30:05,213 - social_media_monitor.scrapers.reddit_scraper - INFO - Restricting Reddit search to subreddits: r/hermesagent+HermesAI
2026-06-20 22:30:05,355 - social_media_monitor.scrapers.reddit_scraper - INFO - Reddit RSS: 0 entries for term 'multica'
2026-06-20 22:30:06,355 - social_media_monitor.scrapers.reddit_scraper - INFO - Total Reddit RSS results found: 0
2026-06-20 22:30:06,356 - social_media_monitor.scrapers.reddit_scraper - INFO - Reddit subreddit feed monitoring enabled.
2026-06-20 22:30:06,356 - social_media_monitor.scrapers.reddit_scraper - INFO - Monitoring feed 'hermes_agent': r/hermesagent+HermesAI+AI_Agents+aiagents+AgentsOfAI+LocalLLaMA+AutoGPT (limit=25)
2026-06-20 22:30:06,356 - social_media_monitor.scrapers.reddit_scraper - INFO - Searching feed 'hermes_agent' for: hermes agent via RSS
2026-06-20 22:30:07,505 - social_media_monitor.scrapers.reddit_scraper - INFO - Searching feed 'hermes_agent' for: hermes AI via RSS
2026-06-20 22:30:08,648 - social_media_monitor.scrapers.reddit_scraper - INFO - Searching feed 'hermes_agent' for: nous research hermes via RSS
2026-06-20 22:30:09,796 - social_media_monitor.scrapers.reddit_scraper - INFO - Searching feed 'hermes_agent' for: multica via RSS
2026-06-20 22:30:10,942 - social_media_monitor.scrapers.reddit_scraper - INFO - Monitoring feed 'paperclip_ai': r/PaperClip_AI+PaperclipAI+pAIperclip (limit=25)
2026-06-20 22:30:10,942 - social_media_monitor.scrapers.reddit_scraper - INFO - Searching feed 'paperclip_ai' for: paperclip ai via RSS
2026-06-20 22:30:12,091 - social_media_monitor.scrapers.reddit_scraper - INFO - Searching feed 'paperclip_ai' for: paperclip agent via RSS
2026-06-20 22:30:13,240 - social_media_monitor.scrapers.reddit_scraper - INFO - Searching feed 'paperclip_ai' for: multica via RSS
2026-06-20 22:30:14,384 - social_media_monitor.scrapers.reddit_scraper - INFO - Monitoring feed 'claude_agents': r/vibecoding (limit=25)
2026-06-20 22:30:14,385 - social_media_monitor.scrapers.reddit_scraper - INFO - Searching feed 'claude_agents' for: multica via RSS
2026-06-20 22:30:15,534 - social_media_monitor.scrapers.reddit_scraper - INFO - Total subreddit feed results: 0
2026-06-20 22:30:15,534 - social_media_monitor.scrapers.reddit_scraper - INFO - Reddit top feed monitoring enabled (t=day).
2026-06-20 22:30:15,534 - social_media_monitor.scrapers.reddit_scraper - INFO - Fetching top/day posts from feed 'hermes_agent': ['hermesagent', 'HermesAI', 'AI_Agents', 'aiagents', 'AgentsOfAI', 'LocalLLaMA', 'AutoGPT']
2026-06-20 22:30:45,980 - social_media_monitor.scrapers.reddit_scraper - WARNING - Could not fetch top RSS for r/hermesagent after retries
2026-06-20 22:31:01,536 - social_media_monitor.scrapers.reddit_scraper - INFO - Reddit top RSS: 0 entries for r/HermesAI
2026-06-20 22:31:34,976 - social_media_monitor.scrapers.reddit_scraper - WARNING - Could not fetch top RSS for r/AI_Agents after retries
2026-06-20 22:32:05,424 - social_media_monitor.scrapers.reddit_scraper - WARNING - Could not fetch top RSS for r/aiagents after retries
2026-06-20 22:32:05,995 - social_media_monitor.scrapers.reddit_scraper - INFO - Reddit top RSS: 14 entries for r/AgentsOfAI
2026-06-20 22:32:39,565 - social_media_monitor.scrapers.reddit_scraper - WARNING - Could not fetch top RSS for r/LocalLLaMA after retries
2026-06-20 22:32:39,848 - social_media_monitor.scrapers.reddit_scraper - INFO - Reddit top RSS: 3 entries for r/AutoGPT
2026-06-20 22:32:39,858 - social_media_monitor.scrapers.reddit_scraper - INFO - New top post from feed 'hermes_agent' r/AutoGPT: Asked AI to Recreate The Rock 101 Times. It Eventually Turned Him Into
2026-06-20 22:32:39,864 - social_media_monitor.scrapers.reddit_scraper - INFO - New top post from feed 'hermes_agent' r/AutoGPT: Most AI tools I've used are wrappers pretending to be agents — what ac
2026-06-20 22:32:39,873 - social_media_monitor.scrapers.reddit_scraper - INFO - New top post from feed 'hermes_agent' r/AutoGPT: The missing tool for a truly autonomous agent is an inbox it owns
2026-06-20 22:32:42,873 - social_media_monitor.scrapers.reddit_scraper - INFO - Fetching top/day posts from feed 'paperclip_ai': ['PaperClip_AI', 'PaperclipAI', 'pAIperclip']
2026-06-20 22:33:13,317 - social_media_monitor.scrapers.reddit_scraper - WARNING - Could not fetch top RSS for r/PaperClip_AI after retries
2026-06-20 22:33:13,573 - social_media_monitor.scrapers.reddit_scraper - INFO - Reddit top RSS: 0 entries for r/PaperclipAI
2026-06-20 22:33:47,219 - social_media_monitor.scrapers.reddit_scraper - WARNING - Could not fetch top RSS for r/pAIperclip after retries
2026-06-20 22:33:47,220 - social_media_monitor.scrapers.reddit_scraper - INFO - Fetching top/day posts from feed 'claude_agents': ['vibecoding']
2026-06-20 22:34:03,281 - social_media_monitor.scrapers.reddit_scraper - INFO - Reddit top RSS: 25 entries for r/vibecoding
2026-06-20 22:34:06,490 - social_media_monitor.scrapers.reddit_scraper - INFO - Total top feed results: 3
2026-06-20 22:34:06,492 - GitHubScraper - INFO - Searching GitHub for terms: ['paperclip', '"Hermes Agent"', 'multica']
2026-06-20 22:34:06,643 - TwitterScraper - INFO - Searching Twitter/X (Google News RSS) for: ['paperclip', '"Hermes Agent"', 'multica']
2026-06-20 22:34:07,023 - TwitterScraper - INFO - Twitter/X RSS: 100 entries for 'paperclip'
2026-06-20 22:34:20,854 - TwitterScraper - INFO - Twitter/X RSS: 100 entries for '"Hermes Agent"'
2026-06-20 22:34:35,084 - TwitterScraper - INFO - Twitter/X RSS: 98 entries for 'multica'
2026-06-20 22:34:48,372 - TwitterScraper - INFO - Twitter scrape complete. Found 0 new results.
2026-06-20 22:34:48,373 - LinkedInScraper - INFO - Searching LinkedIn (Google News RSS) for: ['paperclip', '"Hermes Agent"', 'multica']
2026-06-20 22:34:48,930 - LinkedInScraper - INFO - LinkedIn RSS: 100 entries for 'paperclip'
2026-06-20 22:34:50,024 - LinkedInScraper - INFO - LinkedIn RSS: 100 entries for '"Hermes Agent"'
2026-06-20 22:34:51,313 - LinkedInScraper - INFO - LinkedIn RSS: 100 entries for 'multica'
2026-06-20 22:34:51,951 - LinkedInScraper - INFO - LinkedIn scrape complete. Found 0 new results.
2026-06-20 22:34:51,951 - LinkedInFeedScraper - INFO - Fetching LinkedIn feed (25 posts)
2026-06-20 22:34:52,323 - LinkedInFeedScraper - INFO - Successfully auto-grabbed fresh LinkedIn cookies and User-Agent from running Chrome!
2026-06-20 22:35:01,185 - LinkedInFeedScraper - INFO - LinkedIn Feed scrape complete. Found 0 new posts.
2026-06-20 22:35:02,084 - httpx - INFO - HTTP Request: GET https://api.github.com/search/repositories?q=paperclip+in%3Aname%2Cdescription%2Creadme+stars%3A%3E150&sort=updated&order=desc&per_page=20 "HTTP/1.1 200 OK"
2026-06-20 22:35:02,176 - GitHubScraper - INFO - Found 241 GitHub repositories for term: paperclip
2026-06-20 22:35:03,194 - httpx - INFO - HTTP Request: GET https://api.github.com/search/repositories?q=%22Hermes+Agent%22+in%3Aname%2Cdescription%2Creadme+stars%3A%3E150&sort=updated&order=desc&per_page=20 "HTTP/1.1 200 OK"
2026-06-20 22:35:03,198 - GitHubScraper - INFO - Found 384 GitHub repositories for term: "Hermes Agent"
2026-06-20 22:35:04,126 - httpx - INFO - HTTP Request: GET https://api.github.com/search/repositories?q=multica+in%3Aname%2Cdescription%2Creadme+stars%3A%3E150&sort=updated&order=desc&per_page=20 "HTTP/1.1 200 OK"
2026-06-20 22:35:04,218 - GitHubScraper - INFO - Found 52 GitHub repositories for term: multica
2026-06-20 22:35:04,402 - GitHubScraper - INFO - Total GitHub results found: 0
2026-06-20 22:35:04,402 - __main__ - INFO - Scraper reddit completed: 3 results
2026-06-20 22:35:04,403 - __main__ - INFO - Scraper github completed: 0 results
2026-06-20 22:35:04,403 - __main__ - INFO - Scraper twitter completed: 0 results
2026-06-20 22:35:04,403 - __main__ - INFO - Scraper linkedin completed: 0 results
2026-06-20 22:35:04,403 - __main__ - INFO - Scraper linkedin_feed completed: 0 results
2026-06-20 22:35:05,209 - social_media_monitor.email.sender - INFO - Email sent successfully at 2026-06-20 22:35:05.209773
2026-06-20 22:35:05,210 - __main__ - INFO - Email sent successfully
2026-06-20 22:35:05,210 - __main__ - INFO - Cache stats: 0 items in memory, 0 files on disk
2026-06-20 22:35:05,210 - __main__ - INFO - Monitor complete. Found 3 total results: 3 new reddit results.
2026-06-20 22:35:05,210 - __main__ - INFO - === Social Media Monitor v2.0 Complete ===
2026-06-20 22:35:05,210 - social_media_monitor.database.connection - INFO - Closing all database connections
2026-06-20 22:35:05,313 - social_media_monitor.database.connection - INFO - Closed 5 database connections
2026-06-20 22:35:05,313 - __main__ - INFO - Closed all database connections
2026-06-20 22:35:05,317 - social_media_monitor.database.connection - INFO - Closing all database connections
2026-06-20 22:35:05,317 - social_media_monitor.database.connection - INFO - Closed 0 database connections
2026-06-20 23:00:01,888 - social_media_monitor.utils.logging_config - INFO - Posts log file: /mnt/data/openclaw-shared/home/shared/Social_Media_Monitor/logs/social_media_monitor_posts_2026-06-20.log
2026-06-20 23:00:01,888 - social_media_monitor.utils.logging_config - INFO - Downloads log file: /mnt/data/openclaw-shared/home/shared/Social_Media_Monitor/logs/social_media_monitor_downloads_2026-06-20.log
2026-06-20 23:00:01,888 - social_media_monitor.utils.logging_config - INFO - Logging configured with level INFO, log file: /mnt/data/openclaw-shared/home/shared/Social_Media_Monitor/logs/social_media_monitor_2026-06-20.log
2026-06-20 23:00:01,888 - __main__ - INFO - === Social Media Monitor v2.0 (Async) Started ===
2026-06-20 23:00:01,914 - social_media_monitor.config.settings - INFO - Configuration loaded from /home/openclaw/Social_Media_Monitor/config.yaml
2026-06-20 23:00:01,914 - __main__ - INFO - Configuration loaded and validated successfully
2026-06-20 23:00:01,916 - social_media_monitor.database.connection - INFO - Database connection pool initialized with 5 connections
2026-06-20 23:00:01,917 - social_media_monitor.database.connection - INFO - add_speaker_filters column already exists
2026-06-20 23:00:01,960 - social_media_monitor.database.connection - INFO - Database setup completed successfully
2026-06-20 23:00:01,960 - __main__ - INFO - Database initialized at: /mnt/data/openclaw-shared/home/shared/Social_Media_Monitor/social_media_monitor.db
2026-06-20 23:00:01,963 - social_media_monitor.scrapers.reddit_scraper - INFO - Reddit API initialized successfully
2026-06-20 23:00:01,967 - RSSFeedScraper - INFO - RSS scraper initialized: feeds=2, html_urls=0 (0 with custom selectors), include_terms=0, exclude_terms=0
2026-06-20 23:00:01,969 - __main__ - INFO - RSS scraper initialized and enabled
2026-06-20 23:00:01,976 - Yiddish24Scraper - INFO - File downloader initialized with 55 terms, prefix 'Y2', and file type organization enabled
2026-06-20 23:00:01,976 - Yiddish24Scraper - INFO - Yiddish24 scraper initialized: data_id='latest', page_limit=100, total_pages=10, debug_html=True, include_terms=55, exclude_terms=0
2026-06-20 23:00:01,976 - __main__ - INFO - Yiddish24 scraper initialized and enabled
2026-06-20 23:00:01,982 - AhBlickLiveScraper - INFO - AhBlickLive scraper initialized: urls=1, max_items=100, include_terms=49, exclude_terms=0
2026-06-20 23:00:01,993 - TorahAnytimeScraper - INFO - File downloader initialized with 53 terms, prefix 'TA', and file type organization enabled
2026-06-20 23:00:01,993 - TorahAnytimeScraper - INFO - TorahAnytime scraper initialized: limit=400, offset=0, project_id=1, include_terms=53, exclude_terms=10
2026-06-20 23:00:01,998 - BatorahScraper - INFO - File downloader initialized with 19 terms, prefix 'BT', and file type organization enabled
2026-06-20 23:00:01,998 - BatorahScraper - INFO - Batorah scraper initialized: api_urls=2, page_size=100, include_terms=19, exclude_terms=1
2026-06-20 23:00:01,998 - KolHalashonScraper - INFO - Kol Halashon scraper initialized: base_url=https://www2.kolhalashon.com, max_shiurim=50, fetch_all_speakers=True, headless=False, exclude_filters=1, include_filters=0
2026-06-20 23:00:01,998 - __main__ - INFO - Ivelt scraper initialized and enabled
2026-06-20 23:00:01,998 - __main__ - INFO - Running 3 scrapers concurrently...
2026-06-20 23:00:01,999 - Yiddish24Scraper - INFO - Starting Yiddish24 scraping (max 10 pages, limit 100 items)
2026-06-20 23:00:02,095 - Yiddish24Scraper - INFO - Fetching Yiddish24 page 1 with data_id='latest', page_limit=100
2026-06-20 23:00:02,100 - IveltScraper - INFO - Loaded 181 include, 0 exclude filters from DB topic_filters
2026-06-20 23:00:02,100 - IveltScraper - INFO - Fetching ivelt.com active topics...
2026-06-20 23:00:02,101 - RSSFeedScraper - INFO - Fetching RSS feed: https://www.theyeshivaworld.com/feed
2026-06-20 23:00:02,333 - RSSFeedScraper - INFO - Fetching RSS feed: https://www.thegatewaypundit.com/feed/
2026-06-20 23:00:02,786 - RSSFeedScraper - INFO - Total RSS/HTML results: 3
2026-06-20 23:00:03,156 - httpx - INFO - HTTP Request: POST https://www.yiddish24.com/ajax/cat_pagination.php "HTTP/1.1 200 OK"
2026-06-20 23:00:03,375 - Yiddish24Scraper - INFO - Received response with 455942 characters
2026-06-20 23:00:03,378 - Yiddish24Scraper - INFO - Extracted HTML from JSON result field (299851 chars)
2026-06-20 23:00:03,384 - Yiddish24Scraper - INFO - Saved HTML debug file to: /mnt/data/openclaw-shared/home/shared/Social_Media_Monitor/debug/yiddish24_page1_response.html
2026-06-20 23:00:03,498 - Yiddish24Scraper - INFO - Found 100 potential article elements on page 1
2026-06-20 23:00:03,539 - Yiddish24Scraper - INFO - Page 1 extraction: 100 successful, 0 failed out of 100 elements
2026-06-20 23:00:03,539 - Yiddish24Scraper - INFO - Parsed 100 articles from HTML on page 1
2026-06-20 23:00:04,261 - Yiddish24Scraper - INFO - Page 1 summary: 0 new out of 100 total articles
2026-06-20 23:00:04,261 - Yiddish24Scraper - INFO - Page 1: 0/100 new items
2026-06-20 23:00:04,261 - Yiddish24Scraper - INFO - Low new item ratio (0.0%), stopping pagination
2026-06-20 23:00:04,261 - Yiddish24Scraper - INFO - Total Yiddish24 results found: 0
2026-06-20 23:00:13,593 - IveltScraper - INFO - Found 50 active topics on ivelt.com
2026-06-20 23:00:27,363 - IveltScraper - INFO - Ivelt scrape complete. Checked 4 topics, found 0 new posts.
2026-06-20 23:00:27,363 - __main__ - INFO - Scraper rss completed: 3 results
2026-06-20 23:00:27,363 - __main__ - INFO - Scraper yiddish24 completed: 0 results
2026-06-20 23:00:27,363 - __main__ - INFO - Scraper ivelt completed: 0 results
2026-06-20 23:00:28,160 - social_media_monitor.email.sender - INFO - Email sent successfully at 2026-06-20 23:00:28.159991
2026-06-20 23:00:28,160 - __main__ - INFO - Email sent successfully
2026-06-20 23:00:28,160 - __main__ - INFO - Cache stats: 0 items in memory, 0 files on disk
2026-06-20 23:00:28,160 - __main__ - INFO - Monitor complete. Found 3 total results: 3 new rss results.
2026-06-20 23:00:28,160 - __main__ - INFO - === Social Media Monitor v2.0 Complete ===
2026-06-20 23:00:28,160 - social_media_monitor.database.connection - INFO - Closing all database connections
2026-06-20 23:00:28,291 - social_media_monitor.database.connection - INFO - Closed 5 database connections
2026-06-20 23:00:28,291 - __main__ - INFO - Closed all database connections
2026-06-20 23:00:28,292 - social_media_monitor.database.connection - INFO - Closing all database connections
2026-06-20 23:00:28,292 - social_media_monitor.database.connection - INFO - Closed 0 database connections
2026-06-20 23:30:02,459 - social_media_monitor.utils.logging_config - INFO - Posts log file: /mnt/data/openclaw-shared/home/shared/Social_Media_Monitor/logs/social_media_monitor_posts_2026-06-20.log
2026-06-20 23:30:02,459 - social_media_monitor.utils.logging_config - INFO - Downloads log file: /mnt/data/openclaw-shared/home/shared/Social_Media_Monitor/logs/social_media_monitor_downloads_2026-06-20.log
2026-06-20 23:30:02,459 - social_media_monitor.utils.logging_config - INFO - Logging configured with level INFO, log file: /mnt/data/openclaw-shared/home/shared/Social_Media_Monitor/logs/social_media_monitor_2026-06-20.log
2026-06-20 23:30:02,459 - __main__ - INFO - === Social Media Monitor v2.0 (Async) Started ===
2026-06-20 23:30:02,486 - social_media_monitor.config.settings - INFO - Configuration loaded from /home/openclaw/Social_Media_Monitor/config.yaml
2026-06-20 23:30:02,486 - social_media_monitor.config.settings - WARNING - Twitter monitoring enabled but credentials validation not implemented
2026-06-20 23:30:02,486 - __main__ - INFO - Configuration loaded and validated successfully
2026-06-20 23:30:02,488 - social_media_monitor.database.connection - INFO - Database connection pool initialized with 5 connections
2026-06-20 23:30:02,489 - social_media_monitor.database.connection - INFO - add_speaker_filters column already exists
2026-06-20 23:30:02,503 - social_media_monitor.database.connection - INFO - Database setup completed successfully
2026-06-20 23:30:02,503 - __main__ - INFO - Database initialized at: /mnt/data/openclaw-shared/home/shared/Social_Media_Monitor/social_media_monitor.db
2026-06-20 23:30:02,558 - social_media_monitor.scrapers.reddit_scraper - INFO - Reddit API initialized successfully
2026-06-20 23:30:02,558 - __main__ - INFO - Reddit scraper initialized and enabled
2026-06-20 23:30:02,559 - __main__ - INFO - GitHub scraper initialized and enabled
2026-06-20 23:30:02,559 - __main__ - INFO - Twitter scraper initialized and enabled
2026-06-20 23:30:02,559 - RSSFeedScraper - INFO - RSS scraper initialized: feeds=2, html_urls=0 (0 with custom selectors), include_terms=0, exclude_terms=0
2026-06-20 23:30:02,560 - __main__ - INFO - LinkedIn scraper initialized and enabled
2026-06-20 23:30:02,560 - __main__ - INFO - LinkedIn Feed scraper initialized and enabled
2026-06-20 23:30:02,562 - Yiddish24Scraper - INFO - File downloader initialized with 55 terms, prefix 'Y2', and file type organization enabled
2026-06-20 23:30:02,563 - Yiddish24Scraper - INFO - Yiddish24 scraper initialized: data_id='latest', page_limit=100, total_pages=10, debug_html=True, include_terms=55, exclude_terms=0
2026-06-20 23:30:02,565 - AhBlickLiveScraper - INFO - AhBlickLive scraper initialized: urls=1, max_items=100, include_terms=49, exclude_terms=0
2026-06-20 23:30:02,569 - TorahAnytimeScraper - INFO - File downloader initialized with 53 terms, prefix 'TA', and file type organization enabled
2026-06-20 23:30:02,570 - TorahAnytimeScraper - INFO - TorahAnytime scraper initialized: limit=400, offset=0, project_id=1, include_terms=53, exclude_terms=10
2026-06-20 23:30:02,572 - BatorahScraper - INFO - File downloader initialized with 19 terms, prefix 'BT', and file type organization enabled
2026-06-20 23:30:02,572 - BatorahScraper - INFO - Batorah scraper initialized: api_urls=2, page_size=100, include_terms=19, exclude_terms=1
2026-06-20 23:30:02,572 - KolHalashonScraper - INFO - Kol Halashon scraper initialized: base_url=https://www2.kolhalashon.com, max_shiurim=50, fetch_all_speakers=True, headless=False, exclude_filters=1, include_filters=0
2026-06-20 23:30:02,572 - __main__ - INFO - Running 5 scrapers concurrently...
2026-06-20 23:30:02,572 - social_media_monitor.scrapers.reddit_scraper - INFO - Reddit search monitoring enabled. Running search with terms: ['paperclip', '"Hermes Agent"', 'multica']
2026-06-20 23:30:02,572 - social_media_monitor.scrapers.reddit_scraper - INFO - Searching Reddit RSS for terms: ['"Hermes Agent"']
2026-06-20 23:30:03,525 - social_media_monitor.scrapers.reddit_scraper - INFO - Reddit RSS: 25 entries for term '"Hermes Agent"'
2026-06-20 23:30:04,809 - social_media_monitor.scrapers.reddit_scraper - INFO - Total Reddit RSS results found: 0
2026-06-20 23:30:04,809 - social_media_monitor.scrapers.reddit_scraper - INFO - Searching 'paperclip' restricted to subreddits: ['PaperClip_AI', 'PaperclipAI']
2026-06-20 23:30:04,809 - social_media_monitor.scrapers.reddit_scraper - INFO - Searching Reddit RSS for terms: ['paperclip']
2026-06-20 23:30:04,809 - social_media_monitor.scrapers.reddit_scraper - INFO - Restricting Reddit search to subreddits: r/PaperClip_AI+PaperclipAI
2026-06-20 23:30:04,950 - social_media_monitor.scrapers.reddit_scraper - INFO - Reddit RSS: 0 entries for term 'paperclip'
2026-06-20 23:30:05,950 - social_media_monitor.scrapers.reddit_scraper - INFO - Total Reddit RSS results found: 0
2026-06-20 23:30:05,950 - social_media_monitor.scrapers.reddit_scraper - INFO - Searching 'multica' restricted to subreddits: ['hermesagent', 'HermesAI']
2026-06-20 23:30:05,950 - social_media_monitor.scrapers.reddit_scraper - INFO - Searching Reddit RSS for terms: ['multica']
2026-06-20 23:30:05,951 - social_media_monitor.scrapers.reddit_scraper - INFO - Restricting Reddit search to subreddits: r/hermesagent+HermesAI
2026-06-20 23:30:06,091 - social_media_monitor.scrapers.reddit_scraper - INFO - Reddit RSS: 0 entries for term 'multica'
2026-06-20 23:30:07,092 - social_media_monitor.scrapers.reddit_scraper - INFO - Total Reddit RSS results found: 0
2026-06-20 23:30:07,092 - social_media_monitor.scrapers.reddit_scraper - INFO - Reddit subreddit feed monitoring enabled.
2026-06-20 23:30:07,092 - social_media_monitor.scrapers.reddit_scraper - INFO - Monitoring feed 'hermes_agent': r/hermesagent+HermesAI+AI_Agents+aiagents+AgentsOfAI+LocalLLaMA+AutoGPT (limit=25)
2026-06-20 23:30:07,092 - social_media_monitor.scrapers.reddit_scraper - INFO - Searching feed 'hermes_agent' for: hermes agent via RSS
2026-06-20 23:30:08,243 - social_media_monitor.scrapers.reddit_scraper - INFO - Searching feed 'hermes_agent' for: hermes AI via RSS
2026-06-20 23:30:09,392 - social_media_monitor.scrapers.reddit_scraper - INFO - Searching feed 'hermes_agent' for: nous research hermes via RSS
2026-06-20 23:30:10,539 - social_media_monitor.scrapers.reddit_scraper - INFO - Searching feed 'hermes_agent' for: multica via RSS
2026-06-20 23:30:11,688 - social_media_monitor.scrapers.reddit_scraper - INFO - Monitoring feed 'paperclip_ai': r/PaperClip_AI+PaperclipAI+pAIperclip (limit=25)
2026-06-20 23:30:11,689 - social_media_monitor.scrapers.reddit_scraper - INFO - Searching feed 'paperclip_ai' for: paperclip ai via RSS
2026-06-20 23:30:12,834 - social_media_monitor.scrapers.reddit_scraper - INFO - Searching feed 'paperclip_ai' for: paperclip agent via RSS
2026-06-20 23:30:13,984 - social_media_monitor.scrapers.reddit_scraper - INFO - Searching feed 'paperclip_ai' for: multica via RSS
2026-06-20 23:30:15,130 - social_media_monitor.scrapers.reddit_scraper - INFO - Monitoring feed 'claude_agents': r/vibecoding (limit=25)
2026-06-20 23:30:15,130 - social_media_monitor.scrapers.reddit_scraper - INFO - Searching feed 'claude_agents' for: multica via RSS
2026-06-20 23:30:16,279 - social_media_monitor.scrapers.reddit_scraper - INFO - Total subreddit feed results: 0
2026-06-20 23:30:16,279 - social_media_monitor.scrapers.reddit_scraper - INFO - Reddit top feed monitoring enabled (t=day).
2026-06-20 23:30:16,280 - social_media_monitor.scrapers.reddit_scraper - INFO - Fetching top/day posts from feed 'hermes_agent': ['hermesagent', 'HermesAI', 'AI_Agents', 'aiagents', 'AgentsOfAI', 'LocalLLaMA', 'AutoGPT']
2026-06-20 23:30:46,720 - social_media_monitor.scrapers.reddit_scraper - WARNING - Could not fetch top RSS for r/hermesagent after retries
2026-06-20 23:31:02,252 - social_media_monitor.scrapers.reddit_scraper - INFO - Reddit top RSS: 0 entries for r/HermesAI
2026-06-20 23:31:35,697 - social_media_monitor.scrapers.reddit_scraper - WARNING - Could not fetch top RSS for r/AI_Agents after retries
2026-06-20 23:32:06,131 - social_media_monitor.scrapers.reddit_scraper - WARNING - Could not fetch top RSS for r/aiagents after retries
2026-06-20 23:32:06,733 - social_media_monitor.scrapers.reddit_scraper - INFO - Reddit top RSS: 14 entries for r/AgentsOfAI
2026-06-20 23:32:40,509 - social_media_monitor.scrapers.reddit_scraper - WARNING - Could not fetch top RSS for r/LocalLLaMA after retries
2026-06-20 23:33:10,957 - social_media_monitor.scrapers.reddit_scraper - WARNING - Could not fetch top RSS for r/AutoGPT after retries
2026-06-20 23:33:10,958 - social_media_monitor.scrapers.reddit_scraper - INFO - Fetching top/day posts from feed 'paperclip_ai': ['PaperClip_AI', 'PaperclipAI', 'pAIperclip']
2026-06-20 23:33:11,206 - social_media_monitor.scrapers.reddit_scraper - INFO - Reddit top RSS: 0 entries for r/PaperClip_AI
2026-06-20 23:33:44,634 - social_media_monitor.scrapers.reddit_scraper - WARNING - Could not fetch top RSS for r/PaperclipAI after retries
2026-06-20 23:33:44,875 - social_media_monitor.scrapers.reddit_scraper - INFO - Reddit top RSS: 0 entries for r/pAIperclip
2026-06-20 23:33:47,875 - social_media_monitor.scrapers.reddit_scraper - INFO - Fetching top/day posts from feed 'claude_agents': ['vibecoding']
2026-06-20 23:34:03,830 - social_media_monitor.scrapers.reddit_scraper - INFO - Reddit top RSS: 25 entries for r/vibecoding
2026-06-20 23:34:04,032 - social_media_monitor.scrapers.reddit_scraper - INFO - New top post from feed 'claude_agents' r/vibecoding: Confused about what to do while building a website!
2026-06-20 23:34:07,033 - social_media_monitor.scrapers.reddit_scraper - INFO - Total top feed results: 1
2026-06-20 23:34:07,035 - GitHubScraper - INFO - Searching GitHub for terms: ['paperclip', '"Hermes Agent"', 'multica']
2026-06-20 23:34:07,165 - TwitterScraper - INFO - Searching Twitter/X (Google News RSS) for: ['paperclip', '"Hermes Agent"', 'multica']
2026-06-20 23:34:07,614 - TwitterScraper - INFO - Twitter/X RSS: 100 entries for 'paperclip'
2026-06-20 23:34:21,308 - TwitterScraper - INFO - Twitter/X RSS: 100 entries for '"Hermes Agent"'
2026-06-20 23:34:34,662 - TwitterScraper - INFO - Twitter/X RSS: 100 entries for 'multica'
2026-06-20 23:34:47,897 - TwitterScraper - INFO - Twitter scrape complete. Found 0 new results.
2026-06-20 23:34:47,898 - LinkedInScraper - INFO - Searching LinkedIn (Google News RSS) for: ['paperclip', '"Hermes Agent"', 'multica']
2026-06-20 23:34:48,474 - LinkedInScraper - INFO - LinkedIn RSS: 100 entries for 'paperclip'
2026-06-20 23:34:49,795 - LinkedInScraper - INFO - LinkedIn RSS: 100 entries for '"Hermes Agent"'
2026-06-20 23:34:50,914 - LinkedInScraper - INFO - LinkedIn RSS: 100 entries for 'multica'
2026-06-20 23:34:51,548 - LinkedInScraper - INFO - LinkedIn scrape complete. Found 0 new results.
2026-06-20 23:34:51,549 - LinkedInFeedScraper - INFO - Fetching LinkedIn feed (25 posts)
2026-06-20 23:34:51,793 - LinkedInFeedScraper - INFO - Successfully auto-grabbed fresh LinkedIn cookies and User-Agent from running Chrome!
2026-06-20 23:35:02,537 - LinkedInFeedScraper - INFO - LinkedIn Feed scrape complete. Found 0 new posts.
2026-06-20 23:35:03,606 - httpx - INFO - HTTP Request: GET https://api.github.com/search/repositories?q=paperclip+in%3Aname%2Cdescription%2Creadme+stars%3A%3E150&sort=updated&order=desc&per_page=20 "HTTP/1.1 200 OK"
2026-06-20 23:35:03,632 - GitHubScraper - INFO - Found 241 GitHub repositories for term: paperclip
2026-06-20 23:35:04,754 - httpx - INFO - HTTP Request: GET https://api.github.com/search/repositories?q=%22Hermes+Agent%22+in%3Aname%2Cdescription%2Creadme+stars%3A%3E150&sort=updated&order=desc&per_page=20 "HTTP/1.1 200 OK"
2026-06-20 23:35:04,758 - GitHubScraper - INFO - Found 384 GitHub repositories for term: "Hermes Agent"
2026-06-20 23:35:05,727 - httpx - INFO - HTTP Request: GET https://api.github.com/search/repositories?q=multica+in%3Aname%2Cdescription%2Creadme+stars%3A%3E150&sort=updated&order=desc&per_page=20 "HTTP/1.1 200 OK"
2026-06-20 23:35:05,730 - GitHubScraper - INFO - Found 52 GitHub repositories for term: multica
2026-06-20 23:35:05,861 - GitHubScraper - INFO - Total GitHub results found: 0
2026-06-20 23:35:05,861 - __main__ - INFO - Scraper reddit completed: 1 results
2026-06-20 23:35:05,861 - __main__ - INFO - Scraper github completed: 0 results
2026-06-20 23:35:05,862 - __main__ - INFO - Scraper twitter completed: 0 results
2026-06-20 23:35:05,862 - __main__ - INFO - Scraper linkedin completed: 0 results
2026-06-20 23:35:05,862 - __main__ - INFO - Scraper linkedin_feed completed: 0 results
2026-06-20 23:35:06,616 - social_media_monitor.email.sender - INFO - Email sent successfully at 2026-06-20 23:35:06.616227
2026-06-20 23:35:06,616 - __main__ - INFO - Email sent successfully
2026-06-20 23:35:06,616 - __main__ - INFO - Cache stats: 0 items in memory, 0 files on disk
2026-06-20 23:35:06,617 - __main__ - INFO - Monitor complete. Found 1 total results: 1 new reddit results.
2026-06-20 23:35:06,617 - __main__ - INFO - === Social Media Monitor v2.0 Complete ===
2026-06-20 23:35:06,617 - social_media_monitor.database.connection - INFO - Closing all database connections
2026-06-20 23:35:06,758 - social_media_monitor.database.connection - INFO - Closed 5 database connections
2026-06-20 23:35:06,758 - __main__ - INFO - Closed all database connections
2026-06-20 23:35:06,763 - social_media_monitor.database.connection - INFO - Closing all database connections
2026-06-20 23:35:06,763 - social_media_monitor.database.connection - INFO - Closed 0 database connections