โ Logs
social_media_monitor_2026-06-25.log
Traceback (most recent call last):
File "/home/openclaw/Social_Media_Monitor/social_media_monitor.py", line 219, in main
_check_linkedin_feed_failure('linkedin_feed' in scrapers, len(results.get('linkedin_feed_results', [])))
^^^^^^^^
NameError: name 'scrapers' is not defined
2026-06-25 20:30:20,294 - social_media_monitor.database.connection - INFO - Closing all database connections
2026-06-25 20:30:20,420 - social_media_monitor.database.connection - INFO - Closed 5 database connections
2026-06-25 20:30:20,420 - __main__ - INFO - Closed all database connections
2026-06-25 20:30:20,422 - social_media_monitor.database.connection - INFO - Closing all database connections
2026-06-25 20:30:20,422 - social_media_monitor.database.connection - INFO - Closed 0 database connections
2026-06-25 21:00:01,949 - social_media_monitor.utils.logging_config - INFO - Posts log file: /mnt/data/openclaw-shared/home/shared/Social_Media_Monitor/logs/social_media_monitor_posts_2026-06-25.log
2026-06-25 21:00:01,949 - social_media_monitor.utils.logging_config - INFO - Downloads log file: /mnt/data/openclaw-shared/home/shared/Social_Media_Monitor/logs/social_media_monitor_downloads_2026-06-25.log
2026-06-25 21:00:01,949 - social_media_monitor.utils.logging_config - INFO - Logging configured with level INFO, log file: /mnt/data/openclaw-shared/home/shared/Social_Media_Monitor/logs/social_media_monitor_2026-06-25.log
2026-06-25 21:00:01,949 - __main__ - INFO - === Social Media Monitor v2.0 (Async) Started ===
2026-06-25 21:00:01,976 - social_media_monitor.config.settings - INFO - Configuration loaded from /home/openclaw/Social_Media_Monitor/config.yaml
2026-06-25 21:00:01,976 - __main__ - INFO - Configuration loaded and validated successfully
2026-06-25 21:00:01,978 - social_media_monitor.database.connection - INFO - Database connection pool initialized with 5 connections
2026-06-25 21:00:01,979 - social_media_monitor.database.connection - INFO - add_speaker_filters column already exists
2026-06-25 21:00:02,001 - social_media_monitor.database.connection - INFO - Database setup completed successfully
2026-06-25 21:00:02,001 - __main__ - INFO - Database initialized at: /mnt/data/openclaw-shared/home/shared/Social_Media_Monitor/social_media_monitor.db
2026-06-25 21:00:02,004 - social_media_monitor.scrapers.reddit_scraper - INFO - Reddit API initialized successfully
2026-06-25 21:00:02,005 - RSSFeedScraper - INFO - RSS scraper initialized: feeds=2, html_urls=0 (0 with custom selectors), include_terms=0, exclude_terms=0
2026-06-25 21:00:02,005 - __main__ - INFO - RSS scraper initialized and enabled
2026-06-25 21:00:02,008 - Yiddish24Scraper - INFO - File downloader initialized with 55 terms, prefix 'Y2', and file type organization enabled
2026-06-25 21:00:02,008 - Yiddish24Scraper - INFO - Yiddish24 scraper initialized: data_id='latest', page_limit=100, total_pages=10, debug_html=True, include_terms=55, exclude_terms=0
2026-06-25 21:00:02,008 - __main__ - INFO - Yiddish24 scraper initialized and enabled
2026-06-25 21:00:02,012 - AhBlickLiveScraper - INFO - AhBlickLive scraper initialized: urls=1, max_items=100, include_terms=49, exclude_terms=0
2026-06-25 21:00:02,018 - TorahAnytimeScraper - INFO - File downloader initialized with 53 terms, prefix 'TA', and file type organization enabled
2026-06-25 21:00:02,018 - TorahAnytimeScraper - INFO - TorahAnytime scraper initialized: limit=400, offset=0, project_id=1, include_terms=53, exclude_terms=10
2026-06-25 21:00:02,020 - BatorahScraper - INFO - File downloader initialized with 19 terms, prefix 'BT', and file type organization enabled
2026-06-25 21:00:02,021 - BatorahScraper - INFO - Batorah scraper initialized: api_urls=2, page_size=100, include_terms=19, exclude_terms=1
2026-06-25 21:00:02,021 - KolHalashonScraper - INFO - Kol Halashon scraper initialized: base_url=https://www2.kolhalashon.com, max_shiurim=50, fetch_all_speakers=True, headless=False, exclude_filters=1, include_filters=0
2026-06-25 21:00:02,021 - __main__ - INFO - Ivelt scraper initialized and enabled
2026-06-25 21:00:02,021 - __main__ - INFO - Running 3 scrapers concurrently...
2026-06-25 21:00:02,021 - Yiddish24Scraper - INFO - Starting Yiddish24 scraping (max 10 pages, limit 100 items)
2026-06-25 21:00:02,117 - Yiddish24Scraper - INFO - Fetching Yiddish24 page 1 with data_id='latest', page_limit=100
2026-06-25 21:00:02,122 - IveltScraper - INFO - Loaded 181 include, 0 exclude filters from DB topic_filters
2026-06-25 21:00:02,122 - IveltScraper - INFO - Fetching ivelt active topics from https://www.ivelt.com/forum/search.php?search_id=active_topics...
2026-06-25 21:00:02,124 - RSSFeedScraper - INFO - Fetching RSS feed: https://www.theyeshivaworld.com/feed
2026-06-25 21:00:02,441 - RSSFeedScraper - INFO - Fetching RSS feed: https://www.thegatewaypundit.com/feed/
2026-06-25 21:00:02,841 - RSSFeedScraper - INFO - Total RSS/HTML results: 4
2026-06-25 21:00:06,160 - httpx - INFO - HTTP Request: POST https://www.yiddish24.com/ajax/cat_pagination.php "HTTP/1.1 200 OK"
2026-06-25 21:00:06,375 - Yiddish24Scraper - INFO - Received response with 450142 characters
2026-06-25 21:00:06,377 - Yiddish24Scraper - INFO - Extracted HTML from JSON result field (299907 chars)
2026-06-25 21:00:06,380 - Yiddish24Scraper - INFO - Saved HTML debug file to: /mnt/data/openclaw-shared/home/shared/Social_Media_Monitor/debug/yiddish24_page1_response.html
2026-06-25 21:00:06,459 - Yiddish24Scraper - INFO - Found 100 potential article elements on page 1
2026-06-25 21:00:06,500 - Yiddish24Scraper - INFO - Page 1 extraction: 100 successful, 0 failed out of 100 elements
2026-06-25 21:00:06,500 - Yiddish24Scraper - INFO - Parsed 100 articles from HTML on page 1
2026-06-25 21:00:06,550 - Yiddish24Scraper - INFO - Term 'ืฉืืื' matched in DESCRIPTION: '"ืคืืจืืขื ืืืื ืืื ืขืชืืงืื" ืืื ืฉืืฉื ืืกืืจ ืืืงืช-ืืืง ืชืฉืค"ื. - "ื ืืืื ืืชืขืืจืจืืช" ืืืจื"ืง ืจืื ืืืืื ืืงืจืืืืืืืฅ ืืืข"ื. - "ืืืืคื ืืืจื" ืืขืืื ืืขื ืืืจื ืืขื ืืื ืืจื"ื ืจ' ืืขืจื ืืืืฆืืื ืข"ื. - "ื ืืืจืฉ" ืืขืืื ืืขื ืืืจื ืืจื...'
2026-06-25 21:00:06,550 - Yiddish24Scraper - INFO - Downloading 1 media file(s) for article: ืคืืจืืขื ืืืื ืืื ืขืชืืงืื
2026-06-25 21:00:07,051 - social_media_monitor.utils.file_downloader - INFO - Starting download from: https://cloudfront.yiddish24.com/fargenigen_565655_134_260625040647.mp3
2026-06-25 21:00:07,534 - social_media_monitor.utils.file_downloader - INFO - Successfully downloaded: Y2_fargenigen_565655_134_260625040647.mp3 (65.4 MB)
2026-06-25 21:00:07,534 - social_media_monitor.utils.file_downloader - INFO - Organized audio file to: /mnt/data/openclaw-shared/home/shared/Social_Media_Monitor/downloads/Audio (audio/mpeg)
2026-06-25 21:00:07,537 - social_media_monitor.utils.media_tagger - INFO - Tagged Y2_fargenigen_565655_134_260625040647.mp3: title, artist, album, comment, source_url, genre, publisher
2026-06-25 21:00:07,537 - Yiddish24Scraper - INFO - Renamed downloaded file to include match term: Y2_fargenigen_565655_134_260625040647_ืฉืืื.mp3
2026-06-25 21:00:08,336 - Yiddish24Scraper - INFO - Page 1 summary: 9 new out of 100 total articles
2026-06-25 21:00:08,337 - Yiddish24Scraper - INFO - Page 1: 9/100 new items
2026-06-25 21:00:08,337 - Yiddish24Scraper - INFO - Low new item ratio (9.0%), stopping pagination
2026-06-25 21:00:08,337 - Yiddish24Scraper - INFO - Total Yiddish24 results found: 9
2026-06-25 21:00:13,913 - IveltScraper - INFO - Found 50 active topics on ivelt
2026-06-25 21:00:17,060 - IveltScraper - INFO - Filtering post containing exclude term: '*ืืืืข ืืืื ืืืืขืจ ืืืืื ืฆื ืืืจืืืื ืงืจืื ืืข ื ืืืขืก*'
2026-06-25 21:00:38,137 - IveltScraper - INFO - ivelt scrape complete. Checked 4 topics.
2026-06-25 21:00:38,137 - IveltScraper - INFO - All forums scrape complete. Found 23 new posts.
2026-06-25 21:00:38,138 - __main__ - INFO - Scraper rss completed: 4 results
2026-06-25 21:00:38,138 - __main__ - INFO - Scraper yiddish24 completed: 9 results
2026-06-25 21:00:38,138 - __main__ - INFO - Scraper ivelt completed: 23 results
2026-06-25 21:00:38,140 - __main__ - CRITICAL - Critical error in main: name 'scrapers' is not defined
Traceback (most recent call last):
File "/home/openclaw/Social_Media_Monitor/social_media_monitor.py", line 219, in main
_check_linkedin_feed_failure('linkedin_feed' in scrapers, len(results.get('linkedin_feed_results', [])))
^^^^^^^^
NameError: name 'scrapers' is not defined
2026-06-25 21:00:38,142 - social_media_monitor.database.connection - INFO - Closing all database connections
2026-06-25 21:00:38,311 - social_media_monitor.database.connection - INFO - Closed 5 database connections
2026-06-25 21:00:38,311 - __main__ - INFO - Closed all database connections
2026-06-25 21:00:38,313 - social_media_monitor.database.connection - INFO - Closing all database connections
2026-06-25 21:00:38,313 - social_media_monitor.database.connection - INFO - Closed 0 database connections
2026-06-25 21:30:01,948 - social_media_monitor.utils.logging_config - INFO - Posts log file: /mnt/data/openclaw-shared/home/shared/Social_Media_Monitor/logs/social_media_monitor_posts_2026-06-25.log
2026-06-25 21:30:01,948 - social_media_monitor.utils.logging_config - INFO - Downloads log file: /mnt/data/openclaw-shared/home/shared/Social_Media_Monitor/logs/social_media_monitor_downloads_2026-06-25.log
2026-06-25 21:30:01,948 - social_media_monitor.utils.logging_config - INFO - Logging configured with level INFO, log file: /mnt/data/openclaw-shared/home/shared/Social_Media_Monitor/logs/social_media_monitor_2026-06-25.log
2026-06-25 21:30:01,948 - __main__ - INFO - === Social Media Monitor v2.0 (Async) Started ===
2026-06-25 21:30:01,975 - social_media_monitor.config.settings - INFO - Configuration loaded from /home/openclaw/Social_Media_Monitor/config.yaml
2026-06-25 21:30:01,976 - social_media_monitor.config.settings - WARNING - Twitter monitoring enabled but credentials validation not implemented
2026-06-25 21:30:01,976 - __main__ - INFO - Configuration loaded and validated successfully
2026-06-25 21:30:01,978 - social_media_monitor.database.connection - INFO - Database connection pool initialized with 5 connections
2026-06-25 21:30:01,978 - social_media_monitor.database.connection - INFO - add_speaker_filters column already exists
2026-06-25 21:30:01,992 - social_media_monitor.database.connection - INFO - Database setup completed successfully
2026-06-25 21:30:01,993 - __main__ - INFO - Database initialized at: /mnt/data/openclaw-shared/home/shared/Social_Media_Monitor/social_media_monitor.db
2026-06-25 21:30:01,995 - social_media_monitor.scrapers.reddit_scraper - INFO - Reddit API initialized successfully
2026-06-25 21:30:01,995 - __main__ - INFO - GitHub scraper initialized and enabled
2026-06-25 21:30:01,995 - __main__ - INFO - GitHub feed scraper initialized and enabled
2026-06-25 21:30:01,996 - RSSFeedScraper - INFO - RSS scraper initialized: feeds=2, html_urls=0 (0 with custom selectors), include_terms=0, exclude_terms=0
2026-06-25 21:30:01,996 - __main__ - INFO - LinkedIn Feed scraper initialized and enabled
2026-06-25 21:30:02,000 - Yiddish24Scraper - INFO - File downloader initialized with 55 terms, prefix 'Y2', and file type organization enabled
2026-06-25 21:30:02,000 - Yiddish24Scraper - INFO - Yiddish24 scraper initialized: data_id='latest', page_limit=100, total_pages=10, debug_html=True, include_terms=55, exclude_terms=0
2026-06-25 21:30:02,004 - AhBlickLiveScraper - INFO - AhBlickLive scraper initialized: urls=1, max_items=100, include_terms=49, exclude_terms=0
2026-06-25 21:30:02,010 - TorahAnytimeScraper - INFO - File downloader initialized with 53 terms, prefix 'TA', and file type organization enabled
2026-06-25 21:30:02,010 - TorahAnytimeScraper - INFO - TorahAnytime scraper initialized: limit=400, offset=0, project_id=1, include_terms=53, exclude_terms=10
2026-06-25 21:30:02,013 - BatorahScraper - INFO - File downloader initialized with 19 terms, prefix 'BT', and file type organization enabled
2026-06-25 21:30:02,013 - BatorahScraper - INFO - Batorah scraper initialized: api_urls=2, page_size=100, include_terms=19, exclude_terms=1
2026-06-25 21:30:02,013 - KolHalashonScraper - INFO - Kol Halashon scraper initialized: base_url=https://www2.kolhalashon.com, max_shiurim=50, fetch_all_speakers=True, headless=False, exclude_filters=1, include_filters=0
2026-06-25 21:30:02,013 - __main__ - INFO - Running 3 scrapers concurrently...
2026-06-25 21:30:02,014 - GitHubScraper - INFO - Searching GitHub for terms: ['paperclip', '"Hermes Agent"', 'multica']
2026-06-25 21:30:02,119 - LinkedInFeedScraper - INFO - Fetching LinkedIn feed (25 posts)
2026-06-25 21:30:02,391 - LinkedInFeedScraper - INFO - Auto-grabbed fresh LinkedIn cookies from running Chrome (fallback).
2026-06-25 21:30:09,414 - LinkedInFeedScraper - INFO - Successfully sorted fetched LinkedIn posts chronologically (most recent first)
2026-06-25 21:30:09,424 - LinkedInFeedScraper - INFO - New LinkedIn feed post: Chevy Karp - ๐ข New Active Listings โ HedgeStone Business Advisors
Looking for your next acquisition? We have som
2026-06-25 21:30:09,436 - LinkedInFeedScraper - INFO - New LinkedIn feed post: Hershy Goldstein - It has definitely been a long and beautiful journeyโฆ ๐๐ป
Follow hershygoldstein to see the journey o
2026-06-25 21:30:09,443 - LinkedInFeedScraper - INFO - New LinkedIn feed post: Alan Shipper - ๐จ We're Hiring: Project Manager (Spring Valley, NY) ๐จ
Culinary Depot is looking for a Project Manag
2026-06-25 21:30:09,450 - LinkedInFeedScraper - INFO - New LinkedIn feed post: Isaac Drezdner - Every customization solves a problem.
Most create a future problem.
A customer needs a unique work
2026-06-25 21:30:09,458 - LinkedInFeedScraper - INFO - New LinkedIn feed post: Nussi Einhorn - Just wrapped up a mini UX audit for the Path for Life mobile app, used daily by thousands of listene
2026-06-25 21:30:09,470 - LinkedInFeedScraper - INFO - New LinkedIn feed post: Eilon Reshef - We've just announced the Gong Revenue Harness. Check out what this means to revenue organizations.
2026-06-25 21:30:09,484 - LinkedInFeedScraper - INFO - New LinkedIn feed post: Ahuva Fischer - Celebrating another great Frum Women in Business event with Temech.
I am so grateful for the oppor
2026-06-25 21:30:09,512 - LinkedInFeedScraper - INFO - New LinkedIn feed post: Mark Ryan - Hi Network!
I'm excited to be recruiting for a Program Manager, Software Delivery with WDTech at Wal
2026-06-25 21:30:09,532 - LinkedInFeedScraper - INFO - New LinkedIn feed post: Isaac Bardos - The bigger your business gets...
the more lonely it can become.
Last week, I spent time talking wi
2026-06-25 21:30:09,547 - LinkedInFeedScraper - INFO - New LinkedIn feed post: Isaac Litchfield - When we first started offering lead abatement, we outsourced the work to a subcontractor.
It was a
2026-06-25 21:30:09,557 - LinkedInFeedScraper - INFO - LinkedIn Feed scrape complete. Found 10 new posts.
2026-06-25 21:30:10,564 - httpx - INFO - HTTP Request: GET https://api.github.com/users/abeperl/received_events?per_page=100&page=1 "HTTP/1.1 200 OK"
2026-06-25 21:30:11,316 - httpx - INFO - HTTP Request: GET https://api.github.com/search/repositories?q=paperclip+in%3Aname%2Cdescription%2Creadme+stars%3A%3E150&sort=updated&order=desc&per_page=20 "HTTP/1.1 200 OK"
2026-06-25 21:30:11,321 - GitHubScraper - INFO - Found 240 GitHub repositories for term: paperclip
2026-06-25 21:30:12,409 - httpx - INFO - HTTP Request: GET https://api.github.com/users/abeperl/received_events?per_page=100&page=2 "HTTP/1.1 200 OK"
2026-06-25 21:30:13,065 - GitHubScraper - INFO - GitHub feed: 199 fetched across pages, 0 new events for abeperl
2026-06-25 21:30:13,067 - httpx - INFO - HTTP Request: GET https://api.github.com/search/repositories?q=%22Hermes+Agent%22+in%3Aname%2Cdescription%2Creadme+stars%3A%3E150&sort=updated&order=desc&per_page=20 "HTTP/1.1 200 OK"
2026-06-25 21:30:13,070 - GitHubScraper - INFO - Found 398 GitHub repositories for term: "Hermes Agent"
2026-06-25 21:30:14,107 - httpx - INFO - HTTP Request: GET https://api.github.com/search/repositories?q=multica+in%3Aname%2Cdescription%2Creadme+stars%3A%3E150&sort=updated&order=desc&per_page=20 "HTTP/1.1 200 OK"
2026-06-25 21:30:14,110 - GitHubScraper - INFO - Found 54 GitHub repositories for term: multica
2026-06-25 21:30:14,239 - GitHubScraper - INFO - Total GitHub results found: 0
2026-06-25 21:30:14,239 - __main__ - INFO - Scraper github completed: 0 results
2026-06-25 21:30:14,240 - __main__ - INFO - Scraper github_feed completed: 0 results
2026-06-25 21:30:14,240 - __main__ - INFO - Scraper linkedin_feed completed: 10 results
2026-06-25 21:30:14,240 - social_media_monitor.sheets.webinar_sheets_writer - INFO - Checking 10 LinkedIn feed posts for webinar/course + Torah mentions...
2026-06-25 21:30:15,485 - social_media_monitor.sheets.webinar_sheets_writer - INFO - Loaded 41 existing URLs from sheet for dedup
2026-06-25 21:30:16,491 - social_media_monitor.sheets.webinar_sheets_writer - INFO - Loaded 19 existing links from Torah Matches sheet for dedup
2026-06-25 21:30:16,492 - social_media_monitor.sheets.webinar_sheets_writer - INFO - Torah match: Alan Shipper - keyword 'ger' - https://www.linkedin.com/feed/update/urn:li:activity:7476014562882826240
2026-06-25 21:30:16,492 - social_media_monitor.sheets.webinar_sheets_writer - INFO - Torah match: Nussi Einhorn - keyword 'ger' - https://www.linkedin.com/feed/update/urn:li:activity:7476012427499888640
2026-06-25 21:30:16,492 - social_media_monitor.sheets.webinar_sheets_writer - INFO - Torah match: Ahuva Fischer - keyword 'frum' - https://www.linkedin.com/feed/update/urn:li:activity:7476016717417082881
2026-06-25 21:30:16,493 - social_media_monitor.sheets.webinar_sheets_writer - INFO - Torah match: Mark Ryan - keyword 'ger' - https://www.linkedin.com/feed/update/urn:li:activity:7476011514685104128
2026-06-25 21:30:16,493 - social_media_monitor.sheets.webinar_sheets_writer - INFO - Torah match: Isaac Bardos - keyword 'ger' - https://www.linkedin.com/feed/update/urn:li:activity:7476016183197220865
2026-06-25 21:30:16,493 - social_media_monitor.sheets.webinar_sheets_writer - INFO - Webinar/course match: Isaac Litchfield - keyword 'lesson' - https://www.linkedin.com/feed/update/urn:li:activity:7476012926726742016
2026-06-25 21:30:17,527 - social_media_monitor.sheets.webinar_sheets_writer - INFO - Appended 1 rows to Google Sheet 'Webinar Recording DMs'
2026-06-25 21:30:19,023 - social_media_monitor.sheets.webinar_sheets_writer - INFO - Appended 5 rows to Google Sheet 'Torah Matches'
2026-06-25 21:30:19,023 - __main__ - INFO - Webinar/Course: appended 6 posts to Google Sheet
2026-06-25 21:30:19,023 - __main__ - CRITICAL - Critical error in main: name 'scrapers' is not defined
Traceback (most recent call last):
File "/home/openclaw/Social_Media_Monitor/social_media_monitor.py", line 219, in main
_check_linkedin_feed_failure('linkedin_feed' in scrapers, len(results.get('linkedin_feed_results', [])))
^^^^^^^^
NameError: name 'scrapers' is not defined
2026-06-25 21:30:19,024 - social_media_monitor.database.connection - INFO - Closing all database connections
2026-06-25 21:30:19,096 - social_media_monitor.database.connection - INFO - Closed 5 database connections
2026-06-25 21:30:19,096 - __main__ - INFO - Closed all database connections
2026-06-25 21:30:19,097 - social_media_monitor.database.connection - INFO - Closing all database connections
2026-06-25 21:30:19,097 - social_media_monitor.database.connection - INFO - Closed 0 database connections
2026-06-25 22:00:02,655 - social_media_monitor.utils.logging_config - INFO - Posts log file: /mnt/data/openclaw-shared/home/shared/Social_Media_Monitor/logs/social_media_monitor_posts_2026-06-25.log
2026-06-25 22:00:02,655 - social_media_monitor.utils.logging_config - INFO - Downloads log file: /mnt/data/openclaw-shared/home/shared/Social_Media_Monitor/logs/social_media_monitor_downloads_2026-06-25.log
2026-06-25 22:00:02,655 - social_media_monitor.utils.logging_config - INFO - Logging configured with level INFO, log file: /mnt/data/openclaw-shared/home/shared/Social_Media_Monitor/logs/social_media_monitor_2026-06-25.log
2026-06-25 22:00:02,655 - __main__ - INFO - === Social Media Monitor v2.0 (Async) Started ===
2026-06-25 22:00:02,680 - social_media_monitor.config.settings - INFO - Configuration loaded from /home/openclaw/Social_Media_Monitor/config.yaml
2026-06-25 22:00:02,680 - __main__ - INFO - Configuration loaded and validated successfully
2026-06-25 22:00:02,682 - social_media_monitor.database.connection - INFO - Database connection pool initialized with 5 connections
2026-06-25 22:00:02,683 - social_media_monitor.database.connection - INFO - add_speaker_filters column already exists
2026-06-25 22:00:02,703 - social_media_monitor.database.connection - INFO - Database setup completed successfully
2026-06-25 22:00:02,703 - __main__ - INFO - Database initialized at: /mnt/data/openclaw-shared/home/shared/Social_Media_Monitor/social_media_monitor.db
2026-06-25 22:00:02,750 - social_media_monitor.scrapers.reddit_scraper - INFO - Reddit API initialized successfully
2026-06-25 22:00:02,751 - RSSFeedScraper - INFO - RSS scraper initialized: feeds=2, html_urls=0 (0 with custom selectors), include_terms=0, exclude_terms=0
2026-06-25 22:00:02,751 - __main__ - INFO - RSS scraper initialized and enabled
2026-06-25 22:00:02,754 - Yiddish24Scraper - INFO - File downloader initialized with 55 terms, prefix 'Y2', and file type organization enabled
2026-06-25 22:00:02,754 - Yiddish24Scraper - INFO - Yiddish24 scraper initialized: data_id='latest', page_limit=100, total_pages=10, debug_html=True, include_terms=55, exclude_terms=0
2026-06-25 22:00:02,754 - __main__ - INFO - Yiddish24 scraper initialized and enabled
2026-06-25 22:00:02,756 - AhBlickLiveScraper - INFO - AhBlickLive scraper initialized: urls=1, max_items=100, include_terms=49, exclude_terms=0
2026-06-25 22:00:02,761 - TorahAnytimeScraper - INFO - File downloader initialized with 53 terms, prefix 'TA', and file type organization enabled
2026-06-25 22:00:02,761 - TorahAnytimeScraper - INFO - TorahAnytime scraper initialized: limit=400, offset=0, project_id=1, include_terms=53, exclude_terms=10
2026-06-25 22:00:02,763 - BatorahScraper - INFO - File downloader initialized with 19 terms, prefix 'BT', and file type organization enabled
2026-06-25 22:00:02,763 - BatorahScraper - INFO - Batorah scraper initialized: api_urls=2, page_size=100, include_terms=19, exclude_terms=1
2026-06-25 22:00:02,763 - KolHalashonScraper - INFO - Kol Halashon scraper initialized: base_url=https://www2.kolhalashon.com, max_shiurim=50, fetch_all_speakers=True, headless=False, exclude_filters=1, include_filters=0
2026-06-25 22:00:02,763 - __main__ - INFO - Ivelt scraper initialized and enabled
2026-06-25 22:00:02,763 - __main__ - INFO - Running 3 scrapers concurrently...
2026-06-25 22:00:02,763 - Yiddish24Scraper - INFO - Starting Yiddish24 scraping (max 10 pages, limit 100 items)
2026-06-25 22:00:02,847 - Yiddish24Scraper - INFO - Fetching Yiddish24 page 1 with data_id='latest', page_limit=100
2026-06-25 22:00:02,851 - IveltScraper - INFO - Loaded 181 include, 0 exclude filters from DB topic_filters
2026-06-25 22:00:02,851 - IveltScraper - INFO - Fetching ivelt active topics from https://www.ivelt.com/forum/search.php?search_id=active_topics...
2026-06-25 22:00:02,853 - RSSFeedScraper - INFO - Fetching RSS feed: https://www.theyeshivaworld.com/feed
2026-06-25 22:00:03,162 - RSSFeedScraper - INFO - Fetching RSS feed: https://www.thegatewaypundit.com/feed/
2026-06-25 22:00:03,596 - RSSFeedScraper - INFO - Total RSS/HTML results: 3
2026-06-25 22:00:15,085 - IveltScraper - INFO - Found 50 active topics on ivelt
2026-06-25 22:00:17,425 - httpx - INFO - HTTP Request: POST https://www.yiddish24.com/ajax/cat_pagination.php "HTTP/1.1 200 OK"
2026-06-25 22:00:17,637 - Yiddish24Scraper - INFO - Received response with 447396 characters
2026-06-25 22:00:17,639 - Yiddish24Scraper - INFO - Extracted HTML from JSON result field (300712 chars)
2026-06-25 22:00:17,641 - Yiddish24Scraper - INFO - Saved HTML debug file to: /mnt/data/openclaw-shared/home/shared/Social_Media_Monitor/debug/yiddish24_page1_response.html
2026-06-25 22:00:17,718 - Yiddish24Scraper - INFO - Found 100 potential article elements on page 1
2026-06-25 22:00:17,765 - Yiddish24Scraper - INFO - Page 1 extraction: 100 successful, 0 failed out of 100 elements
2026-06-25 22:00:17,765 - Yiddish24Scraper - INFO - Parsed 100 articles from HTML on page 1
2026-06-25 22:00:17,816 - Yiddish24Scraper - INFO - Term 'ืืง ืืืฉืจืื' matched in TITLE: 'ืืง ืืืฉืจืื - ืืจื ืืืื ืืขืืืืืืืฉ'
2026-06-25 22:00:17,816 - Yiddish24Scraper - INFO - Downloading 1 media file(s) for article: ืืง ืืืฉืจืื - ืืจื ืืืื ืืขืืืืืืืฉ
2026-06-25 22:00:18,316 - social_media_monitor.utils.file_downloader - INFO - Starting download from: https://cloudfront.yiddish24.com/20260625052803-159672.mp3
2026-06-25 22:00:19,607 - social_media_monitor.utils.file_downloader - INFO - Successfully downloaded: Y2_20260625052803-159672.mp3 (13.5 MB)
2026-06-25 22:00:19,607 - social_media_monitor.utils.file_downloader - INFO - Organized audio file to: /mnt/data/openclaw-shared/home/shared/Social_Media_Monitor/downloads/Audio (audio/mpeg)
2026-06-25 22:00:19,616 - social_media_monitor.utils.media_tagger - INFO - Tagged Y2_20260625052803-159672.mp3: title, artist, album, comment, source_url, genre, publisher
2026-06-25 22:00:19,617 - Yiddish24Scraper - INFO - Renamed downloaded file to include match term: Y2_20260625052803-159672_ืืง_ืืืฉืจืื.mp3
2026-06-25 22:00:20,224 - Yiddish24Scraper - INFO - Page 1 summary: 7 new out of 100 total articles
2026-06-25 22:00:20,224 - Yiddish24Scraper - INFO - Page 1: 7/100 new items
2026-06-25 22:00:20,224 - Yiddish24Scraper - INFO - Low new item ratio (7.0%), stopping pagination
2026-06-25 22:00:20,225 - Yiddish24Scraper - INFO - Total Yiddish24 results found: 7
2026-06-25 22:00:32,235 - IveltScraper - INFO - ivelt scrape complete. Checked 4 topics.
2026-06-25 22:00:32,236 - IveltScraper - INFO - All forums scrape complete. Found 13 new posts.
2026-06-25 22:00:32,236 - __main__ - INFO - Scraper rss completed: 3 results
2026-06-25 22:00:32,236 - __main__ - INFO - Scraper yiddish24 completed: 7 results
2026-06-25 22:00:32,236 - __main__ - INFO - Scraper ivelt completed: 13 results
2026-06-25 22:00:32,237 - __main__ - CRITICAL - Critical error in main: name 'scrapers' is not defined
Traceback (most recent call last):
File "/home/openclaw/Social_Media_Monitor/social_media_monitor.py", line 219, in main
_check_linkedin_feed_failure('linkedin_feed' in scrapers, len(results.get('linkedin_feed_results', [])))
^^^^^^^^
NameError: name 'scrapers' is not defined
2026-06-25 22:00:32,238 - social_media_monitor.database.connection - INFO - Closing all database connections
2026-06-25 22:00:32,301 - social_media_monitor.database.connection - INFO - Closed 5 database connections
2026-06-25 22:00:32,301 - __main__ - INFO - Closed all database connections
2026-06-25 22:00:32,302 - social_media_monitor.database.connection - INFO - Closing all database connections
2026-06-25 22:00:32,302 - social_media_monitor.database.connection - INFO - Closed 0 database connections
2026-06-25 22:30:01,777 - social_media_monitor.utils.logging_config - INFO - Posts log file: /mnt/data/openclaw-shared/home/shared/Social_Media_Monitor/logs/social_media_monitor_posts_2026-06-25.log
2026-06-25 22:30:01,778 - social_media_monitor.utils.logging_config - INFO - Downloads log file: /mnt/data/openclaw-shared/home/shared/Social_Media_Monitor/logs/social_media_monitor_downloads_2026-06-25.log
2026-06-25 22:30:01,778 - social_media_monitor.utils.logging_config - INFO - Logging configured with level INFO, log file: /mnt/data/openclaw-shared/home/shared/Social_Media_Monitor/logs/social_media_monitor_2026-06-25.log
2026-06-25 22:30:01,778 - __main__ - INFO - === Social Media Monitor v2.0 (Async) Started ===
2026-06-25 22:30:01,804 - social_media_monitor.config.settings - INFO - Configuration loaded from /home/openclaw/Social_Media_Monitor/config.yaml
2026-06-25 22:30:01,804 - social_media_monitor.config.settings - WARNING - Twitter monitoring enabled but credentials validation not implemented
2026-06-25 22:30:01,804 - __main__ - INFO - Configuration loaded and validated successfully
2026-06-25 22:30:01,806 - social_media_monitor.database.connection - INFO - Database connection pool initialized with 5 connections
2026-06-25 22:30:01,807 - social_media_monitor.database.connection - INFO - add_speaker_filters column already exists
2026-06-25 22:30:01,820 - social_media_monitor.database.connection - INFO - Database setup completed successfully
2026-06-25 22:30:01,820 - __main__ - INFO - Database initialized at: /mnt/data/openclaw-shared/home/shared/Social_Media_Monitor/social_media_monitor.db
2026-06-25 22:30:01,822 - social_media_monitor.scrapers.reddit_scraper - INFO - Reddit API initialized successfully
2026-06-25 22:30:01,822 - __main__ - INFO - GitHub scraper initialized and enabled
2026-06-25 22:30:01,822 - __main__ - INFO - GitHub feed scraper initialized and enabled
2026-06-25 22:30:01,823 - RSSFeedScraper - INFO - RSS scraper initialized: feeds=2, html_urls=0 (0 with custom selectors), include_terms=0, exclude_terms=0
2026-06-25 22:30:01,823 - __main__ - INFO - LinkedIn Feed scraper initialized and enabled
2026-06-25 22:30:01,825 - Yiddish24Scraper - INFO - File downloader initialized with 55 terms, prefix 'Y2', and file type organization enabled
2026-06-25 22:30:01,825 - Yiddish24Scraper - INFO - Yiddish24 scraper initialized: data_id='latest', page_limit=100, total_pages=10, debug_html=True, include_terms=55, exclude_terms=0
2026-06-25 22:30:01,828 - AhBlickLiveScraper - INFO - AhBlickLive scraper initialized: urls=1, max_items=100, include_terms=49, exclude_terms=0
2026-06-25 22:30:01,832 - TorahAnytimeScraper - INFO - File downloader initialized with 53 terms, prefix 'TA', and file type organization enabled
2026-06-25 22:30:01,832 - TorahAnytimeScraper - INFO - TorahAnytime scraper initialized: limit=400, offset=0, project_id=1, include_terms=53, exclude_terms=10
2026-06-25 22:30:01,834 - BatorahScraper - INFO - File downloader initialized with 19 terms, prefix 'BT', and file type organization enabled
2026-06-25 22:30:01,834 - BatorahScraper - INFO - Batorah scraper initialized: api_urls=2, page_size=100, include_terms=19, exclude_terms=1
2026-06-25 22:30:01,835 - KolHalashonScraper - INFO - Kol Halashon scraper initialized: base_url=https://www2.kolhalashon.com, max_shiurim=50, fetch_all_speakers=True, headless=False, exclude_filters=1, include_filters=0
2026-06-25 22:30:01,835 - __main__ - INFO - Running 3 scrapers concurrently...
2026-06-25 22:30:01,835 - GitHubScraper - INFO - Searching GitHub for terms: ['paperclip', '"Hermes Agent"', 'multica']
2026-06-25 22:30:01,926 - LinkedInFeedScraper - INFO - Fetching LinkedIn feed (25 posts)
2026-06-25 22:30:02,230 - LinkedInFeedScraper - INFO - Auto-grabbed fresh LinkedIn cookies from running Chrome (fallback).
2026-06-25 22:30:09,195 - LinkedInFeedScraper - INFO - Successfully sorted fetched LinkedIn posts chronologically (most recent first)
2026-06-25 22:30:09,327 - LinkedInFeedScraper - INFO - LinkedIn Feed scrape complete. Found 0 new posts.
2026-06-25 22:30:10,484 - httpx - INFO - HTTP Request: GET https://api.github.com/search/repositories?q=paperclip+in%3Aname%2Cdescription%2Creadme+stars%3A%3E150&sort=updated&order=desc&per_page=20 "HTTP/1.1 200 OK"
2026-06-25 22:30:10,575 - GitHubScraper - INFO - Found 240 GitHub repositories for term: paperclip
2026-06-25 22:30:10,725 - httpx - INFO - HTTP Request: GET https://api.github.com/users/abeperl/received_events?per_page=100&page=1 "HTTP/1.1 200 OK"
2026-06-25 22:30:11,841 - httpx - INFO - HTTP Request: GET https://api.github.com/search/repositories?q=%22Hermes+Agent%22+in%3Aname%2Cdescription%2Creadme+stars%3A%3E150&sort=updated&order=desc&per_page=20 "HTTP/1.1 200 OK"
2026-06-25 22:30:11,846 - GitHubScraper - INFO - Found 398 GitHub repositories for term: "Hermes Agent"
2026-06-25 22:30:12,830 - httpx - INFO - HTTP Request: GET https://api.github.com/search/repositories?q=multica+in%3Aname%2Cdescription%2Creadme+stars%3A%3E150&sort=updated&order=desc&per_page=20 "HTTP/1.1 200 OK"
2026-06-25 22:30:12,926 - GitHubScraper - INFO - Found 54 GitHub repositories for term: multica
2026-06-25 22:30:13,068 - GitHubScraper - INFO - Total GitHub results found: 0
2026-06-25 22:30:13,069 - httpx - INFO - HTTP Request: GET https://api.github.com/users/abeperl/received_events?per_page=100&page=2 "HTTP/1.1 200 OK"
2026-06-25 22:30:14,490 - httpx - INFO - HTTP Request: GET https://api.github.com/users/abeperl/received_events?per_page=100&page=3 "HTTP/1.1 200 OK"
2026-06-25 22:30:14,519 - GitHubScraper - INFO - GitHub feed: 204 fetched across pages, 7 new events for abeperl
2026-06-25 22:30:14,520 - __main__ - INFO - Scraper github completed: 0 results
2026-06-25 22:30:14,520 - __main__ - INFO - Scraper github_feed completed: 7 results
2026-06-25 22:30:14,520 - __main__ - INFO - Scraper linkedin_feed completed: 0 results
2026-06-25 22:30:14,520 - __main__ - CRITICAL - Critical error in main: name 'scrapers' is not defined
Traceback (most recent call last):
File "/home/openclaw/Social_Media_Monitor/social_media_monitor.py", line 219, in main
_check_linkedin_feed_failure('linkedin_feed' in scrapers, len(results.get('linkedin_feed_results', [])))
^^^^^^^^
NameError: name 'scrapers' is not defined
2026-06-25 22:30:14,521 - social_media_monitor.database.connection - INFO - Closing all database connections
2026-06-25 22:30:14,565 - social_media_monitor.database.connection - INFO - Closed 5 database connections
2026-06-25 22:30:14,566 - __main__ - INFO - Closed all database connections
2026-06-25 22:30:14,567 - social_media_monitor.database.connection - INFO - Closing all database connections
2026-06-25 22:30:14,568 - social_media_monitor.database.connection - INFO - Closed 0 database connections
2026-06-25 22:37:13,114 - social_media_monitor.utils.logging_config - INFO - Posts log file: /mnt/data/openclaw-shared/home/shared/Social_Media_Monitor/logs/social_media_monitor_posts_2026-06-25.log
2026-06-25 22:37:13,115 - social_media_monitor.utils.logging_config - INFO - Downloads log file: /mnt/data/openclaw-shared/home/shared/Social_Media_Monitor/logs/social_media_monitor_downloads_2026-06-25.log
2026-06-25 22:37:13,115 - social_media_monitor.utils.logging_config - INFO - Logging configured with level INFO, log file: /mnt/data/openclaw-shared/home/shared/Social_Media_Monitor/logs/social_media_monitor_2026-06-25.log
2026-06-25 22:37:13,115 - __main__ - INFO - === Social Media Monitor v2.0 (Async) Started ===
2026-06-25 22:37:13,146 - social_media_monitor.config.settings - INFO - Configuration loaded from /home/openclaw/Social_Media_Monitor/config.yaml
2026-06-25 22:37:13,146 - __main__ - INFO - Configuration loaded and validated successfully
2026-06-25 22:37:13,149 - social_media_monitor.database.connection - INFO - Database connection pool initialized with 5 connections
2026-06-25 22:37:13,150 - social_media_monitor.database.connection - INFO - add_speaker_filters column already exists
2026-06-25 22:37:13,167 - social_media_monitor.database.connection - INFO - Database setup completed successfully
2026-06-25 22:37:13,167 - __main__ - INFO - Database initialized at: /mnt/data/openclaw-shared/home/shared/Social_Media_Monitor/social_media_monitor.db
2026-06-25 22:37:13,170 - social_media_monitor.scrapers.reddit_scraper - INFO - Reddit API initialized successfully
2026-06-25 22:37:13,171 - RSSFeedScraper - INFO - RSS scraper initialized: feeds=2, html_urls=0 (0 with custom selectors), include_terms=0, exclude_terms=0
2026-06-25 22:37:13,171 - __main__ - INFO - RSS scraper initialized and enabled
2026-06-25 22:37:13,174 - Yiddish24Scraper - INFO - File downloader initialized with 55 terms, prefix 'Y2', and file type organization enabled
2026-06-25 22:37:13,175 - Yiddish24Scraper - INFO - Yiddish24 scraper initialized: data_id='latest', page_limit=100, total_pages=10, debug_html=True, include_terms=55, exclude_terms=0
2026-06-25 22:37:13,175 - __main__ - INFO - Yiddish24 scraper initialized and enabled
2026-06-25 22:37:13,178 - AhBlickLiveScraper - INFO - AhBlickLive scraper initialized: urls=1, max_items=100, include_terms=49, exclude_terms=0
2026-06-25 22:37:13,182 - TorahAnytimeScraper - INFO - File downloader initialized with 53 terms, prefix 'TA', and file type organization enabled
2026-06-25 22:37:13,182 - TorahAnytimeScraper - INFO - TorahAnytime scraper initialized: limit=400, offset=0, project_id=1, include_terms=53, exclude_terms=10
2026-06-25 22:37:13,185 - BatorahScraper - INFO - File downloader initialized with 19 terms, prefix 'BT', and file type organization enabled
2026-06-25 22:37:13,185 - BatorahScraper - INFO - Batorah scraper initialized: api_urls=2, page_size=100, include_terms=19, exclude_terms=1
2026-06-25 22:37:13,185 - KolHalashonScraper - INFO - Kol Halashon scraper initialized: base_url=https://www2.kolhalashon.com, max_shiurim=50, fetch_all_speakers=True, headless=False, exclude_filters=1, include_filters=0
2026-06-25 22:37:13,185 - __main__ - INFO - Ivelt scraper initialized and enabled
2026-06-25 22:37:13,185 - __main__ - INFO - Running 3 scrapers concurrently...
2026-06-25 22:37:13,185 - Yiddish24Scraper - INFO - Starting Yiddish24 scraping (max 10 pages, limit 100 items)
2026-06-25 22:37:13,306 - Yiddish24Scraper - INFO - Fetching Yiddish24 page 1 with data_id='latest', page_limit=100
2026-06-25 22:37:13,312 - IveltScraper - INFO - Loaded 181 include, 0 exclude filters from DB topic_filters
2026-06-25 22:37:13,312 - IveltScraper - INFO - Fetching ivelt active topics from https://www.ivelt.com/forum/search.php?search_id=active_topics...
2026-06-25 22:37:13,314 - RSSFeedScraper - INFO - Fetching RSS feed: https://www.theyeshivaworld.com/feed
2026-06-25 22:37:13,565 - RSSFeedScraper - INFO - Fetching RSS feed: https://www.thegatewaypundit.com/feed/
2026-06-25 22:37:13,995 - RSSFeedScraper - INFO - Total RSS/HTML results: 2
2026-06-25 22:37:25,419 - IveltScraper - INFO - Found 50 active topics on ivelt
2026-06-25 22:37:28,521 - IveltScraper - INFO - Filtering post containing exclude term: '*ืืืืข ืืืื ืืืืขืจ ืืืืื ืฆื ืืืจืืืื ืงืจืื ืืข ื ืืืขืก*'
2026-06-25 22:37:32,114 - IveltScraper - INFO - Filtering post containing exclude term: '*ืืืืข ืืืื ืืืืขืจ ืืืืื ืฆื ืืืจืืืื ืงืจืื ืืข ื ืืืขืก*'
2026-06-25 22:37:36,617 - IveltScraper - INFO - ivelt scrape complete. Checked 2 topics.
2026-06-25 22:37:36,618 - IveltScraper - INFO - All forums scrape complete. Found 5 new posts.
2026-06-25 22:37:37,567 - httpx - INFO - HTTP Request: POST https://www.yiddish24.com/ajax/cat_pagination.php "HTTP/1.1 200 OK"
2026-06-25 22:37:37,568 - Yiddish24Scraper - INFO - Received response with 16 characters
2026-06-25 22:37:37,568 - Yiddish24Scraper - WARNING - No 'result' field in JSON response
2026-06-25 22:37:37,569 - Yiddish24Scraper - INFO - Saved HTML debug file to: /mnt/data/openclaw-shared/home/shared/Social_Media_Monitor/debug/yiddish24_page1_response.html
2026-06-25 22:37:37,570 - Yiddish24Scraper - INFO - No bulletin-news-des-row found, trying generic article containers
2026-06-25 22:37:37,570 - Yiddish24Scraper - INFO - Found 0 potential article elements on page 1
2026-06-25 22:37:37,570 - Yiddish24Scraper - INFO - Page 1 extraction: 0 successful, 0 failed out of 0 elements
2026-06-25 22:37:37,570 - Yiddish24Scraper - INFO - Parsed 0 articles from HTML on page 1
2026-06-25 22:37:37,570 - Yiddish24Scraper - INFO - Page 1 summary: 0 new out of 0 total articles
2026-06-25 22:37:37,570 - Yiddish24Scraper - INFO - Page 1: 0/0 new items
2026-06-25 22:37:37,570 - Yiddish24Scraper - INFO - No more results found at page 1, stopping pagination
2026-06-25 22:37:37,571 - Yiddish24Scraper - INFO - Total Yiddish24 results found: 0
2026-06-25 22:37:37,571 - __main__ - INFO - Scraper rss completed: 2 results
2026-06-25 22:37:37,571 - __main__ - INFO - Scraper yiddish24 completed: 0 results
2026-06-25 22:37:37,571 - __main__ - INFO - Scraper ivelt completed: 5 results
2026-06-25 22:37:38,456 - social_media_monitor.email.sender - INFO - Email sent successfully at 2026-06-25 22:37:38.456004
2026-06-25 22:37:38,456 - __main__ - INFO - Email sent successfully
2026-06-25 22:37:38,456 - __main__ - INFO - Cache stats: 0 items in memory, 0 files on disk
2026-06-25 22:37:38,456 - __main__ - INFO - Monitor complete. Found 7 total results: 2 new rss results, 5 new ivelt results.
2026-06-25 22:37:38,456 - __main__ - INFO - === Social Media Monitor v2.0 Complete ===
2026-06-25 22:37:38,456 - social_media_monitor.database.connection - INFO - Closing all database connections
2026-06-25 22:37:38,483 - social_media_monitor.database.connection - INFO - Closed 5 database connections
2026-06-25 22:37:38,484 - __main__ - INFO - Closed all database connections
2026-06-25 22:37:38,485 - social_media_monitor.database.connection - INFO - Closing all database connections
2026-06-25 22:37:38,486 - social_media_monitor.database.connection - INFO - Closed 0 database connections
2026-06-25 23:00:02,610 - social_media_monitor.utils.logging_config - INFO - Posts log file: /mnt/data/openclaw-shared/home/shared/Social_Media_Monitor/logs/social_media_monitor_posts_2026-06-25.log
2026-06-25 23:00:02,610 - social_media_monitor.utils.logging_config - INFO - Downloads log file: /mnt/data/openclaw-shared/home/shared/Social_Media_Monitor/logs/social_media_monitor_downloads_2026-06-25.log
2026-06-25 23:00:02,610 - social_media_monitor.utils.logging_config - INFO - Logging configured with level INFO, log file: /mnt/data/openclaw-shared/home/shared/Social_Media_Monitor/logs/social_media_monitor_2026-06-25.log
2026-06-25 23:00:02,610 - __main__ - INFO - === Social Media Monitor v2.0 (Async) Started ===
2026-06-25 23:00:02,656 - social_media_monitor.config.settings - INFO - Configuration loaded from /home/openclaw/Social_Media_Monitor/config.yaml
2026-06-25 23:00:02,656 - __main__ - INFO - Configuration loaded and validated successfully
2026-06-25 23:00:02,661 - social_media_monitor.database.connection - INFO - Database connection pool initialized with 5 connections
2026-06-25 23:00:02,662 - social_media_monitor.database.connection - INFO - add_speaker_filters column already exists
2026-06-25 23:00:02,722 - social_media_monitor.database.connection - INFO - Database setup completed successfully
2026-06-25 23:00:02,722 - __main__ - INFO - Database initialized at: /mnt/data/openclaw-shared/home/shared/Social_Media_Monitor/social_media_monitor.db
2026-06-25 23:00:02,725 - social_media_monitor.scrapers.reddit_scraper - INFO - Reddit API initialized successfully
2026-06-25 23:00:02,728 - RSSFeedScraper - INFO - RSS scraper initialized: feeds=2, html_urls=0 (0 with custom selectors), include_terms=0, exclude_terms=0
2026-06-25 23:00:02,728 - __main__ - INFO - RSS scraper initialized and enabled
2026-06-25 23:00:02,730 - Yiddish24Scraper - INFO - File downloader initialized with 55 terms, prefix 'Y2', and file type organization enabled
2026-06-25 23:00:02,730 - Yiddish24Scraper - INFO - Yiddish24 scraper initialized: data_id='latest', page_limit=100, total_pages=10, debug_html=True, include_terms=55, exclude_terms=0
2026-06-25 23:00:02,730 - __main__ - INFO - Yiddish24 scraper initialized and enabled
2026-06-25 23:00:02,733 - AhBlickLiveScraper - INFO - AhBlickLive scraper initialized: urls=1, max_items=100, include_terms=49, exclude_terms=0
2026-06-25 23:00:02,741 - TorahAnytimeScraper - INFO - File downloader initialized with 53 terms, prefix 'TA', and file type organization enabled
2026-06-25 23:00:02,741 - TorahAnytimeScraper - INFO - TorahAnytime scraper initialized: limit=400, offset=0, project_id=1, include_terms=53, exclude_terms=10
2026-06-25 23:00:02,748 - BatorahScraper - INFO - File downloader initialized with 19 terms, prefix 'BT', and file type organization enabled
2026-06-25 23:00:02,748 - BatorahScraper - INFO - Batorah scraper initialized: api_urls=2, page_size=100, include_terms=19, exclude_terms=1
2026-06-25 23:00:02,748 - KolHalashonScraper - INFO - Kol Halashon scraper initialized: base_url=https://www2.kolhalashon.com, max_shiurim=50, fetch_all_speakers=True, headless=False, exclude_filters=1, include_filters=0
2026-06-25 23:00:02,749 - __main__ - INFO - Ivelt scraper initialized and enabled
2026-06-25 23:00:02,749 - __main__ - INFO - Running 3 scrapers concurrently...
2026-06-25 23:00:02,750 - Yiddish24Scraper - INFO - Starting Yiddish24 scraping (max 10 pages, limit 100 items)
2026-06-25 23:00:02,909 - Yiddish24Scraper - INFO - Fetching Yiddish24 page 1 with data_id='latest', page_limit=100
2026-06-25 23:00:02,918 - IveltScraper - INFO - Loaded 181 include, 0 exclude filters from DB topic_filters
2026-06-25 23:00:02,918 - IveltScraper - INFO - Fetching ivelt active topics from https://www.ivelt.com/forum/search.php?search_id=active_topics...
2026-06-25 23:00:02,922 - RSSFeedScraper - INFO - Fetching RSS feed: https://www.theyeshivaworld.com/feed
2026-06-25 23:00:03,236 - RSSFeedScraper - INFO - Fetching RSS feed: https://www.thegatewaypundit.com/feed/
2026-06-25 23:00:03,629 - RSSFeedScraper - INFO - Total RSS/HTML results: 1
2026-06-25 23:00:08,712 - httpx - INFO - HTTP Request: POST https://www.yiddish24.com/ajax/cat_pagination.php "HTTP/1.1 200 OK"
2026-06-25 23:00:08,941 - Yiddish24Scraper - INFO - Received response with 456333 characters
2026-06-25 23:00:08,943 - Yiddish24Scraper - INFO - Extracted HTML from JSON result field (306627 chars)
2026-06-25 23:00:08,945 - Yiddish24Scraper - INFO - Saved HTML debug file to: /mnt/data/openclaw-shared/home/shared/Social_Media_Monitor/debug/yiddish24_page1_response.html
2026-06-25 23:00:09,034 - Yiddish24Scraper - INFO - Found 100 potential article elements on page 1
2026-06-25 23:00:09,081 - Yiddish24Scraper - INFO - Page 1 extraction: 100 successful, 0 failed out of 100 elements
2026-06-25 23:00:09,081 - Yiddish24Scraper - INFO - Parsed 100 articles from HTML on page 1
2026-06-25 23:00:09,247 - Yiddish24Scraper - INFO - Term 'ืฉืืื' matched in DESCRIPTION: 'ื"ืง ืืืื"ืจ ืืืฉืขืจื ืืืื ืฉืืื"ื ืคืื ืืจื"ืง ืืื ืืจืืืกืืขืงืืืขื ืืื ื ืจืืฃ ืฆืื ืฆืืืืจ ืื ืฆืืฆืื ืื ืื ืืขืจืฆืขืจ, ืื ืื ืืจืืืืข ืืืืืืจืืืขืื ืืืื ืืืืก ืฉื ืขืืขืจ ืืืคืจืืืขื ืืขื ืืืฃ ืงืืื ืืจืฅ ืืฉืจืื ืืงืืืจื. ืืขืจ ืจืื ืฉืืื"ื ืืขื ืคื...'
2026-06-25 23:00:09,247 - Yiddish24Scraper - INFO - Downloading 1 media file(s) for article: ืขืกืงื ืื ืคืื ืืจ"ื ืืื ืืืืจืืคื ืืจืืืืื ืื ืื ืืืืืืข ื
2026-06-25 23:00:09,990 - social_media_monitor.utils.file_downloader - INFO - Starting download from: https://cloudfront.yiddish24.com/KM_N113_91_1782425188.mp3
2026-06-25 23:00:10,046 - social_media_monitor.utils.file_downloader - INFO - Successfully downloaded: Y2_KM_N113_91_1782425188.mp3 (163.9 KB)
2026-06-25 23:00:10,046 - social_media_monitor.utils.file_downloader - INFO - Organized audio file to: /mnt/data/openclaw-shared/home/shared/Social_Media_Monitor/downloads/Audio (audio/mpeg)
2026-06-25 23:00:10,048 - social_media_monitor.utils.media_tagger - INFO - Tagged Y2_KM_N113_91_1782425188.mp3: title, artist, album, comment, source_url, genre, publisher
2026-06-25 23:00:10,049 - Yiddish24Scraper - INFO - Renamed downloaded file to include match term: Y2_KM_N113_91_1782425188_ืฉืืื.mp3
2026-06-25 23:00:10,606 - Yiddish24Scraper - INFO - Page 1 summary: 19 new out of 100 total articles
2026-06-25 23:00:10,607 - Yiddish24Scraper - INFO - Page 1: 19/100 new items
2026-06-25 23:00:10,607 - Yiddish24Scraper - INFO - Low new item ratio (19.0%), stopping pagination
2026-06-25 23:00:10,607 - Yiddish24Scraper - INFO - Total Yiddish24 results found: 19
2026-06-25 23:00:15,753 - IveltScraper - INFO - Found 50 active topics on ivelt
2026-06-25 23:00:25,348 - IveltScraper - INFO - Filtering post containing exclude term: '*ืืืืข ืืืื ืืืืขืจ ืืืืื ืฆื ืืืจืืืื ืงืจืื ืืข ื ืืืขืก*'
2026-06-25 23:00:27,525 - IveltScraper - INFO - ivelt scrape complete. Checked 3 topics.
2026-06-25 23:00:27,525 - IveltScraper - INFO - All forums scrape complete. Found 4 new posts.
2026-06-25 23:00:27,525 - __main__ - INFO - Scraper rss completed: 1 results
2026-06-25 23:00:27,525 - __main__ - INFO - Scraper yiddish24 completed: 19 results
2026-06-25 23:00:27,525 - __main__ - INFO - Scraper ivelt completed: 4 results
2026-06-25 23:00:28,431 - social_media_monitor.email.sender - INFO - Email sent successfully at 2026-06-25 23:00:28.431423
2026-06-25 23:00:28,431 - __main__ - INFO - Email sent successfully
2026-06-25 23:00:28,431 - __main__ - INFO - Cache stats: 0 items in memory, 0 files on disk
2026-06-25 23:00:28,431 - __main__ - INFO - Monitor complete. Found 24 total results: 1 new rss results, 19 new yiddish24 results, 4 new ivelt results.
2026-06-25 23:00:28,431 - __main__ - INFO - === Social Media Monitor v2.0 Complete ===
2026-06-25 23:00:28,431 - social_media_monitor.database.connection - INFO - Closing all database connections
2026-06-25 23:00:28,482 - social_media_monitor.database.connection - INFO - Closed 5 database connections
2026-06-25 23:00:28,482 - __main__ - INFO - Closed all database connections
2026-06-25 23:00:28,484 - social_media_monitor.database.connection - INFO - Closing all database connections
2026-06-25 23:00:28,484 - social_media_monitor.database.connection - INFO - Closed 0 database connections
2026-06-25 23:30:02,149 - social_media_monitor.utils.logging_config - INFO - Posts log file: /mnt/data/openclaw-shared/home/shared/Social_Media_Monitor/logs/social_media_monitor_posts_2026-06-25.log
2026-06-25 23:30:02,149 - social_media_monitor.utils.logging_config - INFO - Downloads log file: /mnt/data/openclaw-shared/home/shared/Social_Media_Monitor/logs/social_media_monitor_downloads_2026-06-25.log
2026-06-25 23:30:02,149 - social_media_monitor.utils.logging_config - INFO - Logging configured with level INFO, log file: /mnt/data/openclaw-shared/home/shared/Social_Media_Monitor/logs/social_media_monitor_2026-06-25.log
2026-06-25 23:30:02,149 - __main__ - INFO - === Social Media Monitor v2.0 (Async) Started ===
2026-06-25 23:30:02,174 - social_media_monitor.config.settings - INFO - Configuration loaded from /home/openclaw/Social_Media_Monitor/config.yaml
2026-06-25 23:30:02,174 - social_media_monitor.config.settings - WARNING - Twitter monitoring enabled but credentials validation not implemented
2026-06-25 23:30:02,174 - __main__ - INFO - Configuration loaded and validated successfully
2026-06-25 23:30:02,176 - social_media_monitor.database.connection - INFO - Database connection pool initialized with 5 connections
2026-06-25 23:30:02,177 - social_media_monitor.database.connection - INFO - add_speaker_filters column already exists
2026-06-25 23:30:02,196 - social_media_monitor.database.connection - INFO - Database setup completed successfully
2026-06-25 23:30:02,196 - __main__ - INFO - Database initialized at: /mnt/data/openclaw-shared/home/shared/Social_Media_Monitor/social_media_monitor.db
2026-06-25 23:30:02,343 - social_media_monitor.scrapers.reddit_scraper - INFO - Reddit API initialized successfully
2026-06-25 23:30:02,343 - __main__ - INFO - GitHub scraper initialized and enabled
2026-06-25 23:30:02,344 - __main__ - INFO - GitHub feed scraper initialized and enabled
2026-06-25 23:30:02,344 - RSSFeedScraper - INFO - RSS scraper initialized: feeds=2, html_urls=0 (0 with custom selectors), include_terms=0, exclude_terms=0
2026-06-25 23:30:02,345 - __main__ - INFO - LinkedIn Feed scraper initialized and enabled
2026-06-25 23:30:02,348 - Yiddish24Scraper - INFO - File downloader initialized with 55 terms, prefix 'Y2', and file type organization enabled
2026-06-25 23:30:02,348 - Yiddish24Scraper - INFO - Yiddish24 scraper initialized: data_id='latest', page_limit=100, total_pages=10, debug_html=True, include_terms=55, exclude_terms=0
2026-06-25 23:30:02,351 - AhBlickLiveScraper - INFO - AhBlickLive scraper initialized: urls=1, max_items=100, include_terms=49, exclude_terms=0
2026-06-25 23:30:02,356 - TorahAnytimeScraper - INFO - File downloader initialized with 53 terms, prefix 'TA', and file type organization enabled
2026-06-25 23:30:02,357 - TorahAnytimeScraper - INFO - TorahAnytime scraper initialized: limit=400, offset=0, project_id=1, include_terms=53, exclude_terms=10
2026-06-25 23:30:02,360 - BatorahScraper - INFO - File downloader initialized with 19 terms, prefix 'BT', and file type organization enabled
2026-06-25 23:30:02,360 - BatorahScraper - INFO - Batorah scraper initialized: api_urls=2, page_size=100, include_terms=19, exclude_terms=1
2026-06-25 23:30:02,360 - KolHalashonScraper - INFO - Kol Halashon scraper initialized: base_url=https://www2.kolhalashon.com, max_shiurim=50, fetch_all_speakers=True, headless=False, exclude_filters=1, include_filters=0
2026-06-25 23:30:02,360 - __main__ - INFO - Running 3 scrapers concurrently...
2026-06-25 23:30:02,360 - GitHubScraper - INFO - Searching GitHub for terms: ['paperclip', '"Hermes Agent"', 'multica']
2026-06-25 23:30:02,465 - LinkedInFeedScraper - INFO - Fetching LinkedIn feed (25 posts)
2026-06-25 23:30:02,702 - LinkedInFeedScraper - INFO - Auto-grabbed fresh LinkedIn cookies from running Chrome (fallback).
2026-06-25 23:30:11,203 - LinkedInFeedScraper - INFO - Successfully sorted fetched LinkedIn posts chronologically (most recent first)
2026-06-25 23:30:11,210 - LinkedInFeedScraper - INFO - New LinkedIn feed post: .Naftuly (Tuli) Kraus - Live from Cannes.
Not!
2026-06-25 23:30:11,220 - LinkedInFeedScraper - INFO - New LinkedIn feed post: Ariel Rubynstein - ืืืื ื ืชืืืื ืขื ืงืืื ืงืื, hermes ืืื openclaw,
ืื ื ืืืืจ ืฉ #openclaw ืืื ืขืืืื ืืืื
https://lnkd.in/d
2026-06-25 23:30:11,231 - LinkedInFeedScraper - INFO - New LinkedIn feed post: Simcha Friedman - Some of the most important meetings donโt happen in the office.
As a salesman, when a project requi
2026-06-25 23:30:11,238 - LinkedInFeedScraper - INFO - New LinkedIn feed post: Coby S. - Weโre hiring! ๐ If youโre looking for a great team and a rewarding place to work, check out the posi
2026-06-25 23:30:11,246 - LinkedInFeedScraper - INFO - New LinkedIn feed post: Meir Silberberg - Iโm #hiring. Know anyone who might be interested?
2026-06-25 23:30:11,252 - LinkedInFeedScraper - INFO - New LinkedIn feed post: Zachary Fish - Really exciting to deliver one of our most requested features: translation.
We didnโt want to just
2026-06-25 23:30:11,274 - LinkedInFeedScraper - INFO - New LinkedIn feed post: Jenna Ballman - Today I had the chance to stop by the LinkedIn rooftop at the Carlton in during Cannes Lions.
I li
2026-06-25 23:30:11,289 - LinkedInFeedScraper - INFO - New LinkedIn feed post: Jordyn Lake - At Cannes Lions 2026, I had the opportunity to attend a LinkedIn activation at the Carlton Hotel, wh
2026-06-25 23:30:11,295 - LinkedInFeedScraper - INFO - New LinkedIn feed post: Kathryn Belletire - Today I had the opportunity to visit the LinkedIn Activation at the Carlton Hotel during the Cannes
2026-06-25 23:30:11,333 - LinkedInFeedScraper - INFO - LinkedIn Feed scrape complete. Found 9 new posts.
2026-06-25 23:30:12,322 - httpx - INFO - HTTP Request: GET https://api.github.com/users/abeperl/received_events?per_page=100&page=1 "HTTP/1.1 200 OK"
2026-06-25 23:30:13,002 - httpx - INFO - HTTP Request: GET https://api.github.com/search/repositories?q=paperclip+in%3Aname%2Cdescription%2Creadme+stars%3A%3E150&sort=updated&order=desc&per_page=20 "HTTP/1.1 200 OK"
2026-06-25 23:30:13,006 - GitHubScraper - INFO - Found 240 GitHub repositories for term: paperclip
2026-06-25 23:30:13,700 - httpx - INFO - HTTP Request: GET https://api.github.com/users/abeperl/received_events?per_page=100&page=2 "HTTP/1.1 200 OK"
2026-06-25 23:30:14,070 - GitHubScraper - INFO - GitHub feed: 150 fetched across pages, 1 new events for abeperl
2026-06-25 23:30:14,275 - httpx - INFO - HTTP Request: GET https://api.github.com/search/repositories?q=%22Hermes+Agent%22+in%3Aname%2Cdescription%2Creadme+stars%3A%3E150&sort=updated&order=desc&per_page=20 "HTTP/1.1 200 OK"
2026-06-25 23:30:14,362 - GitHubScraper - INFO - Found 398 GitHub repositories for term: "Hermes Agent"
2026-06-25 23:30:15,567 - httpx - INFO - HTTP Request: GET https://api.github.com/search/repositories?q=multica+in%3Aname%2Cdescription%2Creadme+stars%3A%3E150&sort=updated&order=desc&per_page=20 "HTTP/1.1 200 OK"
2026-06-25 23:30:15,657 - GitHubScraper - INFO - Found 54 GitHub repositories for term: multica
2026-06-25 23:30:15,801 - GitHubScraper - INFO - Total GitHub results found: 0
2026-06-25 23:30:15,801 - __main__ - INFO - Scraper github completed: 0 results
2026-06-25 23:30:15,801 - __main__ - INFO - Scraper github_feed completed: 1 results
2026-06-25 23:30:15,801 - __main__ - INFO - Scraper linkedin_feed completed: 9 results
2026-06-25 23:30:15,803 - social_media_monitor.sheets.webinar_sheets_writer - INFO - Checking 9 LinkedIn feed posts for webinar/course + Torah mentions...
2026-06-25 23:30:17,157 - social_media_monitor.sheets.webinar_sheets_writer - INFO - Loaded 42 existing URLs from sheet for dedup
2026-06-25 23:30:18,089 - social_media_monitor.sheets.webinar_sheets_writer - INFO - Loaded 24 existing links from Torah Matches sheet for dedup
2026-06-25 23:30:18,090 - social_media_monitor.sheets.webinar_sheets_writer - INFO - Torah match: Zachary Fish - keyword 'torah' - https://www.linkedin.com/feed/update/urn:li:activity:7476042246727999488
2026-06-25 23:30:18,090 - social_media_monitor.sheets.webinar_sheets_writer - INFO - Webinar/course match: Jordyn Lake - keyword 'session' - https://www.linkedin.com/feed/update/urn:li:activity:7476052756534030336
2026-06-25 23:30:18,091 - social_media_monitor.sheets.webinar_sheets_writer - INFO - Webinar/course match: Kathryn Belletire - keyword 'session' - https://www.linkedin.com/feed/update/urn:li:activity:7476053006698979328
2026-06-25 23:30:18,531 - social_media_monitor.sheets.webinar_sheets_writer - INFO - Appended 2 rows to Google Sheet 'Webinar Recording DMs'
2026-06-25 23:30:20,095 - social_media_monitor.sheets.webinar_sheets_writer - INFO - Appended 1 rows to Google Sheet 'Torah Matches'
2026-06-25 23:30:20,095 - __main__ - INFO - Webinar/Course: appended 3 posts to Google Sheet
2026-06-25 23:30:20,827 - social_media_monitor.email.sender - INFO - Email sent successfully at 2026-06-25 23:30:20.827900
2026-06-25 23:30:20,828 - __main__ - INFO - Email sent successfully
2026-06-25 23:30:20,828 - __main__ - INFO - Cache stats: 0 items in memory, 0 files on disk
2026-06-25 23:30:20,828 - __main__ - INFO - Monitor complete. Found 10 total results: 1 new github feed results, 9 new linkedin feed results.
2026-06-25 23:30:20,828 - __main__ - INFO - === Social Media Monitor v2.0 Complete ===
2026-06-25 23:30:20,828 - social_media_monitor.database.connection - INFO - Closing all database connections
2026-06-25 23:30:20,877 - social_media_monitor.database.connection - INFO - Closed 5 database connections
2026-06-25 23:30:20,877 - __main__ - INFO - Closed all database connections
2026-06-25 23:30:20,879 - social_media_monitor.database.connection - INFO - Closing all database connections
2026-06-25 23:30:20,879 - social_media_monitor.database.connection - INFO - Closed 0 database connections
Live tail active โ waiting for new lines...