โ Logs
social_media_monitor_2026-06-02.log
2026-06-02 20:31:19,566 - LinkedInScraper - INFO - New LinkedIn post: Younรจs BENSAFIAโ - โCentre de Dรฉveloppement des Technologies Avancรฉes
2026-06-02 20:31:19,580 - LinkedInScraper - INFO - New LinkedIn post: I studied physics before I moved into machine learning, and the first
2026-06-02 20:31:19,609 - LinkedInScraper - INFO - LinkedIn scrape complete. Found 24 new results.
2026-06-02 20:31:19,610 - LinkedInFeedScraper - INFO - Fetching LinkedIn feed (25 posts)
2026-06-02 20:31:26,842 - httpx - INFO - HTTP Request: GET https://api.github.com/search/repositories?q=paperclip+in%3Aname%2Cdescription%2Creadme+stars%3A%3E150&sort=updated&order=desc&per_page=20 "HTTP/1.1 200 OK"
2026-06-02 20:31:26,867 - GitHubScraper - INFO - Found 235 GitHub repositories for term: paperclip
2026-06-02 20:31:28,189 - httpx - INFO - HTTP Request: GET https://api.github.com/search/repositories?q=%22Hermes+Agent%22+in%3Aname%2Cdescription%2Creadme+stars%3A%3E150&sort=updated&order=desc&per_page=20 "HTTP/1.1 200 OK"
2026-06-02 20:31:28,286 - GitHubScraper - INFO - Found 315 GitHub repositories for term: "Hermes Agent"
2026-06-02 20:31:29,576 - httpx - INFO - HTTP Request: GET https://api.github.com/search/repositories?q=multica+in%3Aname%2Cdescription%2Creadme+stars%3A%3E150&sort=updated&order=desc&per_page=20 "HTTP/1.1 200 OK"
2026-06-02 20:31:29,580 - GitHubScraper - INFO - Found 47 GitHub repositories for term: multica
2026-06-02 20:31:29,785 - GitHubScraper - INFO - Total GitHub results found: 0
2026-06-02 20:31:29,785 - __main__ - INFO - Scraper reddit completed: 20 results
2026-06-02 20:31:29,785 - __main__ - INFO - Scraper github completed: 0 results
2026-06-02 20:31:29,786 - __main__ - INFO - Scraper twitter completed: 0 results
2026-06-02 20:31:29,786 - __main__ - INFO - Scraper linkedin completed: 24 results
2026-06-02 20:31:29,786 - __main__ - INFO - Scraper linkedin_feed completed: 0 results
2026-06-02 20:31:30,514 - social_media_monitor.email.sender - INFO - Email sent successfully at 2026-06-02 20:31:30.514697
2026-06-02 20:31:30,515 - __main__ - INFO - Email sent successfully
2026-06-02 20:31:30,515 - __main__ - INFO - Cache stats: 0 items in memory, 0 files on disk
2026-06-02 20:31:30,515 - __main__ - INFO - Monitor complete. Found 44 total results: 20 new reddit results, 24 new linkedin results.
2026-06-02 20:31:30,515 - __main__ - INFO - === Social Media Monitor v2.0 Complete ===
2026-06-02 20:31:30,515 - social_media_monitor.database.connection - INFO - Closing all database connections
2026-06-02 20:31:30,620 - social_media_monitor.database.connection - INFO - Closed 5 database connections
2026-06-02 20:31:30,620 - __main__ - INFO - Closed all database connections
2026-06-02 20:31:30,670 - social_media_monitor.database.connection - INFO - Closing all database connections
2026-06-02 20:31:30,682 - social_media_monitor.database.connection - INFO - Closed 0 database connections
2026-06-02 21:00:02,661 - social_media_monitor.utils.logging_config - INFO - Posts log file: /mnt/data/openclaw-shared/home/shared/Social_Media_Monitor/logs/social_media_monitor_posts_2026-06-02.log
2026-06-02 21:00:02,661 - social_media_monitor.utils.logging_config - INFO - Downloads log file: /mnt/data/openclaw-shared/home/shared/Social_Media_Monitor/logs/social_media_monitor_downloads_2026-06-02.log
2026-06-02 21:00:02,661 - social_media_monitor.utils.logging_config - INFO - Logging configured with level INFO, log file: /mnt/data/openclaw-shared/home/shared/Social_Media_Monitor/logs/social_media_monitor_2026-06-02.log
2026-06-02 21:00:02,661 - __main__ - INFO - === Social Media Monitor v2.0 (Async) Started ===
2026-06-02 21:00:02,693 - social_media_monitor.config.settings - INFO - Configuration loaded from /home/openclaw/Social_Media_Monitor/config.yaml
2026-06-02 21:00:02,694 - __main__ - INFO - Configuration loaded and validated successfully
2026-06-02 21:00:02,696 - social_media_monitor.database.connection - INFO - Database connection pool initialized with 5 connections
2026-06-02 21:00:02,698 - social_media_monitor.database.connection - INFO - add_speaker_filters column already exists
2026-06-02 21:00:02,714 - social_media_monitor.database.connection - INFO - Database setup completed successfully
2026-06-02 21:00:02,714 - __main__ - INFO - Database initialized at: /mnt/data/openclaw-shared/home/shared/Social_Media_Monitor/social_media_monitor.db
2026-06-02 21:00:02,765 - social_media_monitor.scrapers.reddit_scraper - INFO - Reddit API initialized successfully
2026-06-02 21:00:02,765 - RSSFeedScraper - INFO - RSS scraper initialized: feeds=2, html_urls=0 (0 with custom selectors), include_terms=1, exclude_terms=0
2026-06-02 21:00:02,765 - __main__ - INFO - RSS scraper initialized and enabled
2026-06-02 21:00:02,768 - Yiddish24Scraper - INFO - File downloader initialized with 30 terms, prefix 'Y2', and file type organization enabled
2026-06-02 21:00:02,768 - Yiddish24Scraper - INFO - Yiddish24 scraper initialized: data_id='latest', page_limit=100, total_pages=10, debug_html=True, include_terms=30, exclude_terms=0
2026-06-02 21:00:02,768 - __main__ - INFO - Yiddish24 scraper initialized and enabled
2026-06-02 21:00:02,770 - AhBlickLiveScraper - INFO - AhBlickLive scraper initialized: urls=1, max_items=100, include_terms=1, exclude_terms=0
2026-06-02 21:00:02,775 - TorahAnytimeScraper - INFO - File downloader initialized with 51 terms, prefix 'TA', and file type organization enabled
2026-06-02 21:00:02,775 - TorahAnytimeScraper - INFO - TorahAnytime scraper initialized: limit=400, offset=0, project_id=1, include_terms=51, exclude_terms=3
2026-06-02 21:00:02,777 - BatorahScraper - INFO - File downloader initialized with 19 terms, prefix 'BT', and file type organization enabled
2026-06-02 21:00:02,777 - BatorahScraper - INFO - Batorah scraper initialized: api_urls=2, page_size=100, include_terms=19, exclude_terms=0
2026-06-02 21:00:02,778 - KolHalashonScraper - INFO - Kol Halashon scraper initialized: base_url=https://www2.kolhalashon.com, max_shiurim=50, fetch_all_speakers=True, headless=False, exclude_filters=1, include_filters=0
2026-06-02 21:00:02,778 - __main__ - INFO - Ivelt scraper initialized and enabled
2026-06-02 21:00:02,778 - __main__ - INFO - Running 3 scrapers concurrently...
2026-06-02 21:00:02,778 - Yiddish24Scraper - INFO - Starting Yiddish24 scraping (max 10 pages, limit 100 items)
2026-06-02 21:00:02,857 - Yiddish24Scraper - INFO - Fetching Yiddish24 page 1 with data_id='latest', page_limit=100
2026-06-02 21:00:02,861 - IveltScraper - INFO - Loaded 178 include, 0 exclude filters from DB topic_filters
2026-06-02 21:00:02,861 - IveltScraper - INFO - Fetching ivelt.com active topics...
2026-06-02 21:00:02,862 - RSSFeedScraper - INFO - Fetching RSS feed: https://www.theyeshivaworld.com/feed
2026-06-02 21:00:03,146 - RSSFeedScraper - INFO - Fetching RSS feed: https://www.thegatewaypundit.com/feed/
2026-06-02 21:00:03,612 - RSSFeedScraper - INFO - Total RSS/HTML results: 3
2026-06-02 21:00:14,593 - IveltScraper - INFO - Found 50 active topics on ivelt.com
2026-06-02 21:00:33,720 - Yiddish24Scraper - ERROR - Error fetching Yiddish24 page 1:
2026-06-02 21:00:33,736 - Yiddish24Scraper - ERROR - Traceback (most recent call last):
File "/home/openclaw/Social_Media_Monitor/venv/lib/python3.14/site-packages/httpx/_transports/default.py", line 101, in map_httpcore_exceptions
yield
File "/home/openclaw/Social_Media_Monitor/venv/lib/python3.14/site-packages/httpx/_transports/default.py", line 394, in handle_async_request
resp = await self._pool.handle_async_request(req)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/openclaw/Social_Media_Monitor/venv/lib/python3.14/site-packages/httpcore/_async/connection_pool.py", line 256, in handle_async_request
raise exc from None
File "/home/openclaw/Social_Media_Monitor/venv/lib/python3.14/site-packages/httpcore/_async/connection_pool.py", line 236, in handle_async_request
response = await connection.handle_async_request(
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
pool_request.request
^^^^^^^^^^^^^^^^^^^^
)
^
File "/home/openclaw/Social_Media_Monitor/venv/lib/python3.14/site-packages/httpcore/_async/connection.py", line 103, in handle_async_request
return await self._connection.handle_async_request(request)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/openclaw/Social_Media_Monitor/venv/lib/python3.14/site-packages/httpcore/_async/http11.py", line 136, in handle_async_request
raise exc
File "/home/openclaw/Social_Media_Monitor/venv/lib/python3.14/site-packages/httpcore/_async/http11.py", line 106, in handle_async_request
) = await self._receive_response_headers(**kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/openclaw/Social_Media_Monitor/venv/lib/python3.14/site-packages/httpcore/_async/http11.py", line 177, in _receive_response_headers
event = await self._receive_event(timeout=timeout)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/openclaw/Social_Media_Monitor/venv/lib/python3.14/site-packages/httpcore/_async/http11.py", line 217, in _receive_event
data = await self._network_stream.read(
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
self.READ_NUM_BYTES, timeout=timeout
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
)
^
File "/home/openclaw/Social_Media_Monitor/venv/lib/python3.14/site-packages/httpcore/_backends/anyio.py", line 32, in read
with map_exceptions(exc_map):
~~~~~~~~~~~~~~^^^^^^^^^
File "/home/linuxbrew/.linuxbrew/opt/python@3.14/lib/python3.14/contextlib.py", line 162, in __exit__
self.gen.throw(value)
~~~~~~~~~~~~~~^^^^^^^
File "/home/openclaw/Social_Media_Monitor/venv/lib/python3.14/site-packages/httpcore/_exceptions.py", line 14, in map_exceptions
raise to_exc(exc) from exc
httpcore.ReadTimeout
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
File "/home/openclaw/Social_Media_Monitor/src/social_media_monitor/scrapers/yiddish24_scraper.py", line 207, in _fetch_page
response = await self.make_request(
^^^^^^^^^^^^^^^^^^^^^^^^
...<5 lines>...
)
^
File "/home/openclaw/Social_Media_Monitor/src/social_media_monitor/scrapers/base_scraper.py", line 231, in make_request
response = await session.post(url, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/openclaw/Social_Media_Monitor/venv/lib/python3.14/site-packages/httpx/_client.py", line 1859, in post
return await self.request(
^^^^^^^^^^^^^^^^^^^
...<13 lines>...
)
^
File "/home/openclaw/Social_Media_Monitor/venv/lib/python3.14/site-packages/httpx/_client.py", line 1540, in request
return await self.send(request, auth=auth, follow_redirects=follow_redirects)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/openclaw/Social_Media_Monitor/venv/lib/python3.14/site-packages/httpx/_client.py", line 1629, in send
response = await self._send_handling_auth(
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
...<4 lines>...
)
^
File "/home/openclaw/Social_Media_Monitor/venv/lib/python3.14/site-packages/httpx/_client.py", line 1657, in _send_handling_auth
response = await self._send_handling_redirects(
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
...<3 lines>...
)
^
File "/home/openclaw/Social_Media_Monitor/venv/lib/python3.14/site-packages/httpx/_client.py", line 1694, in _send_handling_redirects
response = await self._send_single_request(request)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/openclaw/Social_Media_Monitor/venv/lib/python3.14/site-packages/httpx/_client.py", line 1730, in _send_single_request
response = await transport.handle_async_request(request)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/openclaw/Social_Media_Monitor/venv/lib/python3.14/site-packages/httpx/_transports/default.py", line 393, in handle_async_request
with map_httpcore_exceptions():
~~~~~~~~~~~~~~~~~~~~~~~^^
File "/home/linuxbrew/.linuxbrew/opt/python@3.14/lib/python3.14/contextlib.py", line 162, in __exit__
self.gen.throw(value)
~~~~~~~~~~~~~~^^^^^^^
File "/home/openclaw/Social_Media_Monitor/venv/lib/python3.14/site-packages/httpx/_transports/default.py", line 118, in map_httpcore_exceptions
raise mapped_exc(message) from exc
httpx.ReadTimeout
2026-06-02 21:00:33,737 - Yiddish24Scraper - INFO - Page 1: 0/0 new items
2026-06-02 21:00:33,737 - Yiddish24Scraper - INFO - No more results found at page 1, stopping pagination
2026-06-02 21:00:33,737 - Yiddish24Scraper - INFO - Total Yiddish24 results found: 0
2026-06-02 21:00:42,625 - IveltScraper - INFO - Ivelt scrape complete. Checked 4 topics, found 38 new posts.
2026-06-02 21:00:42,626 - __main__ - INFO - Scraper rss completed: 3 results
2026-06-02 21:00:42,626 - __main__ - INFO - Scraper yiddish24 completed: 0 results
2026-06-02 21:00:42,626 - __main__ - INFO - Scraper ivelt completed: 38 results
2026-06-02 21:00:43,398 - social_media_monitor.email.sender - INFO - Email sent successfully at 2026-06-02 21:00:43.398200
2026-06-02 21:00:43,398 - __main__ - INFO - Email sent successfully
2026-06-02 21:00:43,398 - __main__ - INFO - Cache stats: 0 items in memory, 0 files on disk
2026-06-02 21:00:43,399 - __main__ - INFO - Monitor complete. Found 41 total results: 3 new rss results, 38 new ivelt results.
2026-06-02 21:00:43,399 - __main__ - INFO - === Social Media Monitor v2.0 Complete ===
2026-06-02 21:00:43,399 - social_media_monitor.database.connection - INFO - Closing all database connections
2026-06-02 21:00:43,490 - social_media_monitor.database.connection - INFO - Closed 5 database connections
2026-06-02 21:00:43,491 - __main__ - INFO - Closed all database connections
2026-06-02 21:00:43,493 - social_media_monitor.database.connection - INFO - Closing all database connections
2026-06-02 21:00:43,493 - social_media_monitor.database.connection - INFO - Closed 0 database connections
2026-06-02 22:00:01,883 - social_media_monitor.utils.logging_config - INFO - Posts log file: /mnt/data/openclaw-shared/home/shared/Social_Media_Monitor/logs/social_media_monitor_posts_2026-06-02.log
2026-06-02 22:00:01,883 - social_media_monitor.utils.logging_config - INFO - Downloads log file: /mnt/data/openclaw-shared/home/shared/Social_Media_Monitor/logs/social_media_monitor_downloads_2026-06-02.log
2026-06-02 22:00:01,884 - social_media_monitor.utils.logging_config - INFO - Logging configured with level INFO, log file: /mnt/data/openclaw-shared/home/shared/Social_Media_Monitor/logs/social_media_monitor_2026-06-02.log
2026-06-02 22:00:01,884 - __main__ - INFO - === Social Media Monitor v2.0 (Async) Started ===
2026-06-02 22:00:01,942 - social_media_monitor.config.settings - INFO - Configuration loaded from /home/openclaw/Social_Media_Monitor/config.yaml
2026-06-02 22:00:01,942 - __main__ - INFO - Configuration loaded and validated successfully
2026-06-02 22:00:01,945 - social_media_monitor.database.connection - INFO - Database connection pool initialized with 5 connections
2026-06-02 22:00:01,945 - social_media_monitor.database.connection - INFO - add_speaker_filters column already exists
2026-06-02 22:00:01,975 - social_media_monitor.database.connection - INFO - Database setup completed successfully
2026-06-02 22:00:01,976 - __main__ - INFO - Database initialized at: /mnt/data/openclaw-shared/home/shared/Social_Media_Monitor/social_media_monitor.db
2026-06-02 22:00:01,977 - social_media_monitor.scrapers.reddit_scraper - INFO - Reddit API initialized successfully
2026-06-02 22:00:01,978 - RSSFeedScraper - INFO - RSS scraper initialized: feeds=2, html_urls=0 (0 with custom selectors), include_terms=1, exclude_terms=0
2026-06-02 22:00:01,978 - __main__ - INFO - RSS scraper initialized and enabled
2026-06-02 22:00:01,982 - Yiddish24Scraper - INFO - File downloader initialized with 30 terms, prefix 'Y2', and file type organization enabled
2026-06-02 22:00:01,982 - Yiddish24Scraper - INFO - Yiddish24 scraper initialized: data_id='latest', page_limit=100, total_pages=10, debug_html=True, include_terms=30, exclude_terms=0
2026-06-02 22:00:01,982 - __main__ - INFO - Yiddish24 scraper initialized and enabled
2026-06-02 22:00:01,986 - AhBlickLiveScraper - INFO - AhBlickLive scraper initialized: urls=1, max_items=100, include_terms=1, exclude_terms=0
2026-06-02 22:00:01,993 - TorahAnytimeScraper - INFO - File downloader initialized with 51 terms, prefix 'TA', and file type organization enabled
2026-06-02 22:00:01,993 - TorahAnytimeScraper - INFO - TorahAnytime scraper initialized: limit=400, offset=0, project_id=1, include_terms=51, exclude_terms=3
2026-06-02 22:00:01,996 - BatorahScraper - INFO - File downloader initialized with 19 terms, prefix 'BT', and file type organization enabled
2026-06-02 22:00:01,996 - BatorahScraper - INFO - Batorah scraper initialized: api_urls=2, page_size=100, include_terms=19, exclude_terms=0
2026-06-02 22:00:01,996 - KolHalashonScraper - INFO - Kol Halashon scraper initialized: base_url=https://www2.kolhalashon.com, max_shiurim=50, fetch_all_speakers=True, headless=False, exclude_filters=1, include_filters=0
2026-06-02 22:00:01,996 - __main__ - INFO - Ivelt scraper initialized and enabled
2026-06-02 22:00:01,997 - __main__ - INFO - Running 3 scrapers concurrently...
2026-06-02 22:00:01,997 - Yiddish24Scraper - INFO - Starting Yiddish24 scraping (max 10 pages, limit 100 items)
2026-06-02 22:00:02,146 - Yiddish24Scraper - INFO - Fetching Yiddish24 page 1 with data_id='latest', page_limit=100
2026-06-02 22:00:02,153 - IveltScraper - INFO - Loaded 178 include, 0 exclude filters from DB topic_filters
2026-06-02 22:00:02,153 - IveltScraper - INFO - Fetching ivelt.com active topics...
2026-06-02 22:00:02,154 - RSSFeedScraper - INFO - Fetching RSS feed: https://www.theyeshivaworld.com/feed
2026-06-02 22:00:02,674 - RSSFeedScraper - INFO - Fetching RSS feed: https://www.thegatewaypundit.com/feed/
2026-06-02 22:00:03,174 - RSSFeedScraper - INFO - Total RSS/HTML results: 3
2026-06-02 22:00:06,728 - httpx - INFO - HTTP Request: POST https://www.yiddish24.com/ajax/cat_pagination.php "HTTP/1.1 200 OK"
2026-06-02 22:00:06,898 - Yiddish24Scraper - INFO - Received response with 461308 characters
2026-06-02 22:00:06,901 - Yiddish24Scraper - INFO - Extracted HTML from JSON result field (301431 chars)
2026-06-02 22:00:06,905 - Yiddish24Scraper - INFO - Saved HTML debug file to: /mnt/data/openclaw-shared/home/shared/Social_Media_Monitor/debug/yiddish24_page1_response.html
2026-06-02 22:00:06,993 - Yiddish24Scraper - INFO - Found 100 potential article elements on page 1
2026-06-02 22:00:07,032 - Yiddish24Scraper - INFO - Page 1 extraction: 100 successful, 0 failed out of 100 elements
2026-06-02 22:00:07,032 - Yiddish24Scraper - INFO - Parsed 100 articles from HTML on page 1
2026-06-02 22:00:07,048 - Yiddish24Scraper - INFO - Term 'ืืจื' matched in DESCRIPTION: 'ืืจื ืื ืฆืืื ืคืขืจืืืืืขืจ'
2026-06-02 22:00:07,048 - Yiddish24Scraper - INFO - Downloading 1 media file(s) for article: ืืื ืืขืจืืืื: ืืืืขืจืืืื ื ืืืืขืืืื ืืื ืฆืคืื ื ืื ืืขืจืขืจ
2026-06-02 22:00:07,824 - social_media_monitor.utils.file_downloader - INFO - Starting download from: https://cloudfront.yiddish24.com/KM_N114_45_1780436750.mp3
2026-06-02 22:00:08,567 - social_media_monitor.utils.file_downloader - INFO - Successfully downloaded: Y2_KM_N114_45_1780436750.mp3 (459.1 KB)
2026-06-02 22:00:08,568 - social_media_monitor.utils.file_downloader - INFO - Organized audio file to: /mnt/data/openclaw-shared/home/shared/Social_Media_Monitor/downloads/Audio (audio/mpeg)
2026-06-02 22:00:08,577 - social_media_monitor.utils.media_tagger - INFO - Tagged Y2_KM_N114_45_1780436750.mp3: title, artist, album, comment, source_url, genre, publisher
2026-06-02 22:00:08,591 - Yiddish24Scraper - INFO - Term 'ืืจื' matched in DESCRIPTION: 'ืืจื ื ืชื ืจืืืขื ืืืื'
2026-06-02 22:00:08,591 - Yiddish24Scraper - INFO - Downloading 1 media file(s) for article: ืืื ืืขืจืืืื: ืืืืจืืช ืืืืืก ืืชืงืคื: ืฉืืืจืขืืืฉืข ืืคืื ืืช ื
2026-06-02 22:00:09,091 - social_media_monitor.utils.file_downloader - INFO - Starting download from: https://cloudfront.yiddish24.com/KM_N114_44_1780436731.mp3
2026-06-02 22:00:09,133 - social_media_monitor.utils.file_downloader - INFO - Successfully downloaded: Y2_KM_N114_44_1780436731.mp3 (718.3 KB)
2026-06-02 22:00:09,134 - social_media_monitor.utils.file_downloader - INFO - Organized audio file to: /mnt/data/openclaw-shared/home/shared/Social_Media_Monitor/downloads/Audio (audio/mpeg)
2026-06-02 22:00:09,136 - social_media_monitor.utils.media_tagger - INFO - Tagged Y2_KM_N114_44_1780436731.mp3: title, artist, album, comment, source_url, genre, publisher
2026-06-02 22:00:09,166 - Yiddish24Scraper - INFO - Term 'ืืื ืืื' matched in TITLE: 'ืืื ืืื'
2026-06-02 22:00:09,166 - Yiddish24Scraper - INFO - Downloading 1 media file(s) for article: ืืื ืืื
2026-06-02 22:00:10,091 - social_media_monitor.utils.file_downloader - INFO - Starting download from: https://cloudfront.yiddish24.com/arye_nusoi__260_260602022713.mp3
2026-06-02 22:00:11,279 - social_media_monitor.utils.file_downloader - INFO - Successfully downloaded: Y2_arye_nusoi__260_260602022713.mp3 (10.8 MB)
2026-06-02 22:00:11,280 - social_media_monitor.utils.file_downloader - INFO - Organized audio file to: /mnt/data/openclaw-shared/home/shared/Social_Media_Monitor/downloads/Audio (audio/mpeg)
2026-06-02 22:00:11,294 - social_media_monitor.utils.media_tagger - INFO - Tagged Y2_arye_nusoi__260_260602022713.mp3: title, artist, album, comment, source_url, genre, publisher
2026-06-02 22:00:11,322 - Yiddish24Scraper - INFO - Term 'ืืจื' matched in DESCRIPTION: 'ืืขืกืงื ืืจื ืฉืืื ืืืกืฃ ืืขืจืฉืงืืืืืฉ'
2026-06-02 22:00:11,322 - Yiddish24Scraper - INFO - Downloading 1 media file(s) for article: ืืื ืืขืจืืืื: ืกืขืงืฉืื 8 ืคืื ืื ืืืขืื ืืจืืื ืงืืืขื ืืืจืื
2026-06-02 22:00:11,822 - social_media_monitor.utils.file_downloader - INFO - Starting download from: https://cloudfront.yiddish24.com/KM_N114_40_1780436354.mp3
2026-06-02 22:00:12,669 - social_media_monitor.utils.file_downloader - INFO - Successfully downloaded: Y2_KM_N114_40_1780436354.mp3 (404.2 KB)
2026-06-02 22:00:12,669 - social_media_monitor.utils.file_downloader - INFO - Organized audio file to: /mnt/data/openclaw-shared/home/shared/Social_Media_Monitor/downloads/Audio (audio/mpeg)
2026-06-02 22:00:12,671 - social_media_monitor.utils.media_tagger - INFO - Tagged Y2_KM_N114_40_1780436354.mp3: title, artist, album, comment, source_url, genre, publisher
2026-06-02 22:00:12,792 - Yiddish24Scraper - INFO - Term 'ืืื ืืื' matched in TITLE: 'ืืื ืืื'
2026-06-02 22:00:12,792 - Yiddish24Scraper - INFO - Downloading 1 media file(s) for article: ืืื ืืื
2026-06-02 22:00:13,292 - social_media_monitor.utils.file_downloader - INFO - Starting download from: https://cloudfront.yiddish24.com/_________________260_260602022319.mp3
2026-06-02 22:00:14,269 - social_media_monitor.utils.file_downloader - INFO - Successfully downloaded: Y2__________________260_260602022319.mp3 (2.6 MB)
2026-06-02 22:00:14,270 - social_media_monitor.utils.file_downloader - INFO - Organized audio file to: /mnt/data/openclaw-shared/home/shared/Social_Media_Monitor/downloads/Audio (audio/mpeg)
2026-06-02 22:00:14,276 - social_media_monitor.utils.media_tagger - INFO - Tagged Y2__________________260_260602022319.mp3: title, artist, album, comment, source_url, genre, publisher
2026-06-02 22:00:15,006 - Yiddish24Scraper - INFO - Page 1 summary: 16 new out of 100 total articles
2026-06-02 22:00:15,006 - Yiddish24Scraper - INFO - Page 1: 16/100 new items
2026-06-02 22:00:15,006 - Yiddish24Scraper - INFO - Low new item ratio (16.0%), stopping pagination
2026-06-02 22:00:15,006 - Yiddish24Scraper - INFO - Total Yiddish24 results found: 16
2026-06-02 22:00:15,014 - IveltScraper - INFO - Found 50 active topics on ivelt.com
2026-06-02 22:00:34,132 - IveltScraper - INFO - Ivelt scrape complete. Checked 4 topics, found 18 new posts.
2026-06-02 22:00:34,132 - __main__ - INFO - Scraper rss completed: 3 results
2026-06-02 22:00:34,132 - __main__ - INFO - Scraper yiddish24 completed: 16 results
2026-06-02 22:00:34,132 - __main__ - INFO - Scraper ivelt completed: 18 results
2026-06-02 22:00:34,775 - social_media_monitor.email.sender - INFO - Email sent successfully at 2026-06-02 22:00:34.775907
2026-06-02 22:00:34,776 - __main__ - INFO - Email sent successfully
2026-06-02 22:00:34,776 - __main__ - INFO - Cache stats: 0 items in memory, 0 files on disk
2026-06-02 22:00:34,776 - __main__ - INFO - Monitor complete. Found 37 total results: 3 new rss results, 16 new yiddish24 results, 18 new ivelt results.
2026-06-02 22:00:34,776 - __main__ - INFO - === Social Media Monitor v2.0 Complete ===
2026-06-02 22:00:34,776 - social_media_monitor.database.connection - INFO - Closing all database connections
2026-06-02 22:00:34,848 - social_media_monitor.database.connection - INFO - Closed 5 database connections
2026-06-02 22:00:34,849 - __main__ - INFO - Closed all database connections
2026-06-02 22:00:34,850 - social_media_monitor.database.connection - INFO - Closing all database connections
2026-06-02 22:00:34,850 - social_media_monitor.database.connection - INFO - Closed 0 database connections
2026-06-02 22:30:02,384 - social_media_monitor.utils.logging_config - INFO - Posts log file: /mnt/data/openclaw-shared/home/shared/Social_Media_Monitor/logs/social_media_monitor_posts_2026-06-02.log
2026-06-02 22:30:02,385 - social_media_monitor.utils.logging_config - INFO - Downloads log file: /mnt/data/openclaw-shared/home/shared/Social_Media_Monitor/logs/social_media_monitor_downloads_2026-06-02.log
2026-06-02 22:30:02,385 - social_media_monitor.utils.logging_config - INFO - Logging configured with level INFO, log file: /mnt/data/openclaw-shared/home/shared/Social_Media_Monitor/logs/social_media_monitor_2026-06-02.log
2026-06-02 22:30:02,385 - __main__ - INFO - === Social Media Monitor v2.0 (Async) Started ===
2026-06-02 22:30:02,433 - social_media_monitor.config.settings - INFO - Configuration loaded from /home/openclaw/Social_Media_Monitor/config.yaml
2026-06-02 22:30:02,433 - social_media_monitor.config.settings - WARNING - Twitter monitoring enabled but credentials validation not implemented
2026-06-02 22:30:02,433 - __main__ - INFO - Configuration loaded and validated successfully
2026-06-02 22:30:02,436 - social_media_monitor.database.connection - INFO - Database connection pool initialized with 5 connections
2026-06-02 22:30:02,436 - social_media_monitor.database.connection - INFO - add_speaker_filters column already exists
2026-06-02 22:30:02,475 - social_media_monitor.database.connection - INFO - Database setup completed successfully
2026-06-02 22:30:02,475 - __main__ - INFO - Database initialized at: /mnt/data/openclaw-shared/home/shared/Social_Media_Monitor/social_media_monitor.db
2026-06-02 22:30:02,536 - social_media_monitor.scrapers.reddit_scraper - INFO - Reddit API initialized successfully
2026-06-02 22:30:02,537 - __main__ - INFO - Reddit scraper initialized and enabled
2026-06-02 22:30:02,537 - __main__ - INFO - GitHub scraper initialized and enabled
2026-06-02 22:30:02,537 - __main__ - INFO - Twitter scraper initialized and enabled
2026-06-02 22:30:02,538 - RSSFeedScraper - INFO - RSS scraper initialized: feeds=2, html_urls=0 (0 with custom selectors), include_terms=1, exclude_terms=0
2026-06-02 22:30:02,538 - __main__ - INFO - LinkedIn scraper initialized and enabled
2026-06-02 22:30:02,539 - __main__ - INFO - LinkedIn Feed scraper initialized and enabled
2026-06-02 22:30:02,544 - Yiddish24Scraper - INFO - File downloader initialized with 30 terms, prefix 'Y2', and file type organization enabled
2026-06-02 22:30:02,544 - Yiddish24Scraper - INFO - Yiddish24 scraper initialized: data_id='latest', page_limit=100, total_pages=10, debug_html=True, include_terms=30, exclude_terms=0
2026-06-02 22:30:02,549 - AhBlickLiveScraper - INFO - AhBlickLive scraper initialized: urls=1, max_items=100, include_terms=1, exclude_terms=0
2026-06-02 22:30:02,557 - TorahAnytimeScraper - INFO - File downloader initialized with 51 terms, prefix 'TA', and file type organization enabled
2026-06-02 22:30:02,557 - TorahAnytimeScraper - INFO - TorahAnytime scraper initialized: limit=400, offset=0, project_id=1, include_terms=51, exclude_terms=3
2026-06-02 22:30:02,560 - BatorahScraper - INFO - File downloader initialized with 19 terms, prefix 'BT', and file type organization enabled
2026-06-02 22:30:02,560 - BatorahScraper - INFO - Batorah scraper initialized: api_urls=2, page_size=100, include_terms=19, exclude_terms=0
2026-06-02 22:30:02,560 - KolHalashonScraper - INFO - Kol Halashon scraper initialized: base_url=https://www2.kolhalashon.com, max_shiurim=50, fetch_all_speakers=True, headless=False, exclude_filters=1, include_filters=0
2026-06-02 22:30:02,560 - __main__ - INFO - Running 5 scrapers concurrently...
2026-06-02 22:30:02,560 - social_media_monitor.scrapers.reddit_scraper - INFO - Reddit search monitoring enabled. Running search with terms: ['paperclip', '"Hermes Agent"', 'multica']
2026-06-02 22:30:02,561 - social_media_monitor.scrapers.reddit_scraper - INFO - Searching Reddit RSS for terms: ['"Hermes Agent"']
2026-06-02 22:30:03,493 - social_media_monitor.scrapers.reddit_scraper - INFO - Reddit RSS: 25 entries for term '"Hermes Agent"'
2026-06-02 22:30:03,677 - social_media_monitor.scrapers.reddit_scraper - INFO - New Reddit post: Hermes Agent model routing guide: AI inbox triage without the stronges
2026-06-02 22:30:03,690 - social_media_monitor.scrapers.reddit_scraper - INFO - New Reddit post: Obsidian Sync on Linux not working
2026-06-02 22:30:03,704 - social_media_monitor.scrapers.reddit_scraper - INFO - New Reddit post: Mercury Agent install note: Mercury Web Dashboard without the setup-on
2026-06-02 22:30:03,722 - social_media_monitor.scrapers.reddit_scraper - INFO - New Reddit post: How are you safely giving Hermes agent access to personal accounts?
2026-06-02 22:30:03,761 - social_media_monitor.scrapers.reddit_scraper - INFO - New Reddit post: Running Gemma4 E4B with the LiteRT engine on desktop results in 2.4x s
2026-06-02 22:30:03,774 - social_media_monitor.scrapers.reddit_scraper - INFO - New Reddit post: Which backend do you guys run your hermes agent on ?
2026-06-02 22:30:03,786 - social_media_monitor.scrapers.reddit_scraper - INFO - New Reddit post: Looking for opinions on this Kanban agent model list
2026-06-02 22:30:04,952 - social_media_monitor.scrapers.reddit_scraper - INFO - Total Reddit RSS results found: 7
2026-06-02 22:30:04,952 - social_media_monitor.scrapers.reddit_scraper - INFO - Searching 'paperclip' restricted to subreddits: ['PaperClip_AI', 'PaperclipAI']
2026-06-02 22:30:04,952 - social_media_monitor.scrapers.reddit_scraper - INFO - Searching Reddit RSS for terms: ['paperclip']
2026-06-02 22:30:04,952 - social_media_monitor.scrapers.reddit_scraper - INFO - Restricting Reddit search to subreddits: r/PaperClip_AI+PaperclipAI
2026-06-02 22:30:05,977 - social_media_monitor.scrapers.reddit_scraper - INFO - Reddit RSS: 25 entries for term 'paperclip'
2026-06-02 22:30:06,022 - social_media_monitor.scrapers.reddit_scraper - INFO - New Reddit post: Can OneManArmy AI Really Replace an Entire Team? Know the Truth
2026-06-02 22:30:07,281 - social_media_monitor.scrapers.reddit_scraper - INFO - Total Reddit RSS results found: 1
2026-06-02 22:30:07,281 - social_media_monitor.scrapers.reddit_scraper - INFO - Searching 'multica' restricted to subreddits: ['hermesagent', 'HermesAI']
2026-06-02 22:30:07,281 - social_media_monitor.scrapers.reddit_scraper - INFO - Searching Reddit RSS for terms: ['multica']
2026-06-02 22:30:07,281 - social_media_monitor.scrapers.reddit_scraper - INFO - Restricting Reddit search to subreddits: r/hermesagent+HermesAI
2026-06-02 22:30:08,098 - social_media_monitor.scrapers.reddit_scraper - INFO - Reddit RSS: 25 entries for term 'multica'
2026-06-02 22:30:09,362 - social_media_monitor.scrapers.reddit_scraper - INFO - Total Reddit RSS results found: 0
2026-06-02 22:30:09,362 - social_media_monitor.scrapers.reddit_scraper - INFO - Reddit subreddit feed monitoring enabled.
2026-06-02 22:30:09,362 - social_media_monitor.scrapers.reddit_scraper - INFO - Monitoring feed 'hermes_agent': r/hermesagent+HermesAI+AI_Agents+aiagents+AgentsOfAI+LocalLLaMA+AutoGPT (limit=25)
2026-06-02 22:30:09,362 - social_media_monitor.scrapers.reddit_scraper - INFO - Searching feed 'hermes_agent' for: hermes agent via RSS
2026-06-02 22:30:10,227 - social_media_monitor.scrapers.reddit_scraper - INFO - New from RSS feed 'hermes_agent' r/hermesagent: I built a supervisor plugin that kills runaway agents
2026-06-02 22:30:10,259 - social_media_monitor.scrapers.reddit_scraper - INFO - New from RSS feed 'hermes_agent' r/LocalLLaMA: What memory system are you using for your agents?
2026-06-02 22:30:10,280 - social_media_monitor.scrapers.reddit_scraper - INFO - New from RSS feed 'hermes_agent' r/hermesagent: Get Hermes Now !!
2026-06-02 22:30:11,456 - social_media_monitor.scrapers.reddit_scraper - INFO - Searching feed 'hermes_agent' for: hermes AI via RSS
2026-06-02 22:30:12,467 - social_media_monitor.scrapers.reddit_scraper - INFO - New from RSS feed 'hermes_agent' r/opencodeCLI: OpenAI models down on OpenCode?
2026-06-02 22:30:13,621 - social_media_monitor.scrapers.reddit_scraper - INFO - Searching feed 'hermes_agent' for: nous research hermes via RSS
2026-06-02 22:30:14,997 - social_media_monitor.scrapers.reddit_scraper - INFO - New from RSS feed 'hermes_agent' r/Starcitizen_trades: ๐๏ธ๐โจ๐ช ๐๐๐๐๐๐๐๐๐๐ & ๐๐๐๐๐๐๐ ๐๐๐๐ ๐ ๏ธ โญ๐๐๐๐พ๐๐ผ๐ ๐๐๐๐๐๐โญโฎPOLARIS $495โฎPRESE
2026-06-02 22:30:16,226 - social_media_monitor.scrapers.reddit_scraper - INFO - Searching feed 'hermes_agent' for: multica via RSS
2026-06-02 22:30:18,207 - social_media_monitor.scrapers.reddit_scraper - INFO - Monitoring feed 'paperclip_ai': r/PaperClip_AI+PaperclipAI+pAIperclip (limit=25)
2026-06-02 22:30:18,208 - social_media_monitor.scrapers.reddit_scraper - INFO - Searching feed 'paperclip_ai' for: paperclip ai via RSS
2026-06-02 22:30:20,633 - social_media_monitor.scrapers.reddit_scraper - INFO - Searching feed 'paperclip_ai' for: paperclip agent via RSS
2026-06-02 22:30:22,716 - social_media_monitor.scrapers.reddit_scraper - INFO - Searching feed 'paperclip_ai' for: multica via RSS
2026-06-02 22:30:24,792 - social_media_monitor.scrapers.reddit_scraper - INFO - Monitoring feed 'claude_agents': r/vibecoding (limit=25)
2026-06-02 22:30:24,792 - social_media_monitor.scrapers.reddit_scraper - INFO - Searching feed 'claude_agents' for: multica via RSS
2026-06-02 22:30:26,861 - social_media_monitor.scrapers.reddit_scraper - INFO - Total subreddit feed results: 5
2026-06-02 22:30:26,861 - GitHubScraper - INFO - Searching GitHub for terms: ['paperclip', '"Hermes Agent"', 'multica']
2026-06-02 22:30:26,999 - TwitterScraper - INFO - Searching Twitter/X (Google News RSS) for: ['paperclip', '"Hermes Agent"', 'multica']
2026-06-02 22:30:27,534 - TwitterScraper - INFO - Twitter/X RSS: 100 entries for 'paperclip'
2026-06-02 22:30:42,190 - TwitterScraper - INFO - Twitter/X RSS: 100 entries for '"Hermes Agent"'
2026-06-02 22:30:56,811 - TwitterScraper - INFO - Twitter/X RSS: 100 entries for 'multica'
2026-06-02 22:31:11,142 - TwitterScraper - INFO - Twitter scrape complete. Found 0 new results.
2026-06-02 22:31:11,143 - LinkedInScraper - INFO - Searching LinkedIn (Google News RSS) for: ['paperclip', '"Hermes Agent"', 'multica']
2026-06-02 22:31:11,723 - LinkedInScraper - INFO - LinkedIn RSS: 100 entries for 'paperclip'
2026-06-02 22:31:13,095 - LinkedInScraper - INFO - LinkedIn RSS: 100 entries for '"Hermes Agent"'
2026-06-02 22:31:14,434 - LinkedInScraper - INFO - LinkedIn RSS: 100 entries for 'multica'
2026-06-02 22:31:15,197 - LinkedInScraper - INFO - New LinkedIn post: Anurag Joshi - Engineering @ JindalX - LinkedIn India
2026-06-02 22:31:15,212 - LinkedInScraper - INFO - LinkedIn scrape complete. Found 1 new results.
2026-06-02 22:31:15,212 - LinkedInFeedScraper - INFO - Fetching LinkedIn feed (25 posts)
2026-06-02 22:31:21,256 - httpx - INFO - HTTP Request: GET https://api.github.com/search/repositories?q=paperclip+in%3Aname%2Cdescription%2Creadme+stars%3A%3E150&sort=updated&order=desc&per_page=20 "HTTP/1.1 200 OK"
2026-06-02 22:31:21,280 - GitHubScraper - INFO - Found 235 GitHub repositories for term: paperclip
2026-06-02 22:31:22,772 - httpx - INFO - HTTP Request: GET https://api.github.com/search/repositories?q=%22Hermes+Agent%22+in%3Aname%2Cdescription%2Creadme+stars%3A%3E150&sort=updated&order=desc&per_page=20 "HTTP/1.1 200 OK"
2026-06-02 22:31:22,871 - GitHubScraper - INFO - Found 315 GitHub repositories for term: "Hermes Agent"
2026-06-02 22:31:23,054 - GitHubScraper - INFO - Added new GitHub repository: hermes-ui
2026-06-02 22:31:24,003 - httpx - INFO - HTTP Request: GET https://api.github.com/search/repositories?q=multica+in%3Aname%2Cdescription%2Creadme+stars%3A%3E150&sort=updated&order=desc&per_page=20 "HTTP/1.1 200 OK"
2026-06-02 22:31:24,006 - GitHubScraper - INFO - Found 47 GitHub repositories for term: multica
2026-06-02 22:31:24,184 - GitHubScraper - INFO - Total GitHub results found: 1
2026-06-02 22:31:24,184 - __main__ - INFO - Scraper reddit completed: 13 results
2026-06-02 22:31:24,184 - __main__ - INFO - Scraper github completed: 1 results
2026-06-02 22:31:24,184 - __main__ - INFO - Scraper twitter completed: 0 results
2026-06-02 22:31:24,184 - __main__ - INFO - Scraper linkedin completed: 1 results
2026-06-02 22:31:24,184 - __main__ - INFO - Scraper linkedin_feed completed: 0 results
2026-06-02 22:31:24,829 - social_media_monitor.email.sender - INFO - Email sent successfully at 2026-06-02 22:31:24.829601
2026-06-02 22:31:24,829 - __main__ - INFO - Email sent successfully
2026-06-02 22:31:24,830 - __main__ - INFO - Cache stats: 0 items in memory, 0 files on disk
2026-06-02 22:31:24,830 - __main__ - INFO - Monitor complete. Found 15 total results: 13 new reddit results, 1 new github results, 1 new linkedin results.
2026-06-02 22:31:24,830 - __main__ - INFO - === Social Media Monitor v2.0 Complete ===
2026-06-02 22:31:24,830 - social_media_monitor.database.connection - INFO - Closing all database connections
2026-06-02 22:31:24,942 - social_media_monitor.database.connection - INFO - Closed 5 database connections
2026-06-02 22:31:24,942 - __main__ - INFO - Closed all database connections
2026-06-02 22:31:24,985 - social_media_monitor.database.connection - INFO - Closing all database connections
2026-06-02 22:31:24,995 - social_media_monitor.database.connection - INFO - Closed 0 database connections
2026-06-02 23:00:01,849 - social_media_monitor.utils.logging_config - INFO - Posts log file: /mnt/data/openclaw-shared/home/shared/Social_Media_Monitor/logs/social_media_monitor_posts_2026-06-02.log
2026-06-02 23:00:01,849 - social_media_monitor.utils.logging_config - INFO - Downloads log file: /mnt/data/openclaw-shared/home/shared/Social_Media_Monitor/logs/social_media_monitor_downloads_2026-06-02.log
2026-06-02 23:00:01,849 - social_media_monitor.utils.logging_config - INFO - Logging configured with level INFO, log file: /mnt/data/openclaw-shared/home/shared/Social_Media_Monitor/logs/social_media_monitor_2026-06-02.log
2026-06-02 23:00:01,850 - __main__ - INFO - === Social Media Monitor v2.0 (Async) Started ===
2026-06-02 23:00:01,884 - social_media_monitor.config.settings - INFO - Configuration loaded from /home/openclaw/Social_Media_Monitor/config.yaml
2026-06-02 23:00:01,884 - __main__ - INFO - Configuration loaded and validated successfully
2026-06-02 23:00:01,886 - social_media_monitor.database.connection - INFO - Database connection pool initialized with 5 connections
2026-06-02 23:00:01,886 - social_media_monitor.database.connection - INFO - add_speaker_filters column already exists
2026-06-02 23:00:01,907 - social_media_monitor.database.connection - INFO - Database setup completed successfully
2026-06-02 23:00:01,907 - __main__ - INFO - Database initialized at: /mnt/data/openclaw-shared/home/shared/Social_Media_Monitor/social_media_monitor.db
2026-06-02 23:00:01,908 - social_media_monitor.scrapers.reddit_scraper - INFO - Reddit API initialized successfully
2026-06-02 23:00:01,909 - RSSFeedScraper - INFO - RSS scraper initialized: feeds=2, html_urls=0 (0 with custom selectors), include_terms=1, exclude_terms=0
2026-06-02 23:00:01,909 - __main__ - INFO - RSS scraper initialized and enabled
2026-06-02 23:00:01,911 - Yiddish24Scraper - INFO - File downloader initialized with 30 terms, prefix 'Y2', and file type organization enabled
2026-06-02 23:00:01,911 - Yiddish24Scraper - INFO - Yiddish24 scraper initialized: data_id='latest', page_limit=100, total_pages=10, debug_html=True, include_terms=30, exclude_terms=0
2026-06-02 23:00:01,912 - __main__ - INFO - Yiddish24 scraper initialized and enabled
2026-06-02 23:00:01,914 - AhBlickLiveScraper - INFO - AhBlickLive scraper initialized: urls=1, max_items=100, include_terms=1, exclude_terms=0
2026-06-02 23:00:01,920 - TorahAnytimeScraper - INFO - File downloader initialized with 51 terms, prefix 'TA', and file type organization enabled
2026-06-02 23:00:01,922 - TorahAnytimeScraper - INFO - TorahAnytime scraper initialized: limit=400, offset=0, project_id=1, include_terms=51, exclude_terms=3
2026-06-02 23:00:01,924 - BatorahScraper - INFO - File downloader initialized with 19 terms, prefix 'BT', and file type organization enabled
2026-06-02 23:00:01,924 - BatorahScraper - INFO - Batorah scraper initialized: api_urls=2, page_size=100, include_terms=19, exclude_terms=0
2026-06-02 23:00:01,924 - KolHalashonScraper - INFO - Kol Halashon scraper initialized: base_url=https://www2.kolhalashon.com, max_shiurim=50, fetch_all_speakers=True, headless=False, exclude_filters=1, include_filters=0
2026-06-02 23:00:01,924 - __main__ - INFO - Ivelt scraper initialized and enabled
2026-06-02 23:00:01,924 - __main__ - INFO - Running 3 scrapers concurrently...
2026-06-02 23:00:01,924 - Yiddish24Scraper - INFO - Starting Yiddish24 scraping (max 10 pages, limit 100 items)
2026-06-02 23:00:02,007 - Yiddish24Scraper - INFO - Fetching Yiddish24 page 1 with data_id='latest', page_limit=100
2026-06-02 23:00:02,011 - IveltScraper - INFO - Loaded 178 include, 0 exclude filters from DB topic_filters
2026-06-02 23:00:02,011 - IveltScraper - INFO - Fetching ivelt.com active topics...
2026-06-02 23:00:02,012 - RSSFeedScraper - INFO - Fetching RSS feed: https://www.theyeshivaworld.com/feed
2026-06-02 23:00:02,342 - RSSFeedScraper - INFO - Fetching RSS feed: https://www.thegatewaypundit.com/feed/
2026-06-02 23:00:02,807 - RSSFeedScraper - INFO - Total RSS/HTML results: 4
2026-06-02 23:00:13,788 - IveltScraper - INFO - Found 50 active topics on ivelt.com
2026-06-02 23:00:32,940 - Yiddish24Scraper - ERROR - Error fetching Yiddish24 page 1:
2026-06-02 23:00:32,949 - Yiddish24Scraper - ERROR - Traceback (most recent call last):
File "/home/openclaw/Social_Media_Monitor/venv/lib/python3.14/site-packages/httpx/_transports/default.py", line 101, in map_httpcore_exceptions
yield
File "/home/openclaw/Social_Media_Monitor/venv/lib/python3.14/site-packages/httpx/_transports/default.py", line 394, in handle_async_request
resp = await self._pool.handle_async_request(req)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/openclaw/Social_Media_Monitor/venv/lib/python3.14/site-packages/httpcore/_async/connection_pool.py", line 256, in handle_async_request
raise exc from None
File "/home/openclaw/Social_Media_Monitor/venv/lib/python3.14/site-packages/httpcore/_async/connection_pool.py", line 236, in handle_async_request
response = await connection.handle_async_request(
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
pool_request.request
^^^^^^^^^^^^^^^^^^^^
)
^
File "/home/openclaw/Social_Media_Monitor/venv/lib/python3.14/site-packages/httpcore/_async/connection.py", line 103, in handle_async_request
return await self._connection.handle_async_request(request)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/openclaw/Social_Media_Monitor/venv/lib/python3.14/site-packages/httpcore/_async/http11.py", line 136, in handle_async_request
raise exc
File "/home/openclaw/Social_Media_Monitor/venv/lib/python3.14/site-packages/httpcore/_async/http11.py", line 106, in handle_async_request
) = await self._receive_response_headers(**kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/openclaw/Social_Media_Monitor/venv/lib/python3.14/site-packages/httpcore/_async/http11.py", line 177, in _receive_response_headers
event = await self._receive_event(timeout=timeout)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/openclaw/Social_Media_Monitor/venv/lib/python3.14/site-packages/httpcore/_async/http11.py", line 217, in _receive_event
data = await self._network_stream.read(
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
self.READ_NUM_BYTES, timeout=timeout
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
)
^
File "/home/openclaw/Social_Media_Monitor/venv/lib/python3.14/site-packages/httpcore/_backends/anyio.py", line 32, in read
with map_exceptions(exc_map):
~~~~~~~~~~~~~~^^^^^^^^^
File "/home/linuxbrew/.linuxbrew/opt/python@3.14/lib/python3.14/contextlib.py", line 162, in __exit__
self.gen.throw(value)
~~~~~~~~~~~~~~^^^^^^^
File "/home/openclaw/Social_Media_Monitor/venv/lib/python3.14/site-packages/httpcore/_exceptions.py", line 14, in map_exceptions
raise to_exc(exc) from exc
httpcore.ReadTimeout
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
File "/home/openclaw/Social_Media_Monitor/src/social_media_monitor/scrapers/yiddish24_scraper.py", line 207, in _fetch_page
response = await self.make_request(
^^^^^^^^^^^^^^^^^^^^^^^^
...<5 lines>...
)
^
File "/home/openclaw/Social_Media_Monitor/src/social_media_monitor/scrapers/base_scraper.py", line 231, in make_request
response = await session.post(url, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/openclaw/Social_Media_Monitor/venv/lib/python3.14/site-packages/httpx/_client.py", line 1859, in post
return await self.request(
^^^^^^^^^^^^^^^^^^^
...<13 lines>...
)
^
File "/home/openclaw/Social_Media_Monitor/venv/lib/python3.14/site-packages/httpx/_client.py", line 1540, in request
return await self.send(request, auth=auth, follow_redirects=follow_redirects)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/openclaw/Social_Media_Monitor/venv/lib/python3.14/site-packages/httpx/_client.py", line 1629, in send
response = await self._send_handling_auth(
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
...<4 lines>...
)
^
File "/home/openclaw/Social_Media_Monitor/venv/lib/python3.14/site-packages/httpx/_client.py", line 1657, in _send_handling_auth
response = await self._send_handling_redirects(
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
...<3 lines>...
)
^
File "/home/openclaw/Social_Media_Monitor/venv/lib/python3.14/site-packages/httpx/_client.py", line 1694, in _send_handling_redirects
response = await self._send_single_request(request)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/openclaw/Social_Media_Monitor/venv/lib/python3.14/site-packages/httpx/_client.py", line 1730, in _send_single_request
response = await transport.handle_async_request(request)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/openclaw/Social_Media_Monitor/venv/lib/python3.14/site-packages/httpx/_transports/default.py", line 393, in handle_async_request
with map_httpcore_exceptions():
~~~~~~~~~~~~~~~~~~~~~~~^^
File "/home/linuxbrew/.linuxbrew/opt/python@3.14/lib/python3.14/contextlib.py", line 162, in __exit__
self.gen.throw(value)
~~~~~~~~~~~~~~^^^^^^^
File "/home/openclaw/Social_Media_Monitor/venv/lib/python3.14/site-packages/httpx/_transports/default.py", line 118, in map_httpcore_exceptions
raise mapped_exc(message) from exc
httpx.ReadTimeout
2026-06-02 23:00:32,949 - Yiddish24Scraper - INFO - Page 1: 0/0 new items
2026-06-02 23:00:32,949 - Yiddish24Scraper - INFO - No more results found at page 1, stopping pagination
2026-06-02 23:00:32,949 - Yiddish24Scraper - INFO - Total Yiddish24 results found: 0
2026-06-02 23:00:37,100 - IveltScraper - INFO - Ivelt scrape complete. Checked 3 topics, found 13 new posts.
2026-06-02 23:00:37,100 - __main__ - INFO - Scraper rss completed: 4 results
2026-06-02 23:00:37,100 - __main__ - INFO - Scraper yiddish24 completed: 0 results
2026-06-02 23:00:37,100 - __main__ - INFO - Scraper ivelt completed: 13 results
2026-06-02 23:00:37,717 - social_media_monitor.email.sender - INFO - Email sent successfully at 2026-06-02 23:00:37.717641
2026-06-02 23:00:37,717 - __main__ - INFO - Email sent successfully
2026-06-02 23:00:37,718 - __main__ - INFO - Cache stats: 0 items in memory, 0 files on disk
2026-06-02 23:00:37,718 - __main__ - INFO - Monitor complete. Found 17 total results: 4 new rss results, 13 new ivelt results.
2026-06-02 23:00:37,718 - __main__ - INFO - === Social Media Monitor v2.0 Complete ===
2026-06-02 23:00:37,718 - social_media_monitor.database.connection - INFO - Closing all database connections
2026-06-02 23:00:37,782 - social_media_monitor.database.connection - INFO - Closed 5 database connections
2026-06-02 23:00:37,782 - __main__ - INFO - Closed all database connections
2026-06-02 23:00:37,784 - social_media_monitor.database.connection - INFO - Closing all database connections
2026-06-02 23:00:37,784 - social_media_monitor.database.connection - INFO - Closed 0 database connections
Live tail active โ waiting for new lines...