← Logs

social_media_monitor_2026-05-19.log

2026-05-19 21:56:35,392 - KolHalashonScraper - INFO - STEP 2 visit processed URL: https://www2.kolhalashon.com/regularSite/searchResults/6/-1/-1/-1/-1/New%20Shiurei%20Torah?urlFilters=3:3%7C14:35%7C

2026-05-19 21:56:36,478 - KolHalashonScraper - INFO - Page loaded with 49 shiur containers after 0s

2026-05-19 21:56:36,479 - KolHalashonScraper - INFO - [DELAY] Waiting 70.0s for page content to stabilize (shiur_found_wait)

2026-05-19 21:57:52,513 - KolHalashonScraper - INFO - Processing page 1 of URL (pagination shows: page 1 of 2, has_next=True)

2026-05-19 21:57:52,513 - KolHalashonScraper - INFO - [DELAY] Waiting 6.0s for DOM extraction (dom_extraction_wait)

2026-05-19 21:57:58,527 - KolHalashonScraper - INFO - Extracted 24 shiurim from DOM

2026-05-19 21:57:58,528 - KolHalashonScraper - INFO - Page 1: Browser extracted 24 shiurim

2026-05-19 21:57:58,537 - KolHalashonScraper - INFO - Navigating to page 2...

2026-05-19 21:57:58,588 - KolHalashonScraper - INFO - [DELAY] Waiting 10s for page to reload after clicking next button

2026-05-19 21:58:15,638 - KolHalashonScraper - INFO - Processing page 2 of URL (pagination shows: page 2 of 2, has_next=False)

2026-05-19 21:58:15,638 - KolHalashonScraper - INFO - [DELAY] Waiting 6.0s for DOM extraction (dom_extraction_wait)

2026-05-19 21:58:21,650 - KolHalashonScraper - INFO - Extracted 21 shiurim from DOM

2026-05-19 21:58:21,651 - KolHalashonScraper - INFO - Page 2: Browser extracted 21 shiurim

2026-05-19 21:58:21,659 - KolHalashonScraper - INFO - No more pages available after page 2

2026-05-19 21:58:21,660 - KolHalashonScraper - INFO - URL complete: 45 shiurim extracted from 2 page(s), 56 new total so far

2026-05-19 21:58:21,825 - KolHalashonScraper - INFO - [DELAY] Waiting 0.7s for filter sections to expand (filter_expand_wait)

2026-05-19 21:58:22,582 - KolHalashonScraper - INFO - [DELAY] Waiting 0.7s after clicking 'display more' buttons (filter_expand_wait)

2026-05-19 21:58:23,297 - KolHalashonScraper - INFO - Expanded filter sections: clicked 2 'display more' buttons

2026-05-19 21:58:24,306 - KolHalashonScraper - INFO - DOM filter debug: {'displayAllSections': 2, 'filterContents': 8, 'filterOptions': 87, 'totalCheckboxes': 85, 'type2Checkboxes': 24}

2026-05-19 21:58:24,317 - KolHalashonScraper - INFO - Extracted 24 speakers from DOM filter checkboxes

2026-05-19 21:58:24,317 - KolHalashonScraper - INFO - Extracted 24 speakers from URL filter dropdown

2026-05-19 21:58:24,328 - social_media_monitor.scrapers.kolhalashon_scraper - INFO - Speaker sync complete: 0 new, 24 updated, 520 total

2026-05-19 21:58:24,328 - KolHalashonScraper - INFO - [DELAY] Waiting 1.0s between URLs (shiur_url_delay)

2026-05-19 21:58:25,329 - KolHalashonScraper - INFO - Processing URL (id=8): https://www.kolhalashon.com/en/regularSite/searchResults/100/49/Meilah?urlFilters=3:3%7C20:17%7C{speakersfilter}

2026-05-19 21:58:25,329 - KolHalashonScraper - INFO - Replaced speakersfilter placeholder with 5 speaker filter(s)

2026-05-19 21:58:25,330 - KolHalashonScraper - INFO - Final processed URL: https://www.kolhalashon.com/en/regularSite/searchResults/100/49/Meilah?urlFilters=3:3%7C20:17%7C2:7651%7C2:1608%7C2:7011%7C2:1762%7C2:7163

2026-05-19 21:58:25,330 - KolHalashonScraper - INFO - STEP 2 visit processed URL: https://www.kolhalashon.com/en/regularSite/searchResults/100/49/Meilah?urlFilters=3:3%7C20:17%7C2:7651%7C2:1608%7C2:7011%7C2:1762%7C2:7163

2026-05-19 21:58:26,405 - KolHalashonScraper - INFO - Page loaded with 49 shiur containers after 0s

2026-05-19 21:58:26,405 - KolHalashonScraper - INFO - [DELAY] Waiting 70.0s for page content to stabilize (shiur_found_wait)

2026-05-19 21:59:42,443 - KolHalashonScraper - INFO - Processing page 1 of URL (pagination shows: page 1 of 1, has_next=False)

2026-05-19 21:59:42,443 - KolHalashonScraper - INFO - [DELAY] Waiting 6.0s for DOM extraction (dom_extraction_wait)

2026-05-19 21:59:48,455 - KolHalashonScraper - INFO - Extracted 7 shiurim from DOM

2026-05-19 21:59:48,456 - KolHalashonScraper - INFO - Page 1: Browser extracted 7 shiurim

2026-05-19 21:59:48,457 - KolHalashonScraper - INFO - Starting inline downloads for page 1 (reusing current page)

2026-05-19 21:59:48,457 - KolHalashonScraper - INFO - Using fallback parsha for 2026-05-23: Naso (נשא)

2026-05-19 21:59:48,458 - KolHalashonScraper - INFO - Download directory: /mnt/data/openclaw-shared/home/shared/Social_Media_Monitor/downloads/Audio

2026-05-19 21:59:48,458 - KolHalashonScraper - INFO - Using 3 provided shiur(s) for download (page-derived, in-memory)

2026-05-19 21:59:48,458 - KolHalashonScraper - INFO - Starting download of 3 shiurim (one_per_speaker=True)

2026-05-19 21:59:48,462 - KolHalashonScraper - INFO - Configured download behavior via Browser.setDownloadBehavior CDP for /mnt/data/openclaw-shared/home/shared/Social_Media_Monitor/downloads/Audio

2026-05-19 21:59:48,466 - KolHalashonScraper - INFO - Configured download behavior via Page.setDownloadBehavior CDP for /mnt/data/openclaw-shared/home/shared/Social_Media_Monitor/downloads/Audio

2026-05-19 21:59:48,466 - KolHalashonScraper - INFO - Reusing current page for downloads (no navigation)

2026-05-19 21:59:54,529 - KolHalashonScraper - INFO - Processing page 1 (pagination shows: page 1 of 1, has_next=False)

2026-05-19 21:59:54,539 - KolHalashonScraper - INFO - Page 1 has 7 shiur items

2026-05-19 21:59:54,539 - KolHalashonScraper - INFO - [p1:0] Attempting download: 57:17 Harav Elazar Nissan Rubin Masechet Meilah, Daf 17 Shiu by Unknown (ID: 41492370)

2026-05-19 21:59:55,736 - KolHalashonScraper - INFO - Download button clicked via ActionChains for shiur 41492370

2026-05-19 21:59:55,736 - KolHalashonScraper - INFO - [DELAY] Waiting 1s after download button click for dialog to appear (implicit stabilization)

2026-05-19 22:00:02,628 - social_media_monitor.utils.logging_config - INFO - Posts log file: /mnt/data/openclaw-shared/home/shared/Social_Media_Monitor/logs/social_media_monitor_posts_2026-05-19.log

2026-05-19 22:00:02,629 - social_media_monitor.utils.logging_config - INFO - Downloads log file: /mnt/data/openclaw-shared/home/shared/Social_Media_Monitor/logs/social_media_monitor_downloads_2026-05-19.log

2026-05-19 22:00:02,629 - social_media_monitor.utils.logging_config - INFO - Logging configured with level INFO, log file: /mnt/data/openclaw-shared/home/shared/Social_Media_Monitor/logs/social_media_monitor_2026-05-19.log

2026-05-19 22:00:02,629 - __main__ - INFO - === Social Media Monitor v2.0 (Async) Started ===

2026-05-19 22:00:02,652 - social_media_monitor.config.settings - INFO - Configuration loaded from /home/openclaw/Social_Media_Monitor/config.yaml

2026-05-19 22:00:02,652 - __main__ - INFO - Configuration loaded and validated successfully

2026-05-19 22:00:02,654 - social_media_monitor.database.connection - INFO - Database connection pool initialized with 5 connections

2026-05-19 22:00:02,793 - social_media_monitor.database.connection - INFO - add_speaker_filters column already exists

2026-05-19 22:00:02,806 - social_media_monitor.database.connection - INFO - Database setup completed successfully

2026-05-19 22:00:02,806 - __main__ - INFO - Database initialized at: /mnt/data/openclaw-shared/home/shared/Social_Media_Monitor/social_media_monitor.db

2026-05-19 22:00:02,808 - social_media_monitor.scrapers.reddit_scraper - INFO - Reddit API initialized successfully

2026-05-19 22:00:02,809 - RSSFeedScraper - INFO - RSS scraper initialized: feeds=2, html_urls=0 (0 with custom selectors), include_terms=1, exclude_terms=0

2026-05-19 22:00:02,810 - __main__ - INFO - RSS scraper initialized and enabled

2026-05-19 22:00:02,815 - Yiddish24Scraper - INFO - File downloader initialized with 26 terms, prefix 'Y2', and file type organization enabled

2026-05-19 22:00:02,815 - Yiddish24Scraper - INFO - Yiddish24 scraper initialized: data_id='latest', page_limit=100, total_pages=10, debug_html=True, include_terms=26, exclude_terms=0

2026-05-19 22:00:02,815 - __main__ - INFO - Yiddish24 scraper initialized and enabled

2026-05-19 22:00:02,838 - AhBlickLiveScraper - INFO - AhBlickLive scraper initialized: urls=1, max_items=100, include_terms=1, exclude_terms=0

2026-05-19 22:00:02,846 - TorahAnytimeScraper - INFO - File downloader initialized with 1 terms, prefix 'TA', and file type organization enabled

2026-05-19 22:00:02,846 - TorahAnytimeScraper - INFO - TorahAnytime scraper initialized: limit=400, offset=0, project_id=1, include_terms=1, exclude_terms=0

2026-05-19 22:00:02,850 - BatorahScraper - INFO - File downloader initialized with 1 terms, prefix 'BT', and file type organization enabled

2026-05-19 22:00:02,850 - BatorahScraper - INFO - Batorah scraper initialized: api_urls=2, page_size=100, include_terms=1, exclude_terms=0

2026-05-19 22:00:02,850 - KolHalashonScraper - INFO - Kol Halashon scraper initialized: base_url=https://www2.kolhalashon.com, max_shiurim=50, fetch_all_speakers=True, headless=False, exclude_filters=1, include_filters=0

2026-05-19 22:00:02,850 - __main__ - INFO - Ivelt scraper initialized and enabled

2026-05-19 22:00:02,850 - __main__ - INFO - Running 3 scrapers concurrently...

2026-05-19 22:00:02,850 - Yiddish24Scraper - INFO - Starting Yiddish24 scraping (max 10 pages, limit 100 items)

2026-05-19 22:00:02,931 - Yiddish24Scraper - INFO - Fetching Yiddish24 page 1 with data_id='latest', page_limit=100

2026-05-19 22:00:02,935 - IveltScraper - INFO - Loaded 178 include, 0 exclude filters from DB topic_filters

2026-05-19 22:00:02,935 - IveltScraper - INFO - Fetching ivelt.com active topics...

2026-05-19 22:00:02,936 - RSSFeedScraper - INFO - Fetching RSS feed: https://www.theyeshivaworld.com/feed

2026-05-19 22:00:03,270 - RSSFeedScraper - INFO - Fetching RSS feed: https://www.thegatewaypundit.com/feed/

2026-05-19 22:00:03,478 - KolHalashonScraper - INFO - Generic dialog found with 1 button(s)

2026-05-19 22:00:03,495 - KolHalashonScraper - INFO - Using first generic button: ''

2026-05-19 22:00:03,716 - RSSFeedScraper - INFO - Total RSS/HTML results: 4

2026-05-19 22:00:03,822 - KolHalashonScraper - WARNING - No dialog found to handle

2026-05-19 22:00:03,822 - KolHalashonScraper - INFO - Retrying download button click for 41492370 after dialog handling attempt

2026-05-19 22:00:13,140 - KolHalashonScraper - INFO - Generic dialog found with 1 button(s)

2026-05-19 22:00:13,163 - KolHalashonScraper - INFO - Using first generic button: ''

2026-05-19 22:00:13,493 - KolHalashonScraper - WARNING - No dialog found to handle

2026-05-19 22:00:13,493 - KolHalashonScraper - WARNING - [p1:0] Failed to download: 57:17 Harav Elazar Nissan Rubin Masechet Meilah, Daf 17 Shiu by Unknown (ID: 41492370) reason=dialog_fail

2026-05-19 22:00:14,493 - KolHalashonScraper - INFO - [p1:1] Skip (not_requested_or_already_downloaded): 30:38 Harav Shimon Spitzer דף יז, Masechet Meilah, Daf 17 Sh by Unknown (ID: 1496771)

2026-05-19 22:00:14,494 - KolHalashonScraper - INFO - [p1:2] Attempting download: 55:12 Harav Shmuel Dovid Friedman Masechet Meilah, Daf 17 Sh by Unknown (ID: 41215690)

2026-05-19 22:00:14,504 - IveltScraper - INFO - Found 50 active topics on ivelt.com

2026-05-19 22:00:14,508 - IveltScraper - INFO - Ivelt scrape complete. Checked 0 topics, found 0 new posts.

2026-05-19 22:00:15,687 - KolHalashonScraper - INFO - Download button clicked via ActionChains for shiur 41215690

2026-05-19 22:00:15,687 - KolHalashonScraper - INFO - [DELAY] Waiting 1s after download button click for dialog to appear (implicit stabilization)

2026-05-19 22:00:23,444 - KolHalashonScraper - INFO - Generic dialog found with 1 button(s)

2026-05-19 22:00:23,467 - KolHalashonScraper - INFO - Using first generic button: ''

2026-05-19 22:00:23,793 - KolHalashonScraper - WARNING - No dialog found to handle

2026-05-19 22:00:23,793 - KolHalashonScraper - INFO - Retrying download button click for 41215690 after dialog handling attempt

2026-05-19 22:00:33,131 - KolHalashonScraper - INFO - Generic dialog found with 1 button(s)

2026-05-19 22:00:33,153 - KolHalashonScraper - INFO - Using first generic button: ''

2026-05-19 22:00:33,484 - KolHalashonScraper - WARNING - No dialog found to handle

2026-05-19 22:00:33,484 - KolHalashonScraper - WARNING - [p1:2] Failed to download: 55:12 Harav Shmuel Dovid Friedman Masechet Meilah, Daf 17 Sh by Unknown (ID: 41215690) reason=dialog_fail

2026-05-19 22:00:33,785 - Yiddish24Scraper - ERROR - Error fetching Yiddish24 page 1:

2026-05-19 22:00:33,794 - Yiddish24Scraper - ERROR - Traceback (most recent call last):

File "/home/openclaw/Social_Media_Monitor/venv/lib/python3.14/site-packages/httpx/_transports/default.py", line 101, in map_httpcore_exceptions

yield

File "/home/openclaw/Social_Media_Monitor/venv/lib/python3.14/site-packages/httpx/_transports/default.py", line 394, in handle_async_request

resp = await self._pool.handle_async_request(req)

^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

File "/home/openclaw/Social_Media_Monitor/venv/lib/python3.14/site-packages/httpcore/_async/connection_pool.py", line 256, in handle_async_request

raise exc from None

File "/home/openclaw/Social_Media_Monitor/venv/lib/python3.14/site-packages/httpcore/_async/connection_pool.py", line 236, in handle_async_request

response = await connection.handle_async_request(

^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

pool_request.request

^^^^^^^^^^^^^^^^^^^^

)

File "/home/openclaw/Social_Media_Monitor/venv/lib/python3.14/site-packages/httpcore/_async/connection.py", line 103, in handle_async_request

return await self._connection.handle_async_request(request)

^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

File "/home/openclaw/Social_Media_Monitor/venv/lib/python3.14/site-packages/httpcore/_async/http11.py", line 136, in handle_async_request

raise exc

File "/home/openclaw/Social_Media_Monitor/venv/lib/python3.14/site-packages/httpcore/_async/http11.py", line 106, in handle_async_request

) = await self._receive_response_headers(**kwargs)

^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

File "/home/openclaw/Social_Media_Monitor/venv/lib/python3.14/site-packages/httpcore/_async/http11.py", line 177, in _receive_response_headers

event = await self._receive_event(timeout=timeout)

^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

File "/home/openclaw/Social_Media_Monitor/venv/lib/python3.14/site-packages/httpcore/_async/http11.py", line 217, in _receive_event

data = await self._network_stream.read(

^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

self.READ_NUM_BYTES, timeout=timeout

^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

)

File "/home/openclaw/Social_Media_Monitor/venv/lib/python3.14/site-packages/httpcore/_backends/anyio.py", line 32, in read

with map_exceptions(exc_map):

~~~~~~~~~~~~~~^^^^^^^^^

File "/home/linuxbrew/.linuxbrew/opt/python@3.14/lib/python3.14/contextlib.py", line 162, in __exit__

self.gen.throw(value)

~~~~~~~~~~~~~~^^^^^^^

File "/home/openclaw/Social_Media_Monitor/venv/lib/python3.14/site-packages/httpcore/_exceptions.py", line 14, in map_exceptions

raise to_exc(exc) from exc

httpcore.ReadTimeout

The above exception was the direct cause of the following exception:

Traceback (most recent call last):

File "/home/openclaw/Social_Media_Monitor/src/social_media_monitor/scrapers/yiddish24_scraper.py", line 207, in _fetch_page

response = await self.make_request(

^^^^^^^^^^^^^^^^^^^^^^^^

...<5 lines>...

)

File "/home/openclaw/Social_Media_Monitor/src/social_media_monitor/scrapers/base_scraper.py", line 231, in make_request

response = await session.post(url, **kwargs)

^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

File "/home/openclaw/Social_Media_Monitor/venv/lib/python3.14/site-packages/httpx/_client.py", line 1859, in post

return await self.request(

^^^^^^^^^^^^^^^^^^^

...<13 lines>...

)

File "/home/openclaw/Social_Media_Monitor/venv/lib/python3.14/site-packages/httpx/_client.py", line 1540, in request

return await self.send(request, auth=auth, follow_redirects=follow_redirects)

^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

File "/home/openclaw/Social_Media_Monitor/venv/lib/python3.14/site-packages/httpx/_client.py", line 1629, in send

response = await self._send_handling_auth(

^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

...<4 lines>...

)

File "/home/openclaw/Social_Media_Monitor/venv/lib/python3.14/site-packages/httpx/_client.py", line 1657, in _send_handling_auth

response = await self._send_handling_redirects(

^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

...<3 lines>...

)

File "/home/openclaw/Social_Media_Monitor/venv/lib/python3.14/site-packages/httpx/_client.py", line 1694, in _send_handling_redirects

response = await self._send_single_request(request)

^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

File "/home/openclaw/Social_Media_Monitor/venv/lib/python3.14/site-packages/httpx/_client.py", line 1730, in _send_single_request

response = await transport.handle_async_request(request)

^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

File "/home/openclaw/Social_Media_Monitor/venv/lib/python3.14/site-packages/httpx/_transports/default.py", line 393, in handle_async_request

with map_httpcore_exceptions():

~~~~~~~~~~~~~~~~~~~~~~~^^

File "/home/linuxbrew/.linuxbrew/opt/python@3.14/lib/python3.14/contextlib.py", line 162, in __exit__

self.gen.throw(value)

~~~~~~~~~~~~~~^^^^^^^

File "/home/openclaw/Social_Media_Monitor/venv/lib/python3.14/site-packages/httpx/_transports/default.py", line 118, in map_httpcore_exceptions

raise mapped_exc(message) from exc

httpx.ReadTimeout

2026-05-19 22:00:33,794 - Yiddish24Scraper - INFO - Page 1: 0/0 new items

2026-05-19 22:00:33,795 - Yiddish24Scraper - INFO - No more results found at page 1, stopping pagination

2026-05-19 22:00:33,795 - Yiddish24Scraper - INFO - Total Yiddish24 results found: 0

2026-05-19 22:00:33,795 - __main__ - INFO - Scraper rss completed: 4 results

2026-05-19 22:00:33,795 - __main__ - INFO - Scraper yiddish24 completed: 0 results

2026-05-19 22:00:33,795 - __main__ - INFO - Scraper ivelt completed: 0 results

2026-05-19 22:00:34,359 - social_media_monitor.email.sender - INFO - Email sent successfully at 2026-05-19 22:00:34.359632

2026-05-19 22:00:34,359 - __main__ - INFO - Email sent successfully

2026-05-19 22:00:34,360 - __main__ - INFO - Cache stats: 0 items in memory, 0 files on disk

2026-05-19 22:00:34,360 - __main__ - INFO - Monitor complete. Found 4 total results: 4 new rss results.

2026-05-19 22:00:34,360 - __main__ - INFO - === Social Media Monitor v2.0 Complete ===

2026-05-19 22:00:34,360 - social_media_monitor.database.connection - INFO - Closing all database connections

2026-05-19 22:00:34,361 - social_media_monitor.database.connection - INFO - Closed 5 database connections

2026-05-19 22:00:34,361 - __main__ - INFO - Closed all database connections

2026-05-19 22:00:34,362 - social_media_monitor.database.connection - INFO - Closing all database connections

2026-05-19 22:00:34,362 - social_media_monitor.database.connection - INFO - Closed 0 database connections

2026-05-19 22:00:34,485 - KolHalashonScraper - INFO - [p1:3] Attempting download: 57:18 Harav Avraham Loifer דף יז, Masechet Meilah, Daf 17 Sh by Unknown (ID: 1489626)

2026-05-19 22:00:35,661 - KolHalashonScraper - INFO - Download button clicked via ActionChains for shiur 1489626

2026-05-19 22:00:35,661 - KolHalashonScraper - INFO - [DELAY] Waiting 1s after download button click for dialog to appear (implicit stabilization)

2026-05-19 22:00:43,407 - KolHalashonScraper - INFO - Generic dialog found with 1 button(s)

2026-05-19 22:00:43,430 - KolHalashonScraper - INFO - Using first generic button: ''

2026-05-19 22:00:43,759 - KolHalashonScraper - WARNING - No dialog found to handle

2026-05-19 22:00:43,759 - KolHalashonScraper - INFO - Retrying download button click for 1489626 after dialog handling attempt

2026-05-19 22:00:53,094 - KolHalashonScraper - INFO - Generic dialog found with 1 button(s)

2026-05-19 22:00:53,116 - KolHalashonScraper - INFO - Using first generic button: ''

2026-05-19 22:00:53,441 - KolHalashonScraper - WARNING - No dialog found to handle

2026-05-19 22:00:53,442 - KolHalashonScraper - WARNING - [p1:3] Failed to download: 57:18 Harav Avraham Loifer דף יז, Masechet Meilah, Daf 17 Sh by Unknown (ID: 1489626) reason=dialog_fail

2026-05-19 22:00:54,442 - KolHalashonScraper - INFO - [p1:4] Skip (not_requested_or_already_downloaded): 55:50 Harav Elazar Nissan Rubin Masechet Meilah, Daf 17 Shiu by Unknown (ID: 41230878)

2026-05-19 22:00:54,442 - KolHalashonScraper - INFO - [p1:5] Skip (not_requested_or_already_downloaded): 55:06 Harav Avraham Loifer דף יז, Masechet Meilah, Daf 17 Sh by Unknown (ID: 1489625)

2026-05-19 22:00:54,442 - KolHalashonScraper - INFO - [p1:6] Skip (not_requested_or_already_downloaded): 55:14 Harav Shmuel Dovid Friedman דף יז, Masechet Meilah, Da by Unknown (ID: 353519)

2026-05-19 22:00:54,442 - KolHalashonScraper - INFO - reuse_current_page=True -> stopping downloads after the current page

2026-05-19 22:00:54,442 - KolHalashonScraper - INFO - Processed 1 page(s), saw 7 unique shiur IDs

2026-05-19 22:00:54,442 - KolHalashonScraper - INFO - Download complete: 0 downloaded, 3 failed, 4 skipped

2026-05-19 22:00:54,443 - KolHalashonScraper - INFO - Page 1 downloads: 0 downloaded, 3 failed

2026-05-19 22:00:54,450 - KolHalashonScraper - INFO - No more pages available after page 1

2026-05-19 22:00:54,450 - KolHalashonScraper - INFO - URL complete: 7 shiurim extracted from 1 page(s), 63 new total so far

2026-05-19 22:00:55,090 - KolHalashonScraper - INFO - [DELAY] Waiting 0.7s for filter sections to expand (filter_expand_wait)

2026-05-19 22:00:55,833 - KolHalashonScraper - INFO - [DELAY] Waiting 0.7s after clicking 'display more' buttons (filter_expand_wait)

2026-05-19 22:00:56,547 - KolHalashonScraper - INFO - Expanded filter sections: clicked 1 'display more' buttons

2026-05-19 22:00:57,558 - KolHalashonScraper - INFO - DOM filter debug: {'displayAllSections': 1, 'filterContents': 2, 'filterOptions': 248, 'totalCheckboxes': 242, 'type2Checkboxes': 240}

2026-05-19 22:00:57,598 - KolHalashonScraper - INFO - Extracted 240 speakers from DOM filter checkboxes

2026-05-19 22:00:57,598 - KolHalashonScraper - INFO - Extracted 240 speakers from URL filter dropdown

2026-05-19 22:00:57,614 - social_media_monitor.scrapers.kolhalashon_scraper - INFO - Speaker sync complete: 0 new, 240 updated, 520 total

2026-05-19 22:00:57,615 - KolHalashonScraper - INFO - [DELAY] Waiting 1.0s between URLs (shiur_url_delay)

2026-05-19 22:00:58,615 - KolHalashonScraper - INFO - Multi-URL scrape complete: 63 new shiurim from 3 URLs (4 total pages), 63 total checked

2026-05-19 22:00:58,615 - KolHalashonScraper - INFO - [parsha:1/63] match | ID: 42452357 | דרשה בענין לימוד התורה נחיצות הלימוד עמילות התורה הצלחת התורה קודם שבועות פרשת נ | Speaker: Harav Shmuel Shmelke Leifer Zt"l

2026-05-19 22:00:58,616 - KolHalashonScraper - INFO - [parsha:2/63] match | ID: 41749768 | Parashat Noso, Sefer Ohr Hachaim, Part 1 | Speaker: Harav Elazar Nissan Rubin

2026-05-19 22:00:58,616 - KolHalashonScraper - INFO - [parsha:3/63] match | ID: 41747695 | Parashat Noso | Speaker: Harav Chaim Shlomo Miller

2026-05-19 22:00:58,616 - KolHalashonScraper - INFO - [parsha:4/63] match | ID: 41747363 | פרשת נשא תשפ"ה, Parashat Noso | Speaker: Harav Yitzchak Aharon Goldhirsh

2026-05-19 22:00:58,616 - KolHalashonScraper - INFO - [parsha:5/63] match | ID: 41792628 | פרשת נשא - ברכת כהנים, Parashat Noso | Speaker: Harav Chaim Yosef Friedman - Brezhan Rav

2026-05-19 22:00:58,616 - KolHalashonScraper - INFO - [parsha:6/63] match | ID: 41745292 | ענינים על פרשת השבוע פרשת נשא, Parashat Noso | Speaker: Harav Mordechai Yehuda Friedman

2026-05-19 22:00:58,616 - KolHalashonScraper - INFO - [parsha:7/63] match | ID: 41747581 | Parashat Noso, Part 1 | Speaker: Harav Elazar Nissan Rubin

2026-05-19 22:00:58,616 - KolHalashonScraper - INFO - [parsha:8/63] match | ID: 35328041 | ענינים על פרשת השבוע פרשת נשא, Parashat Noso | Speaker: Harav Mordechai Yehuda Friedman

2026-05-19 22:00:58,616 - KolHalashonScraper - INFO - [parsha:9/63] match | ID: 33397452 | ענינים על פרשת השבוע פרשת נשא, Parashat Noso | Speaker: Harav Mordechai Yehuda Friedman

2026-05-19 22:00:58,616 - KolHalashonScraper - INFO - [parsha:10/63] match | ID: 31487668 | ענינים על פרשת השבוע פרשת נשא, Parashat Noso | Speaker: Harav Mordechai Yehuda Friedman

2026-05-19 22:00:58,616 - KolHalashonScraper - INFO - [parsha:11/63] match | ID: 31428120 | ענינים על פרשת השבוע פרשת נשא, Parashat Noso | Speaker: Harav Mordechai Yehuda Friedman

2026-05-19 22:00:58,616 - KolHalashonScraper - INFO - [parsha:12/63] match | ID: 42455226 | הטיפול - כדי להגיע לכיבוד הורים כמצוותו, Parashat Noso, Shavuos | Speaker: Harav Yechezkel Shuvaks

2026-05-19 22:00:58,616 - KolHalashonScraper - INFO - [parsha:13/63] match | ID: 42530442 | פרשת נשא בענין הכנה לשבועות, Parashat Noso, Shavuos | Speaker: Harav Shlomo Levinger

2026-05-19 22:00:58,616 - KolHalashonScraper - INFO - [parsha:14/63] match | ID: 42455150 | יום ג' פרשת נשא (שבוע של שבועות בחול) תשפו - חג השבועות - התפללתי ולא נושעתי בענ | Speaker: Harav Shimon Spitzer

2026-05-19 22:00:58,616 - KolHalashonScraper - INFO - [parsha:15/63] match | ID: 42530171 | פרשת נשא שיעור מסכת עבודה זרה ענין שביעית, Masechet Avoda Zarah, Parashat Noso | Speaker: Harav Shlomo Levinger

2026-05-19 22:00:58,616 - KolHalashonScraper - INFO - [parsha:16/63] match | ID: 42530166 | רב בלייער, Parashat Noso | Speaker: ישיבת מאור עיניים

2026-05-19 22:00:58,616 - KolHalashonScraper - INFO - [parsha:17/63] match | ID: 42454987 | מסירות נפש לתורה מה הוא שיעור פרשת השבוע נשא תשפ"ו, Parashat Noso, Shavuos, Sefe | Speaker: הרב סיני הלברשטאם

2026-05-19 22:00:58,616 - KolHalashonScraper - INFO - [parsha:18/63] match | ID: 42530096 | פרשת נשא, די שייכות פון די י"ג מדות צו די ברכת כהנים, שיטת הנצי"ב., Parashat Nos | Speaker: Harav Aharon Bernstein

2026-05-19 22:00:58,617 - KolHalashonScraper - INFO - [parsha:19/63] match | ID: 42454921 | פרשת נשא - סוד ברכות כהנים, Parashat Noso | Speaker: Harav Chaim Shmerler

2026-05-19 22:00:58,617 - KolHalashonScraper - INFO - [parsha:20/63] match | ID: 42454809 | פרשת נשא - קריאת התורה נפרדת לבני ארץ ישראל ובני חו"ל, Parashat Noso | Speaker: Harav Avraham ben Zion Bekerman

2026-05-19 22:00:58,617 - KolHalashonScraper - INFO - [parsha:21/63] match | ID: 42530024 | נשא, שלושת ימי הגבלה, Parashat Noso | Speaker: ישיבת מאור עיניים

2026-05-19 22:00:58,617 - KolHalashonScraper - INFO - [parsha:22/63] match | ID: 42454804 | הכנה לחג השבועות - כיצד לקבל התורה ולהמשיך בה מפרשת נשא, Parashat Noso, Shavuos | Speaker: Harav Asher Druck

2026-05-19 22:00:58,617 - KolHalashonScraper - INFO - [parsha:23/63] match | ID: 42529702 | יום ג’ פרשת נשא ג’ סיון הכנה לחג השבועות תשפ"ו • וַיָּשֶׁב מֹשֶׁה דִּבְרֵי הָעָם | Speaker: Harav Menashe Yisrael Reizman

2026-05-19 22:00:58,617 - KolHalashonScraper - INFO - [parsha:24/63] match | ID: 42529638 | Parashat Noso, Sefer Chumas Rashi | Speaker: Harav Daniel Chaim Alter

2026-05-19 22:00:58,617 - KolHalashonScraper - INFO - [parsha:25/63] match | ID: 42529306 | שיעור ב' בהלכות חג השבועות סאטמאר פרשת נשא, Parashat Noso, Shavuos | Speaker: Harav Yechezkel Derembaremdicker

2026-05-19 22:00:58,617 - KolHalashonScraper - INFO - [parsha:26/63] match | ID: 42529240 | שיעור ב' הלכות חג השבועות נדבורנה פרשת נשא, Parashat Noso | Speaker: Harav Yechezkel Derembaremdicker

2026-05-19 22:00:58,617 - KolHalashonScraper - INFO - [parsha:27/63] match | ID: 42454381 | Parashat Noso | Speaker: Harav Aharon Toissig

2026-05-19 22:00:58,617 - KolHalashonScraper - INFO - [parsha:28/63] match | ID: 42454420 | בענין סוטה, Parashat Noso | Speaker: Harav Yekutial Yitzchak Pollak

2026-05-19 22:00:58,617 - KolHalashonScraper - INFO - [parsha:29/63] match | ID: 42454415 | 243שנה ו גליון רמג נשא שבועות פו רבי אברהם ראגאלער, Parashat Noso | Speaker: Harav Yochanan Friend

2026-05-19 22:00:58,617 - KolHalashonScraper - INFO - [parsha:30/63] match | ID: 42454334 | מסר לפרשת נשא - המנצח הגדול של הסמפוניה העולמית מחכה לשמוע את התווים שלי, Parash | Speaker: Harav Shlomo Miller from Kiryas Sefer

2026-05-19 22:00:58,617 - KolHalashonScraper - INFO - [parsha:31/63] match | ID: 42454033 | מסר לפרשת נשא - כל יום וההתחדשות שלו, Parashat Noso | Speaker: Harav Avraham ben Zion Bekerman

2026-05-19 22:00:58,617 - KolHalashonScraper - INFO - [parsha:32/63] match | ID: 42528352 | שיעור יום ב' פרשת נשא המשך שליחות יד רבא מ"א עמוד ב', Masechet Bava Metzia, Daf | Speaker: Harav David Rechnitzer

2026-05-19 22:00:58,617 - KolHalashonScraper - INFO - [parsha:33/63] match | ID: 42528325 | פרשת נשא, Parashat Noso | Speaker: Harav Mordechai Avraham Shubaks

2026-05-19 22:00:58,617 - KolHalashonScraper - INFO - [parsha:34/63] match | ID: 42528281 | יום ב' פרשת נשא, Parashat Noso | Speaker: ישיבת מאור עיניים

2026-05-19 22:00:58,617 - KolHalashonScraper - INFO - [parsha:35/63] match | ID: 42528233 | פרשת נשא יארצייט פונעם אהבת ישראל סעיף נ' הלכות בשר בחלב, Parashat Noso, Seif 50 | Speaker: Harav Yosef Yehuda Brown

2026-05-19 22:00:58,617 - KolHalashonScraper - INFO - [parsha:36/63] match | ID: 42453833 | יום ב' פרשת נשא (שבוע של שבועות בחול) תשפו - להסתכל על העולם מיט א העכער'ן בליק, | Speaker: Harav Shimon Spitzer

2026-05-19 22:00:58,617 - KolHalashonScraper - INFO - [parsha:37/63] match | ID: 42528070 | שיעור בענייני חג השבועות, Parashat Noso, Shavuos | Speaker: Harav Shmuel Arye Shapira

2026-05-19 22:00:58,617 - KolHalashonScraper - INFO - [parsha:38/63] match | ID: 42527977 | יום ב’ פרשת נשא הכנה חג השבועות תשפ"ו • יִ’פְרַח בְּ’יָמָיו צַ’דִּיק - הילולת הא | Speaker: Harav Menashe Yisrael Reizman

2026-05-19 22:00:58,618 - KolHalashonScraper - INFO - [parsha:39/63] match | ID: 42527923 | ראשון, Parashat Noso, Sefer Chumas Rashi | Speaker: Harav Daniel Chaim Alter

2026-05-19 22:00:58,618 - KolHalashonScraper - INFO - [parsha:40/63] match | ID: 42453445 | עם אדם ממורמר אין מה לעשות ורק הוא עצמו יכול לעזור לעצמו בהתנערו מכבדות ההתמרמרו | Speaker: Harav Hilel Verthaimer

2026-05-19 22:00:58,618 - KolHalashonScraper - INFO - [parsha:41/63] match | ID: 42453446 | במעלת ה'נשמע' שלאחר אמירת 'נעשה', מתוך לחיים עם רבי מיילעך ר''ח סיון תשפ''ו 2, P | Speaker: Harav Hilel Verthaimer

2026-05-19 22:00:58,618 - KolHalashonScraper - INFO - [parsha:42/63] match | ID: 42453447 | הוספה לרעיון שלכך נוצרת למלא התפקיד המיוחד רק לך, מתוך לחיים עם רבי מיילעך ר''ח | Speaker: Harav Hilel Verthaimer

2026-05-19 22:00:58,618 - KolHalashonScraper - INFO - [parsha:43/63] match | ID: 42453439 | למי מגיע הקרדיט של המופת בשיחרור הפלאי של האסיר, מתוך לחיים עם רבי מיילעך ר''ח ס | Speaker: Harav Hilel Verthaimer

2026-05-19 22:00:58,618 - KolHalashonScraper - INFO - [parsha:44/63] match | ID: 42453441 | מה ענה החסיד לחסיד אחר שהתפאר במעלות רבו, מתוך לחיים עם רבי מיילעך ר''ח סיון תשפ | Speaker: Harav Hilel Verthaimer

2026-05-19 22:00:58,618 - KolHalashonScraper - INFO - [parsha:45/63] match | ID: 42453442 | הרופא לשבורי לב, ועי''ז הוא מחבש לעצבותם, מתוך לחיים עם רבי מיילעך ר''ח סיון תשפ | Speaker: Harav Hilel Verthaimer

2026-05-19 22:00:58,618 - KolHalashonScraper - INFO - [parsha:46/63] match | ID: 42453443 | הזמנה אישית מהקב''ה בכבודו ועצמו, לעומת הזמנה אישית מרבי שמעון, מתוך לחיים עם רב | Speaker: Harav Hilel Verthaimer

2026-05-19 22:00:58,618 - KolHalashonScraper - INFO - [parsha:47/63] match | ID: 42453444 | רבי מיילעך מכריז שבסטנט איילנד אין דברים כאלה כה'מתוקים מדבש' ובגודל ענוותנותו ה | Speaker: Harav Hilel Verthaimer

2026-05-19 22:00:58,618 - KolHalashonScraper - INFO - [parsha:48/63] match | ID: 42526679 | שיעור פרשת נשא סוגיא דשליח עושה שליח, Parashat Noso | Speaker: Harav Meir Dov Rechnitzer

2026-05-19 22:00:58,618 - KolHalashonScraper - INFO - [parsha:49/63] match | ID: 42453148 | והחוט המשולש במהרה לא ינתק, Parashat Noso | Speaker: Harav Aharon Toissig

2026-05-19 22:00:58,618 - KolHalashonScraper - INFO - [parsha:50/63] match | ID: 42453129 | יום א' פרשת נשא (שבוע של שבועות בחול) תשפו - איך עושים שיהיה בכל יום יהיה בעיניך | Speaker: Harav Shimon Spitzer

2026-05-19 22:00:58,618 - KolHalashonScraper - INFO - [parsha:51/63] match | ID: 42526599 | זונטיג נשא ראש חודש סיון, Parashat Noso, Rosh Hodesh | Speaker: ישיבת מאור עיניים

2026-05-19 22:00:58,618 - KolHalashonScraper - INFO - [parsha:52/63] match | ID: 42526575 | פרשת נשא ביום הזה באו מדבר סיני, Parashat Noso | Speaker: ישיבת מאור עיניים

2026-05-19 22:00:58,618 - KolHalashonScraper - INFO - [parsha:53/63] match | ID: 42526398 | פרשת השבוע פרשת נשא ענייני חג השבועות, Parashat Noso, Shavuos | Speaker: Harav Shmuel Arye Shapira

2026-05-19 22:00:58,618 - KolHalashonScraper - INFO - [parsha:54/63] match | ID: 42526325 | יום א’ פרשת נשא הכנה לחג השבועות תשפ"ו • יְ’דוֹתָיו וּ’לְצֶלַע הַ’מִּשְׁכָּן הַ’ | Speaker: Harav Menashe Yisrael Reizman

2026-05-19 22:00:58,618 - KolHalashonScraper - INFO - [parsha:55/63] match | ID: 42452357 | דרשה בענין לימוד התורה נחיצות הלימוד עמילות התורה הצלחת התורה קודם שבועות פרשת נ | Speaker: Harav Shmuel Shmelke Leifer Zt"l

2026-05-19 22:00:58,618 - KolHalashonScraper - INFO - [parsha:56/63] match | ID: 42525025 | פרשת נשא הכנה לחג השבועות, Parashat Noso, Shavuos | Speaker: Rav Schnair Zalman Weiss

2026-05-19 22:00:58,618 - KolHalashonScraper - INFO - [parsha:57/63] match | ID: 41492370 | Masechet Meilah, Daf 17 | Speaker: Harav Elazar Nissan Rubin

2026-05-19 22:00:58,618 - KolHalashonScraper - INFO - [parsha:58/63] match | ID: 1496771 | דף יז, Masechet Meilah, Daf 17 | Speaker: Harav Shimon Spitzer

2026-05-19 22:00:58,619 - KolHalashonScraper - INFO - [parsha:59/63] match | ID: 41215690 | Masechet Meilah, Daf 17 | Speaker: Harav Shmuel Dovid Friedman

2026-05-19 22:00:58,619 - KolHalashonScraper - INFO - [parsha:60/63] match | ID: 1489626 | דף יז, Masechet Meilah, Daf 17 | Speaker: Harav Avraham Loifer

2026-05-19 22:00:58,619 - KolHalashonScraper - INFO - [parsha:61/63] match | ID: 41230878 | Masechet Meilah, Daf 17 | Speaker: Harav Elazar Nissan Rubin

2026-05-19 22:00:58,619 - KolHalashonScraper - INFO - [parsha:62/63] match | ID: 1489625 | דף יז, Masechet Meilah, Daf 17 | Speaker: Harav Avraham Loifer

2026-05-19 22:00:58,619 - KolHalashonScraper - INFO - [parsha:63/63] match | ID: 353519 | דף יז, Masechet Meilah, Daf 17 | Speaker: Harav Shmuel Dovid Friedman

2026-05-19 22:00:58,648 - social_media_monitor.scrapers.kolhalashon_scraper - INFO - Shiurim sync complete: 0 new, 63 already in db, 680 total

2026-05-19 22:00:58,648 - KolHalashonScraper - INFO - Parsha shiurim: 0 new, 63 match filter, 63 total

2026-05-19 22:00:58,649 - KolHalashonScraper - INFO - Kol Halashon scraping complete: 0 NEW shiurim for email, 0 total new, 63 match filter (all)

2026-05-19 22:00:58,649 - __main__ - INFO - Scraper kolhalashon completed: 0 results

2026-05-19 22:00:58,649 - __main__ - INFO - No content to send in email

2026-05-19 22:00:58,649 - __main__ - INFO - Cache stats: 0 items in memory, 0 files on disk

2026-05-19 22:00:58,650 - __main__ - INFO - Monitor complete. No new results found across all sources.

2026-05-19 22:00:58,650 - __main__ - INFO - === Social Media Monitor v2.0 Complete ===

2026-05-19 22:00:58,650 - social_media_monitor.database.connection - INFO - Closing all database connections

2026-05-19 22:00:58,682 - social_media_monitor.database.connection - INFO - Closed 5 database connections

2026-05-19 22:00:58,682 - __main__ - INFO - Closed all database connections

2026-05-19 22:00:58,683 - social_media_monitor.database.connection - INFO - Closing all database connections

2026-05-19 22:00:58,684 - social_media_monitor.database.connection - INFO - Closed 0 database connections

2026-05-19 22:30:01,624 - social_media_monitor.utils.logging_config - INFO - Posts log file: /mnt/data/openclaw-shared/home/shared/Social_Media_Monitor/logs/social_media_monitor_posts_2026-05-19.log

2026-05-19 22:30:01,624 - social_media_monitor.utils.logging_config - INFO - Downloads log file: /mnt/data/openclaw-shared/home/shared/Social_Media_Monitor/logs/social_media_monitor_downloads_2026-05-19.log

2026-05-19 22:30:01,624 - social_media_monitor.utils.logging_config - INFO - Logging configured with level INFO, log file: /mnt/data/openclaw-shared/home/shared/Social_Media_Monitor/logs/social_media_monitor_2026-05-19.log

2026-05-19 22:30:01,624 - __main__ - INFO - === Social Media Monitor v2.0 (Async) Started ===

2026-05-19 22:30:01,649 - social_media_monitor.config.settings - INFO - Configuration loaded from /home/openclaw/Social_Media_Monitor/config.yaml

2026-05-19 22:30:01,649 - social_media_monitor.config.settings - WARNING - Twitter monitoring enabled but credentials validation not implemented

2026-05-19 22:30:01,649 - __main__ - INFO - Configuration loaded and validated successfully

2026-05-19 22:30:01,652 - social_media_monitor.database.connection - INFO - Database connection pool initialized with 5 connections

2026-05-19 22:30:01,652 - social_media_monitor.database.connection - INFO - add_speaker_filters column already exists

2026-05-19 22:30:01,919 - social_media_monitor.database.connection - INFO - Database setup completed successfully

2026-05-19 22:30:01,919 - __main__ - INFO - Database initialized at: /mnt/data/openclaw-shared/home/shared/Social_Media_Monitor/social_media_monitor.db

2026-05-19 22:30:01,978 - social_media_monitor.scrapers.reddit_scraper - INFO - Reddit API initialized successfully

2026-05-19 22:30:01,979 - __main__ - INFO - Reddit scraper initialized and enabled

2026-05-19 22:30:01,979 - __main__ - INFO - GitHub scraper initialized and enabled

2026-05-19 22:30:01,979 - __main__ - INFO - Twitter scraper initialized and enabled

2026-05-19 22:30:01,980 - RSSFeedScraper - INFO - RSS scraper initialized: feeds=2, html_urls=0 (0 with custom selectors), include_terms=1, exclude_terms=0

2026-05-19 22:30:01,980 - __main__ - INFO - LinkedIn scraper initialized and enabled

2026-05-19 22:30:01,980 - __main__ - INFO - LinkedIn Feed scraper initialized and enabled

2026-05-19 22:30:02,001 - Yiddish24Scraper - INFO - File downloader initialized with 26 terms, prefix 'Y2', and file type organization enabled

2026-05-19 22:30:02,001 - Yiddish24Scraper - INFO - Yiddish24 scraper initialized: data_id='latest', page_limit=100, total_pages=10, debug_html=True, include_terms=26, exclude_terms=0

2026-05-19 22:30:02,006 - AhBlickLiveScraper - INFO - AhBlickLive scraper initialized: urls=1, max_items=100, include_terms=1, exclude_terms=0

2026-05-19 22:30:02,012 - TorahAnytimeScraper - INFO - File downloader initialized with 1 terms, prefix 'TA', and file type organization enabled

2026-05-19 22:30:02,012 - TorahAnytimeScraper - INFO - TorahAnytime scraper initialized: limit=400, offset=0, project_id=1, include_terms=1, exclude_terms=0

2026-05-19 22:30:02,014 - BatorahScraper - INFO - File downloader initialized with 1 terms, prefix 'BT', and file type organization enabled

2026-05-19 22:30:02,014 - BatorahScraper - INFO - Batorah scraper initialized: api_urls=2, page_size=100, include_terms=1, exclude_terms=0

2026-05-19 22:30:02,014 - KolHalashonScraper - INFO - Kol Halashon scraper initialized: base_url=https://www2.kolhalashon.com, max_shiurim=50, fetch_all_speakers=True, headless=False, exclude_filters=1, include_filters=0

2026-05-19 22:30:02,015 - __main__ - INFO - Running 5 scrapers concurrently...

2026-05-19 22:30:02,015 - social_media_monitor.scrapers.reddit_scraper - INFO - Reddit search monitoring enabled. Running search with terms: ['paperclip', '"Hermes Agent"', 'multica']

2026-05-19 22:30:02,015 - social_media_monitor.scrapers.reddit_scraper - INFO - Searching Reddit RSS for terms: ['"Hermes Agent"']

2026-05-19 22:30:02,822 - social_media_monitor.scrapers.reddit_scraper - INFO - Reddit RSS: 25 entries for term '"Hermes Agent"'

2026-05-19 22:30:02,855 - social_media_monitor.scrapers.reddit_scraper - INFO - New Reddit post: LocalLlama

2026-05-19 22:30:02,870 - social_media_monitor.scrapers.reddit_scraper - INFO - New Reddit post: Hermes Agent model routing guide: Private AI agent install without the

2026-05-19 22:30:02,881 - social_media_monitor.scrapers.reddit_scraper - INFO - New Reddit post: Teaching non-technical founders to get their first AI agent running —

2026-05-19 22:30:04,266 - social_media_monitor.scrapers.reddit_scraper - INFO - Total Reddit RSS results found: 3

2026-05-19 22:30:04,267 - social_media_monitor.scrapers.reddit_scraper - INFO - Searching 'paperclip' restricted to subreddits: ['PaperClip_AI', 'PaperclipAI']

2026-05-19 22:30:04,267 - social_media_monitor.scrapers.reddit_scraper - INFO - Searching Reddit RSS for terms: ['paperclip']

2026-05-19 22:30:04,267 - social_media_monitor.scrapers.reddit_scraper - INFO - Restricting Reddit search to subreddits: r/PaperClip_AI+PaperclipAI

2026-05-19 22:30:05,396 - social_media_monitor.scrapers.reddit_scraper - INFO - Reddit RSS: 25 entries for term 'paperclip'

2026-05-19 22:30:05,444 - social_media_monitor.scrapers.reddit_scraper - INFO - New Reddit post: Do you need to be a programmer to get the most use out of claude?

2026-05-19 22:30:06,766 - social_media_monitor.scrapers.reddit_scraper - INFO - Total Reddit RSS results found: 1

2026-05-19 22:30:06,767 - social_media_monitor.scrapers.reddit_scraper - INFO - Searching 'multica' restricted to subreddits: ['hermesagent', 'HermesAI']

2026-05-19 22:30:06,767 - social_media_monitor.scrapers.reddit_scraper - INFO - Searching Reddit RSS for terms: ['multica']

2026-05-19 22:30:06,767 - social_media_monitor.scrapers.reddit_scraper - INFO - Restricting Reddit search to subreddits: r/hermesagent+HermesAI

2026-05-19 22:30:07,729 - social_media_monitor.scrapers.reddit_scraper - INFO - Reddit RSS: 25 entries for term 'multica'

2026-05-19 22:30:09,009 - social_media_monitor.scrapers.reddit_scraper - INFO - Total Reddit RSS results found: 0

2026-05-19 22:30:09,009 - social_media_monitor.scrapers.reddit_scraper - INFO - Reddit subreddit feed monitoring enabled.

2026-05-19 22:30:09,009 - social_media_monitor.scrapers.reddit_scraper - INFO - Monitoring feed 'hermes_agent': r/hermesagent+HermesAI+AI_Agents+aiagents+AgentsOfAI+LocalLLaMA+AutoGPT (limit=25)

2026-05-19 22:30:09,009 - social_media_monitor.scrapers.reddit_scraper - INFO - Searching feed 'hermes_agent' for: hermes agent via RSS

2026-05-19 22:30:11,204 - social_media_monitor.scrapers.reddit_scraper - INFO - Searching feed 'hermes_agent' for: hermes AI via RSS

2026-05-19 22:30:13,226 - social_media_monitor.scrapers.reddit_scraper - INFO - Searching feed 'hermes_agent' for: nous research hermes via RSS

2026-05-19 22:30:14,402 - social_media_monitor.scrapers.reddit_scraper - INFO - New from RSS feed 'hermes_agent' r/Hermeticism: I don’t know if this has a spiritual connection, but I wanted opinions

2026-05-19 22:30:14,471 - social_media_monitor.scrapers.reddit_scraper - INFO - New from RSS feed 'hermes_agent' r/JapanTravelTips: Vintage Shopping in Japan

2026-05-19 22:30:15,666 - social_media_monitor.scrapers.reddit_scraper - INFO - Searching feed 'hermes_agent' for: multica via RSS

2026-05-19 22:30:17,676 - social_media_monitor.scrapers.reddit_scraper - INFO - Monitoring feed 'paperclip_ai': r/PaperClip_AI+PaperclipAI+pAIperclip (limit=25)

2026-05-19 22:30:17,677 - social_media_monitor.scrapers.reddit_scraper - INFO - Searching feed 'paperclip_ai' for: paperclip ai via RSS

2026-05-19 22:30:20,284 - social_media_monitor.scrapers.reddit_scraper - INFO - Searching feed 'paperclip_ai' for: paperclip agent via RSS

2026-05-19 22:30:22,393 - social_media_monitor.scrapers.reddit_scraper - INFO - Searching feed 'paperclip_ai' for: multica via RSS

2026-05-19 22:30:24,485 - social_media_monitor.scrapers.reddit_scraper - INFO - Monitoring feed 'claude_agents': r/vibecoding (limit=25)

2026-05-19 22:30:24,485 - social_media_monitor.scrapers.reddit_scraper - INFO - Searching feed 'claude_agents' for: multica via RSS

2026-05-19 22:30:26,482 - social_media_monitor.scrapers.reddit_scraper - INFO - Total subreddit feed results: 2

2026-05-19 22:30:26,482 - GitHubScraper - INFO - Searching GitHub for terms: ['paperclip', '"Hermes Agent"', 'multica']

2026-05-19 22:30:26,654 - TwitterScraper - INFO - Searching Twitter/X (Google News RSS) for: ['paperclip', '"Hermes Agent"', 'multica']

2026-05-19 22:30:27,136 - TwitterScraper - INFO - Twitter/X RSS: 100 entries for 'paperclip'

2026-05-19 22:30:43,651 - TwitterScraper - INFO - Twitter/X RSS: 100 entries for '"Hermes Agent"'

2026-05-19 22:30:59,982 - TwitterScraper - INFO - Twitter/X RSS: 100 entries for 'multica'

2026-05-19 22:31:15,829 - TwitterScraper - INFO - Twitter scrape complete. Found 0 new results.

2026-05-19 22:31:15,830 - LinkedInScraper - INFO - Searching LinkedIn (Google News RSS) for: ['paperclip', '"Hermes Agent"', 'multica']

2026-05-19 22:31:16,448 - LinkedInScraper - INFO - LinkedIn RSS: 100 entries for 'paperclip'

2026-05-19 22:31:16,539 - LinkedInScraper - INFO - New LinkedIn post: My autonomous AI agent just pushed an unauthorised update. Then it apo

2026-05-19 22:31:17,933 - LinkedInScraper - INFO - LinkedIn RSS: 100 entries for '"Hermes Agent"'

2026-05-19 22:31:18,169 - LinkedInScraper - INFO - New LinkedIn post: "Should I use Claude Code or Hermes Agent?" Wrong question. They ship

2026-05-19 22:31:18,377 - LinkedInScraper - INFO - New LinkedIn post: I wrote a short blog post on Hermes Agent and how I'm using it. Fantas

2026-05-19 22:31:18,518 - LinkedInScraper - INFO - New LinkedIn post: I ranked a video on Google in 19 hours with Hermes Agent… Here’s how👇

2026-05-19 22:31:18,604 - LinkedInScraper - INFO - New LinkedIn post: Hermes Agent passed OpenClaw in total tokens on OpenRouter this week.

2026-05-19 22:31:18,704 - LinkedInScraper - INFO - New LinkedIn post: AI Breakthrough: Hermes Agent Learns and Improves - LinkedIn

2026-05-19 22:31:19,288 - LinkedInScraper - INFO - LinkedIn RSS: 100 entries for 'multica'

2026-05-19 22:31:20,043 - LinkedInScraper - INFO - LinkedIn scrape complete. Found 6 new results.

2026-05-19 22:31:20,044 - LinkedInFeedScraper - INFO - Fetching LinkedIn feed (25 posts)

2026-05-19 22:31:26,169 - httpx - INFO - HTTP Request: GET https://api.github.com/search/repositories?q=paperclip+in%3Aname%2Cdescription%2Creadme+stars%3A%3E150&sort=updated&order=desc&per_page=20 "HTTP/1.1 200 OK"

2026-05-19 22:31:26,194 - GitHubScraper - INFO - Found 227 GitHub repositories for term: paperclip

2026-05-19 22:31:27,291 - httpx - INFO - HTTP Request: GET https://api.github.com/search/repositories?q=%22Hermes+Agent%22+in%3Aname%2Cdescription%2Creadme+stars%3A%3E150&sort=updated&order=desc&per_page=20 "HTTP/1.1 200 OK"

2026-05-19 22:31:27,294 - GitHubScraper - INFO - Found 251 GitHub repositories for term: "Hermes Agent"

2026-05-19 22:31:28,410 - httpx - INFO - HTTP Request: GET https://api.github.com/search/repositories?q=multica+in%3Aname%2Cdescription%2Creadme+stars%3A%3E150&sort=updated&order=desc&per_page=20 "HTTP/1.1 200 OK"

2026-05-19 22:31:28,414 - GitHubScraper - INFO - Found 46 GitHub repositories for term: multica

2026-05-19 22:31:28,585 - GitHubScraper - INFO - Total GitHub results found: 0

2026-05-19 22:31:28,585 - __main__ - INFO - Scraper reddit completed: 6 results

2026-05-19 22:31:28,585 - __main__ - INFO - Scraper github completed: 0 results

2026-05-19 22:31:28,585 - __main__ - INFO - Scraper twitter completed: 0 results

2026-05-19 22:31:28,585 - __main__ - INFO - Scraper linkedin completed: 6 results

2026-05-19 22:31:28,586 - __main__ - INFO - Scraper linkedin_feed completed: 0 results

2026-05-19 22:31:29,439 - social_media_monitor.email.sender - INFO - Email sent successfully at 2026-05-19 22:31:29.439151

2026-05-19 22:31:29,439 - __main__ - INFO - Email sent successfully

2026-05-19 22:31:29,439 - __main__ - INFO - Cache stats: 0 items in memory, 0 files on disk

2026-05-19 22:31:29,439 - __main__ - INFO - Monitor complete. Found 12 total results: 6 new reddit results, 6 new linkedin results.

2026-05-19 22:31:29,440 - __main__ - INFO - === Social Media Monitor v2.0 Complete ===

2026-05-19 22:31:29,440 - social_media_monitor.database.connection - INFO - Closing all database connections

2026-05-19 22:31:29,563 - social_media_monitor.database.connection - INFO - Closed 5 database connections

2026-05-19 22:31:29,563 - __main__ - INFO - Closed all database connections

2026-05-19 22:31:29,605 - social_media_monitor.database.connection - INFO - Closing all database connections

2026-05-19 22:31:29,623 - social_media_monitor.database.connection - INFO - Closed 0 database connections

2026-05-19 23:00:02,452 - social_media_monitor.utils.logging_config - INFO - Posts log file: /mnt/data/openclaw-shared/home/shared/Social_Media_Monitor/logs/social_media_monitor_posts_2026-05-19.log

2026-05-19 23:00:02,452 - social_media_monitor.utils.logging_config - INFO - Downloads log file: /mnt/data/openclaw-shared/home/shared/Social_Media_Monitor/logs/social_media_monitor_downloads_2026-05-19.log

2026-05-19 23:00:02,452 - social_media_monitor.utils.logging_config - INFO - Logging configured with level INFO, log file: /mnt/data/openclaw-shared/home/shared/Social_Media_Monitor/logs/social_media_monitor_2026-05-19.log

2026-05-19 23:00:02,452 - __main__ - INFO - === Social Media Monitor v2.0 (Async) Started ===

2026-05-19 23:00:02,475 - social_media_monitor.config.settings - INFO - Configuration loaded from /home/openclaw/Social_Media_Monitor/config.yaml

2026-05-19 23:00:02,475 - __main__ - INFO - Configuration loaded and validated successfully

2026-05-19 23:00:02,478 - social_media_monitor.database.connection - INFO - Database connection pool initialized with 5 connections

2026-05-19 23:00:02,478 - social_media_monitor.database.connection - INFO - add_speaker_filters column already exists

2026-05-19 23:00:02,644 - social_media_monitor.database.connection - INFO - Database setup completed successfully

2026-05-19 23:00:02,644 - __main__ - INFO - Database initialized at: /mnt/data/openclaw-shared/home/shared/Social_Media_Monitor/social_media_monitor.db

2026-05-19 23:00:02,646 - social_media_monitor.scrapers.reddit_scraper - INFO - Reddit API initialized successfully

2026-05-19 23:00:02,648 - RSSFeedScraper - INFO - RSS scraper initialized: feeds=2, html_urls=0 (0 with custom selectors), include_terms=1, exclude_terms=0

2026-05-19 23:00:02,648 - __main__ - INFO - RSS scraper initialized and enabled

2026-05-19 23:00:02,655 - Yiddish24Scraper - INFO - File downloader initialized with 26 terms, prefix 'Y2', and file type organization enabled

2026-05-19 23:00:02,655 - Yiddish24Scraper - INFO - Yiddish24 scraper initialized: data_id='latest', page_limit=100, total_pages=10, debug_html=True, include_terms=26, exclude_terms=0

2026-05-19 23:00:02,655 - __main__ - INFO - Yiddish24 scraper initialized and enabled

2026-05-19 23:00:02,685 - AhBlickLiveScraper - INFO - AhBlickLive scraper initialized: urls=1, max_items=100, include_terms=1, exclude_terms=0

2026-05-19 23:00:02,692 - TorahAnytimeScraper - INFO - File downloader initialized with 1 terms, prefix 'TA', and file type organization enabled

2026-05-19 23:00:02,692 - TorahAnytimeScraper - INFO - TorahAnytime scraper initialized: limit=400, offset=0, project_id=1, include_terms=1, exclude_terms=0

2026-05-19 23:00:02,695 - BatorahScraper - INFO - File downloader initialized with 1 terms, prefix 'BT', and file type organization enabled

2026-05-19 23:00:02,695 - BatorahScraper - INFO - Batorah scraper initialized: api_urls=2, page_size=100, include_terms=1, exclude_terms=0

2026-05-19 23:00:02,695 - KolHalashonScraper - INFO - Kol Halashon scraper initialized: base_url=https://www2.kolhalashon.com, max_shiurim=50, fetch_all_speakers=True, headless=False, exclude_filters=1, include_filters=0

2026-05-19 23:00:02,695 - __main__ - INFO - Ivelt scraper initialized and enabled

2026-05-19 23:00:02,695 - __main__ - INFO - Running 3 scrapers concurrently...

2026-05-19 23:00:02,695 - Yiddish24Scraper - INFO - Starting Yiddish24 scraping (max 10 pages, limit 100 items)

2026-05-19 23:00:02,780 - Yiddish24Scraper - INFO - Fetching Yiddish24 page 1 with data_id='latest', page_limit=100

2026-05-19 23:00:02,785 - IveltScraper - INFO - Loaded 178 include, 0 exclude filters from DB topic_filters

2026-05-19 23:00:02,785 - IveltScraper - INFO - Fetching ivelt.com active topics...

2026-05-19 23:00:02,788 - RSSFeedScraper - INFO - Fetching RSS feed: https://www.theyeshivaworld.com/feed

2026-05-19 23:00:03,134 - RSSFeedScraper - INFO - Fetching RSS feed: https://www.thegatewaypundit.com/feed/

2026-05-19 23:00:03,535 - RSSFeedScraper - INFO - Total RSS/HTML results: 3

2026-05-19 23:00:08,716 - httpx - INFO - HTTP Request: POST https://www.yiddish24.com/ajax/cat_pagination.php "HTTP/1.1 200 OK"

2026-05-19 23:00:08,954 - Yiddish24Scraper - INFO - Received response with 477832 characters

2026-05-19 23:00:08,957 - Yiddish24Scraper - INFO - Extracted HTML from JSON result field (323912 chars)

2026-05-19 23:00:08,961 - Yiddish24Scraper - INFO - Saved HTML debug file to: /mnt/data/openclaw-shared/home/shared/Social_Media_Monitor/debug/yiddish24_page1_response.html

2026-05-19 23:00:09,071 - Yiddish24Scraper - INFO - Found 100 potential article elements on page 1

2026-05-19 23:00:09,115 - Yiddish24Scraper - INFO - Page 1 extraction: 100 successful, 0 failed out of 100 elements

2026-05-19 23:00:09,116 - Yiddish24Scraper - INFO - Parsed 100 articles from HTML on page 1

2026-05-19 23:00:09,124 - Yiddish24Scraper - INFO - Term 'הכל בכל' matched in TITLE: 'הכל בכל'

2026-05-19 23:00:09,124 - Yiddish24Scraper - INFO - Downloading 1 media file(s) for article: הכל בכל

2026-05-19 23:00:09,860 - social_media_monitor.utils.file_downloader - INFO - Starting download from: https://cloudfront.yiddish24.com/perlowitz_260_260519065947.mp3

2026-05-19 23:00:10,948 - social_media_monitor.utils.file_downloader - INFO - Successfully downloaded: Y2_perlowitz_260_260519065947.mp3 (5.2 MB)

2026-05-19 23:00:10,949 - social_media_monitor.utils.file_downloader - INFO - Organized audio file to: /mnt/data/openclaw-shared/home/shared/Social_Media_Monitor/downloads/Audio (audio/mpeg)

2026-05-19 23:00:10,965 - social_media_monitor.utils.media_tagger - INFO - Tagged Y2_perlowitz_260_260519065947.mp3: title, artist, album, comment, source_url, genre, publisher

2026-05-19 23:00:10,982 - Yiddish24Scraper - INFO - Term 'הכל בכל' matched in TITLE: 'הכל בכל'

2026-05-19 23:00:10,982 - Yiddish24Scraper - INFO - Downloading 1 media file(s) for article: הכל בכל

2026-05-19 23:00:11,482 - social_media_monitor.utils.file_downloader - INFO - Starting download from: https://cloudfront.yiddish24.com/051926_yiddish_260_260519065505.mp3

2026-05-19 23:00:14,885 - social_media_monitor.utils.file_downloader - INFO - Successfully downloaded: Y2_051926_yiddish_260_260519065505.mp3 (90.9 MB)

2026-05-19 23:00:14,886 - social_media_monitor.utils.file_downloader - INFO - Organized audio file to: /mnt/data/openclaw-shared/home/shared/Social_Media_Monitor/downloads/Audio (audio/mpeg)

2026-05-19 23:00:14,889 - social_media_monitor.utils.media_tagger - INFO - Tagged Y2_051926_yiddish_260_260519065505.mp3: title, artist, album, comment, source_url, genre, publisher

2026-05-19 23:00:14,898 - Yiddish24Scraper - INFO - Term 'הכל בכל' matched in TITLE: 'הכל בכל'

2026-05-19 23:00:14,899 - Yiddish24Scraper - INFO - Downloading 1 media file(s) for article: הכל בכל

2026-05-19 23:00:15,399 - social_media_monitor.utils.file_downloader - INFO - Starting download from: https://cloudfront.yiddish24.com/local_tus_hdifhesv250pbgmmusicm_260_260519065332.mp3

2026-05-19 23:00:16,500 - social_media_monitor.utils.file_downloader - INFO - Successfully downloaded: Y2_local_tus_hdifhesv250pbgmmusicm_260_260519065332.mp3 (4.8 MB)

2026-05-19 23:00:16,501 - social_media_monitor.utils.file_downloader - INFO - Organized audio file to: /mnt/data/openclaw-shared/home/shared/Social_Media_Monitor/downloads/Audio (audio/mpeg)

2026-05-19 23:00:16,514 - social_media_monitor.utils.media_tagger - INFO - Tagged Y2_local_tus_hdifhesv250pbgmmusicm_260_260519065332.mp3: title, artist, album, comment, source_url, genre, publisher

2026-05-19 23:00:16,674 - Yiddish24Scraper - INFO - Term 'הרב' matched in DESCRIPTION: 'הרב גרשון מענדל טויב'

2026-05-19 23:00:16,674 - Yiddish24Scraper - INFO - Downloading 1 media file(s) for article: אינטערוויו: אסיפת מנהלים פון אלע ת‘‘ת פון ניו יארק

2026-05-19 23:00:17,174 - social_media_monitor.utils.file_downloader - INFO - Starting download from: https://cloudfront.yiddish24.com/KM_N114_238_1779226862.mp3

2026-05-19 23:00:17,261 - social_media_monitor.utils.file_downloader - INFO - Successfully downloaded: Y2_KM_N114_238_1779226862.mp3 (4.3 MB)

2026-05-19 23:00:17,261 - social_media_monitor.utils.file_downloader - INFO - Organized audio file to: /mnt/data/openclaw-shared/home/shared/Social_Media_Monitor/downloads/Audio (audio/mpeg)

2026-05-19 23:00:17,268 - social_media_monitor.utils.media_tagger - INFO - Tagged Y2_KM_N114_238_1779226862.mp3: title, artist, album, comment, source_url, genre, publisher

2026-05-19 23:00:17,284 - Yiddish24Scraper - INFO - Term 'הרב' matched in DESCRIPTION: 'הרב עזרא פריעדלאנדלער'

2026-05-19 23:00:17,284 - Yiddish24Scraper - INFO - Downloading 1 media file(s) for article: אינטערוויו: ווייטעררדיגע באריכטן איבער די געפלאנטע

2026-05-19 23:00:18,175 - social_media_monitor.utils.file_downloader - INFO - Starting download from: https://cloudfront.yiddish24.com/KM_N114_237_1779226839.mp3

2026-05-19 23:00:18,224 - social_media_monitor.utils.file_downloader - INFO - Successfully downloaded: Y2_KM_N114_237_1779226839.mp3 (1.5 MB)

2026-05-19 23:00:18,225 - social_media_monitor.utils.file_downloader - INFO - Organized audio file to: /mnt/data/openclaw-shared/home/shared/Social_Media_Monitor/downloads/Audio (audio/mpeg)

2026-05-19 23:00:18,230 - social_media_monitor.utils.media_tagger - INFO - Tagged Y2_KM_N114_237_1779226839.mp3: title, artist, album, comment, source_url, genre, publisher

2026-05-19 23:00:18,874 - Yiddish24Scraper - INFO - Page 1 summary: 14 new out of 100 total articles

2026-05-19 23:00:18,875 - Yiddish24Scraper - INFO - Page 1: 14/100 new items

2026-05-19 23:00:18,875 - Yiddish24Scraper - INFO - Low new item ratio (14.0%), stopping pagination

2026-05-19 23:00:18,876 - Yiddish24Scraper - INFO - Total Yiddish24 results found: 14

2026-05-19 23:00:18,883 - IveltScraper - INFO - Found 50 active topics on ivelt.com

2026-05-19 23:00:37,768 - IveltScraper - INFO - Ivelt scrape complete. Checked 5 topics, found 15 new posts.

2026-05-19 23:00:37,768 - __main__ - INFO - Scraper rss completed: 3 results

2026-05-19 23:00:37,768 - __main__ - INFO - Scraper yiddish24 completed: 14 results

2026-05-19 23:00:37,768 - __main__ - INFO - Scraper ivelt completed: 15 results

2026-05-19 23:00:38,743 - social_media_monitor.email.sender - INFO - Email sent successfully at 2026-05-19 23:00:38.743862

2026-05-19 23:00:38,744 - __main__ - INFO - Email sent successfully

2026-05-19 23:00:38,744 - __main__ - INFO - Cache stats: 0 items in memory, 0 files on disk

2026-05-19 23:00:38,744 - __main__ - INFO - Monitor complete. Found 32 total results: 3 new rss results, 14 new yiddish24 results, 15 new ivelt results.

2026-05-19 23:00:38,744 - __main__ - INFO - === Social Media Monitor v2.0 Complete ===

2026-05-19 23:00:38,744 - social_media_monitor.database.connection - INFO - Closing all database connections

2026-05-19 23:00:39,083 - social_media_monitor.database.connection - INFO - Closed 5 database connections

2026-05-19 23:00:39,084 - __main__ - INFO - Closed all database connections

2026-05-19 23:00:39,085 - social_media_monitor.database.connection - INFO - Closing all database connections

2026-05-19 23:00:39,086 - social_media_monitor.database.connection - INFO - Closed 0 database connections