โ Logs
social_media_monitor_2026-05-17.log
2026-05-17 23:02:16,331 - Yiddish24Scraper - INFO - File downloader initialized with 1 terms, prefix 'Y2', and file type organization enabled
2026-05-17 23:02:16,332 - Yiddish24Scraper - INFO - Yiddish24 scraper initialized: data_id='latest', page_limit=100, total_pages=10, debug_html=True, include_terms=1, exclude_terms=0
2026-05-17 23:02:16,347 - AhBlickLiveScraper - INFO - AhBlickLive scraper initialized: urls=1, max_items=100, include_terms=1, exclude_terms=0
2026-05-17 23:02:16,351 - TorahAnytimeScraper - INFO - File downloader initialized with 1 terms, prefix 'TA', and file type organization enabled
2026-05-17 23:02:16,351 - TorahAnytimeScraper - INFO - TorahAnytime scraper initialized: limit=400, offset=0, project_id=1, include_terms=1, exclude_terms=0
2026-05-17 23:02:16,353 - BatorahScraper - INFO - File downloader initialized with 1 terms, prefix 'BT', and file type organization enabled
2026-05-17 23:02:16,354 - BatorahScraper - INFO - Batorah scraper initialized: api_urls=2, page_size=100, include_terms=1, exclude_terms=0
2026-05-17 23:02:16,354 - KolHalashonScraper - INFO - Kol Halashon scraper initialized: base_url=https://www2.kolhalashon.com, max_shiurim=50, fetch_all_speakers=True, headless=False, exclude_filters=1, include_filters=0
2026-05-17 23:02:16,354 - __main__ - INFO - Kol Halashon scraper initialized and enabled
2026-05-17 23:02:16,354 - __main__ - INFO - Running 1 scrapers concurrently...
2026-05-17 23:02:16,354 - KolHalashonScraper - INFO - Starting Kol Halashon scraping
2026-05-17 23:02:16,383 - KolHalashonScraper - WARNING - Could not match parasha 'none' to database parsha_list
2026-05-17 23:02:16,383 - KolHalashonScraper - ERROR - Could not determine current parsha, continuing with other search URLs
2026-05-17 23:02:16,383 - KolHalashonScraper - INFO - Processing parsha URL (id=1)
2026-05-17 23:02:16,383 - KolHalashonScraper - INFO - Processing 8 total URLs (1 parsha + 7 search)
2026-05-17 23:02:16,383 - KolHalashonScraper - WARNING - Could not match parasha 'none' to database parsha_list
2026-05-17 23:02:16,384 - KolHalashonScraper - WARNING - Could not resolve parsha_info for URL placeholder replacement
2026-05-17 23:02:16,384 - KolHalashonScraper - WARNING - No daf schedule found for today - cannot replace daf placeholders
2026-05-17 23:02:16,384 - KolHalashonScraper - INFO - Scraping all 5 URLs with unified method...
2026-05-17 23:02:16,384 - KolHalashonScraper - INFO - Starting multi-URL browser scrape with 5 URLs, max 10 pages per URL (download_inline=False)
2026-05-17 23:02:16,384 - KolHalashonScraper - INFO - Running browser in non-headless mode (Cloudflare bypass)
2026-05-17 23:02:16,384 - KolHalashonScraper - INFO - Using undetected-chromedriver for Cloudflare bypass
2026-05-17 23:02:16,586 - undetected_chromedriver.patcher - INFO - patching driver executable /home/openclaw/.local/share/undetected_chromedriver/undetected_chromedriver
2026-05-17 23:02:17,139 - KolHalashonScraper - INFO - Browser created for multi-URL scrape
2026-05-17 23:02:17,139 - KolHalashonScraper - INFO - Processing URL (id=3): https://www.kolhalashon.com/en/regularSite/searchResults/100/49/Meilah?urlFilters=3:3%7C20:13%7C|Harav Avraham Loifer
2026-05-17 23:02:18,988 - KolHalashonScraper - INFO - Page loaded with 49 shiur containers after 0s
2026-05-17 23:02:18,988 - KolHalashonScraper - INFO - [DELAY] Waiting 60.0s for page content to stabilize (shiur_found_wait)
2026-05-17 23:03:25,027 - KolHalashonScraper - INFO - Processing page 1 of URL (pagination shows: page 1 of 3, has_next=True)
2026-05-17 23:03:25,028 - KolHalashonScraper - INFO - [DELAY] Waiting 6.0s for DOM extraction (dom_extraction_wait)
2026-05-17 23:03:31,043 - KolHalashonScraper - INFO - Extracted 24 shiurim from DOM
2026-05-17 23:03:31,043 - KolHalashonScraper - INFO - Page 1: Browser extracted 24 shiurim
2026-05-17 23:03:31,051 - KolHalashonScraper - INFO - Navigating to page 2...
2026-05-17 23:03:31,095 - KolHalashonScraper - INFO - [DELAY] Waiting 10s for page to reload after clicking next button
2026-05-17 23:03:48,143 - KolHalashonScraper - INFO - Processing page 2 of URL (pagination shows: page 2 of 3, has_next=True)
2026-05-17 23:03:48,143 - KolHalashonScraper - INFO - [DELAY] Waiting 6.0s for DOM extraction (dom_extraction_wait)
2026-05-17 23:03:54,154 - KolHalashonScraper - INFO - Extracted 24 shiurim from DOM
2026-05-17 23:03:54,154 - KolHalashonScraper - INFO - Page 2: Browser extracted 24 shiurim
2026-05-17 23:03:54,163 - KolHalashonScraper - INFO - Navigating to page 3...
2026-05-17 23:03:54,196 - KolHalashonScraper - INFO - [DELAY] Waiting 10s for page to reload after clicking next button
2026-05-17 23:04:11,239 - KolHalashonScraper - INFO - Processing page 3 of URL (pagination shows: page 3 of 3, has_next=False)
2026-05-17 23:04:11,239 - KolHalashonScraper - INFO - [DELAY] Waiting 6.0s for DOM extraction (dom_extraction_wait)
2026-05-17 23:04:17,249 - KolHalashonScraper - INFO - Extracted 4 shiurim from DOM
2026-05-17 23:04:17,249 - KolHalashonScraper - INFO - Page 3: Browser extracted 4 shiurim
2026-05-17 23:04:17,256 - KolHalashonScraper - INFO - No more pages available after page 3
2026-05-17 23:04:17,257 - KolHalashonScraper - INFO - URL complete: 52 shiurim extracted from 3 page(s), 52 new total so far
2026-05-17 23:04:17,344 - KolHalashonScraper - INFO - [DELAY] Waiting 0.7s for filter sections to expand (filter_expand_wait)
2026-05-17 23:04:18,077 - KolHalashonScraper - INFO - [DELAY] Waiting 0.7s after clicking 'display more' buttons (filter_expand_wait)
2026-05-17 23:04:18,788 - KolHalashonScraper - INFO - Expanded filter sections: clicked 2 'display more' buttons
2026-05-17 23:04:19,798 - KolHalashonScraper - INFO - DOM filter debug: {'displayAllSections': 2, 'filterContents': 5, 'filterOptions': 45, 'totalCheckboxes': 43, 'type2Checkboxes': 33}
2026-05-17 23:04:19,809 - KolHalashonScraper - INFO - Extracted 31 speakers from DOM filter checkboxes
2026-05-17 23:04:19,809 - KolHalashonScraper - INFO - Extracted 31 speakers from URL filter dropdown
2026-05-17 23:04:19,818 - social_media_monitor.scrapers.kolhalashon_scraper - INFO - Speaker sync complete: 0 new, 31 updated, 517 total
2026-05-17 23:04:19,818 - KolHalashonScraper - INFO - [DELAY] Waiting 1.0s between URLs (shiur_url_delay)
2026-05-17 23:04:20,819 - KolHalashonScraper - INFO - Processing URL (id=4): https://www.kolhalashon.com/en/regularSite/searchResults/100/49/Meilah?urlFilters=3:3%7C20:13%7C|Harav Avraham Shlomo Yavo Zt"l
2026-05-17 23:04:21,223 - KolHalashonScraper - INFO - Login required โ attempting to log in
2026-05-17 23:04:21,372 - KolHalashonScraper - INFO - [DELAY] Waiting 1.0s after login button click
2026-05-17 23:04:24,420 - KolHalashonScraper - INFO - Submitted login form with stored credentials
2026-05-17 23:04:26,512 - KolHalashonScraper - INFO - Login confirmed
2026-05-17 23:04:27,450 - KolHalashonScraper - INFO - Page loaded with 49 shiur containers after 0s
2026-05-17 23:04:27,451 - KolHalashonScraper - INFO - [DELAY] Waiting 60.0s for page content to stabilize (shiur_found_wait)
2026-05-17 23:05:33,493 - KolHalashonScraper - INFO - Processing page 1 of URL (pagination shows: page 1 of 3, has_next=True)
2026-05-17 23:05:33,493 - KolHalashonScraper - INFO - [DELAY] Waiting 6.0s for DOM extraction (dom_extraction_wait)
2026-05-17 23:05:39,503 - KolHalashonScraper - INFO - Extracted 24 shiurim from DOM
2026-05-17 23:05:39,503 - KolHalashonScraper - INFO - Page 1: Browser extracted 24 shiurim
2026-05-17 23:05:39,510 - KolHalashonScraper - INFO - Navigating to page 2...
2026-05-17 23:05:39,546 - KolHalashonScraper - INFO - [DELAY] Waiting 10s for page to reload after clicking next button
2026-05-17 23:05:56,595 - KolHalashonScraper - INFO - Processing page 2 of URL (pagination shows: page 2 of 3, has_next=True)
2026-05-17 23:05:56,595 - KolHalashonScraper - INFO - [DELAY] Waiting 6.0s for DOM extraction (dom_extraction_wait)
2026-05-17 23:06:02,607 - KolHalashonScraper - INFO - Extracted 24 shiurim from DOM
2026-05-17 23:06:02,607 - KolHalashonScraper - INFO - Page 2: Browser extracted 24 shiurim
2026-05-17 23:06:02,615 - KolHalashonScraper - INFO - Navigating to page 3...
2026-05-17 23:06:02,644 - KolHalashonScraper - INFO - [DELAY] Waiting 10s for page to reload after clicking next button
2026-05-17 23:06:19,685 - KolHalashonScraper - INFO - Processing page 3 of URL (pagination shows: page 3 of 3, has_next=False)
2026-05-17 23:06:19,685 - KolHalashonScraper - INFO - [DELAY] Waiting 6.0s for DOM extraction (dom_extraction_wait)
2026-05-17 23:06:25,695 - KolHalashonScraper - INFO - Extracted 4 shiurim from DOM
2026-05-17 23:06:25,695 - KolHalashonScraper - INFO - Page 3: Browser extracted 4 shiurim
2026-05-17 23:06:25,702 - KolHalashonScraper - INFO - No more pages available after page 3
2026-05-17 23:06:25,703 - KolHalashonScraper - INFO - URL complete: 52 shiurim extracted from 3 page(s), 104 new total so far
2026-05-17 23:06:25,783 - KolHalashonScraper - INFO - [DELAY] Waiting 0.7s for filter sections to expand (filter_expand_wait)
2026-05-17 23:06:26,518 - KolHalashonScraper - INFO - [DELAY] Waiting 0.7s after clicking 'display more' buttons (filter_expand_wait)
2026-05-17 23:06:27,231 - KolHalashonScraper - INFO - Expanded filter sections: clicked 2 'display more' buttons
2026-05-17 23:06:28,243 - KolHalashonScraper - INFO - DOM filter debug: {'displayAllSections': 2, 'filterContents': 5, 'filterOptions': 45, 'totalCheckboxes': 43, 'type2Checkboxes': 33}
2026-05-17 23:06:28,256 - KolHalashonScraper - INFO - Extracted 31 speakers from DOM filter checkboxes
2026-05-17 23:06:28,256 - KolHalashonScraper - INFO - Extracted 31 speakers from URL filter dropdown
2026-05-17 23:06:28,265 - social_media_monitor.scrapers.kolhalashon_scraper - INFO - Speaker sync complete: 0 new, 31 updated, 517 total
2026-05-17 23:06:28,265 - KolHalashonScraper - INFO - [DELAY] Waiting 1.0s between URLs (shiur_url_delay)
2026-05-17 23:06:29,265 - KolHalashonScraper - INFO - Processing URL (id=5): https://www.kolhalashon.com/en/regularSite/searchResults/100/49/Meilah?urlFilters=3:3%7C20:13%7C|Harav Elazar Nissan Rubin
2026-05-17 23:06:30,738 - KolHalashonScraper - INFO - Page loaded with 49 shiur containers after 0s
2026-05-17 23:06:30,738 - KolHalashonScraper - INFO - [DELAY] Waiting 60.0s for page content to stabilize (shiur_found_wait)
2026-05-17 23:07:36,774 - KolHalashonScraper - INFO - Processing page 1 of URL (pagination shows: page 1 of 3, has_next=True)
2026-05-17 23:07:36,774 - KolHalashonScraper - INFO - [DELAY] Waiting 6.0s for DOM extraction (dom_extraction_wait)
2026-05-17 23:07:42,788 - KolHalashonScraper - INFO - Extracted 24 shiurim from DOM
2026-05-17 23:07:42,788 - KolHalashonScraper - INFO - Page 1: Browser extracted 24 shiurim
2026-05-17 23:07:42,797 - KolHalashonScraper - INFO - Navigating to page 2...
2026-05-17 23:07:42,838 - KolHalashonScraper - INFO - [DELAY] Waiting 10s for page to reload after clicking next button
2026-05-17 23:07:59,885 - KolHalashonScraper - INFO - Processing page 2 of URL (pagination shows: page 2 of 3, has_next=True)
2026-05-17 23:07:59,885 - KolHalashonScraper - INFO - [DELAY] Waiting 6.0s for DOM extraction (dom_extraction_wait)
2026-05-17 23:08:05,898 - KolHalashonScraper - INFO - Extracted 24 shiurim from DOM
2026-05-17 23:08:05,898 - KolHalashonScraper - INFO - Page 2: Browser extracted 24 shiurim
2026-05-17 23:08:05,908 - KolHalashonScraper - INFO - Navigating to page 3...
2026-05-17 23:08:05,938 - KolHalashonScraper - INFO - [DELAY] Waiting 10s for page to reload after clicking next button
2026-05-17 23:08:22,982 - KolHalashonScraper - INFO - Processing page 3 of URL (pagination shows: page 3 of 3, has_next=False)
2026-05-17 23:08:22,982 - KolHalashonScraper - INFO - [DELAY] Waiting 6.0s for DOM extraction (dom_extraction_wait)
2026-05-17 23:08:28,992 - KolHalashonScraper - INFO - Extracted 4 shiurim from DOM
2026-05-17 23:08:28,992 - KolHalashonScraper - INFO - Page 3: Browser extracted 4 shiurim
2026-05-17 23:08:28,999 - KolHalashonScraper - INFO - No more pages available after page 3
2026-05-17 23:08:29,000 - KolHalashonScraper - INFO - URL complete: 52 shiurim extracted from 3 page(s), 156 new total so far
2026-05-17 23:08:29,078 - KolHalashonScraper - INFO - [DELAY] Waiting 0.7s for filter sections to expand (filter_expand_wait)
2026-05-17 23:08:29,812 - KolHalashonScraper - INFO - [DELAY] Waiting 0.7s after clicking 'display more' buttons (filter_expand_wait)
2026-05-17 23:08:30,525 - KolHalashonScraper - INFO - Expanded filter sections: clicked 2 'display more' buttons
2026-05-17 23:08:31,535 - KolHalashonScraper - INFO - DOM filter debug: {'displayAllSections': 2, 'filterContents': 5, 'filterOptions': 45, 'totalCheckboxes': 43, 'type2Checkboxes': 33}
2026-05-17 23:08:31,544 - KolHalashonScraper - INFO - Extracted 31 speakers from DOM filter checkboxes
2026-05-17 23:08:31,544 - KolHalashonScraper - INFO - Extracted 31 speakers from URL filter dropdown
2026-05-17 23:08:31,554 - social_media_monitor.scrapers.kolhalashon_scraper - INFO - Speaker sync complete: 0 new, 31 updated, 517 total
2026-05-17 23:08:31,554 - KolHalashonScraper - INFO - [DELAY] Waiting 1.0s between URLs (shiur_url_delay)
2026-05-17 23:08:32,555 - KolHalashonScraper - INFO - Processing URL (id=6): https://www.kolhalashon.com/en/regularSite/searchResults/100/49/Meilah?urlFilters=3:3%7C20:13%7C|Harav Shimon Spitzer
2026-05-17 23:08:33,963 - KolHalashonScraper - INFO - Page loaded with 49 shiur containers after 0s
2026-05-17 23:08:33,964 - KolHalashonScraper - INFO - [DELAY] Waiting 60.0s for page content to stabilize (shiur_found_wait)
2026-05-17 23:09:39,999 - KolHalashonScraper - INFO - Processing page 1 of URL (pagination shows: page 1 of 3, has_next=True)
2026-05-17 23:09:39,999 - KolHalashonScraper - INFO - [DELAY] Waiting 6.0s for DOM extraction (dom_extraction_wait)
2026-05-17 23:09:46,008 - KolHalashonScraper - INFO - Extracted 24 shiurim from DOM
2026-05-17 23:09:46,008 - KolHalashonScraper - INFO - Page 1: Browser extracted 24 shiurim
2026-05-17 23:09:46,015 - KolHalashonScraper - INFO - Navigating to page 2...
2026-05-17 23:09:46,034 - KolHalashonScraper - INFO - [DELAY] Waiting 10s for page to reload after clicking next button
2026-05-17 23:10:03,077 - KolHalashonScraper - INFO - Processing page 2 of URL (pagination shows: page 2 of 3, has_next=True)
2026-05-17 23:10:03,077 - KolHalashonScraper - INFO - [DELAY] Waiting 6.0s for DOM extraction (dom_extraction_wait)
2026-05-17 23:10:09,091 - KolHalashonScraper - INFO - Extracted 24 shiurim from DOM
2026-05-17 23:10:09,091 - KolHalashonScraper - INFO - Page 2: Browser extracted 24 shiurim
2026-05-17 23:10:09,101 - KolHalashonScraper - INFO - Navigating to page 3...
2026-05-17 23:10:09,131 - KolHalashonScraper - INFO - [DELAY] Waiting 10s for page to reload after clicking next button
2026-05-17 23:10:26,174 - KolHalashonScraper - INFO - Processing page 3 of URL (pagination shows: page 3 of 3, has_next=False)
2026-05-17 23:10:26,174 - KolHalashonScraper - INFO - [DELAY] Waiting 6.0s for DOM extraction (dom_extraction_wait)
2026-05-17 23:10:32,184 - KolHalashonScraper - INFO - Extracted 4 shiurim from DOM
2026-05-17 23:10:32,184 - KolHalashonScraper - INFO - Page 3: Browser extracted 4 shiurim
2026-05-17 23:10:32,191 - KolHalashonScraper - INFO - No more pages available after page 3
2026-05-17 23:10:32,192 - KolHalashonScraper - INFO - URL complete: 52 shiurim extracted from 3 page(s), 208 new total so far
2026-05-17 23:10:32,273 - KolHalashonScraper - INFO - [DELAY] Waiting 0.7s for filter sections to expand (filter_expand_wait)
2026-05-17 23:10:33,006 - KolHalashonScraper - INFO - [DELAY] Waiting 0.7s after clicking 'display more' buttons (filter_expand_wait)
2026-05-17 23:10:33,717 - KolHalashonScraper - INFO - Expanded filter sections: clicked 2 'display more' buttons
2026-05-17 23:10:34,728 - KolHalashonScraper - INFO - DOM filter debug: {'displayAllSections': 2, 'filterContents': 5, 'filterOptions': 45, 'totalCheckboxes': 43, 'type2Checkboxes': 33}
2026-05-17 23:10:34,740 - KolHalashonScraper - INFO - Extracted 31 speakers from DOM filter checkboxes
2026-05-17 23:10:34,740 - KolHalashonScraper - INFO - Extracted 31 speakers from URL filter dropdown
2026-05-17 23:10:34,751 - social_media_monitor.scrapers.kolhalashon_scraper - INFO - Speaker sync complete: 0 new, 31 updated, 517 total
2026-05-17 23:10:34,751 - KolHalashonScraper - INFO - [DELAY] Waiting 1.0s between URLs (shiur_url_delay)
2026-05-17 23:10:35,752 - KolHalashonScraper - INFO - Processing URL (id=7): https://www.kolhalashon.com/en/regularSite/searchResults/100/49/Meilah?urlFilters=3:3%7C20:13%7C|Harav Shmuel Dovid Friedman
2026-05-17 23:10:37,133 - KolHalashonScraper - INFO - Page loaded with 49 shiur containers after 0s
2026-05-17 23:10:37,133 - KolHalashonScraper - INFO - [DELAY] Waiting 60.0s for page content to stabilize (shiur_found_wait)
2026-05-17 23:11:43,169 - KolHalashonScraper - INFO - Processing page 1 of URL (pagination shows: page 1 of 3, has_next=True)
2026-05-17 23:11:43,169 - KolHalashonScraper - INFO - [DELAY] Waiting 6.0s for DOM extraction (dom_extraction_wait)
2026-05-17 23:11:49,182 - KolHalashonScraper - INFO - Extracted 24 shiurim from DOM
2026-05-17 23:11:49,182 - KolHalashonScraper - INFO - Page 1: Browser extracted 24 shiurim
2026-05-17 23:11:49,191 - KolHalashonScraper - INFO - Navigating to page 2...
2026-05-17 23:11:49,217 - KolHalashonScraper - INFO - [DELAY] Waiting 10s for page to reload after clicking next button
2026-05-17 23:12:06,263 - KolHalashonScraper - INFO - Processing page 2 of URL (pagination shows: page 2 of 3, has_next=True)
2026-05-17 23:12:06,264 - KolHalashonScraper - INFO - [DELAY] Waiting 6.0s for DOM extraction (dom_extraction_wait)
2026-05-17 23:18:56,579 - social_media_monitor.utils.logging_config - INFO - Posts log file: /mnt/data/openclaw-shared/home/shared/Social_Media_Monitor/logs/social_media_monitor_posts_2026-05-17.log
2026-05-17 23:18:56,580 - social_media_monitor.utils.logging_config - INFO - Downloads log file: /mnt/data/openclaw-shared/home/shared/Social_Media_Monitor/logs/social_media_monitor_downloads_2026-05-17.log
2026-05-17 23:18:56,580 - social_media_monitor.utils.logging_config - INFO - Logging configured with level INFO, log file: /mnt/data/openclaw-shared/home/shared/Social_Media_Monitor/logs/social_media_monitor_2026-05-17.log
2026-05-17 23:18:56,580 - __main__ - INFO - === Social Media Monitor v2.0 (Async) Started ===
2026-05-17 23:18:56,605 - social_media_monitor.config.settings - INFO - Configuration loaded from /home/openclaw/Social_Media_Monitor/config.yaml
2026-05-17 23:18:56,605 - __main__ - INFO - Configuration loaded and validated successfully
2026-05-17 23:18:56,607 - social_media_monitor.database.connection - INFO - Database connection pool initialized with 5 connections
2026-05-17 23:18:56,626 - social_media_monitor.database.connection - INFO - add_speaker_filters column already exists
2026-05-17 23:18:56,642 - social_media_monitor.database.connection - INFO - Database setup completed successfully
2026-05-17 23:18:56,642 - __main__ - INFO - Database initialized at: /mnt/data/openclaw-shared/home/shared/Social_Media_Monitor/social_media_monitor.db
2026-05-17 23:18:56,643 - social_media_monitor.scrapers.reddit_scraper - INFO - Reddit API initialized successfully
2026-05-17 23:18:56,644 - RSSFeedScraper - INFO - RSS scraper initialized: feeds=2, html_urls=0 (0 with custom selectors), include_terms=1, exclude_terms=0
2026-05-17 23:18:56,647 - Yiddish24Scraper - INFO - File downloader initialized with 1 terms, prefix 'Y2', and file type organization enabled
2026-05-17 23:18:56,647 - Yiddish24Scraper - INFO - Yiddish24 scraper initialized: data_id='latest', page_limit=100, total_pages=10, debug_html=True, include_terms=1, exclude_terms=0
2026-05-17 23:18:56,649 - AhBlickLiveScraper - INFO - AhBlickLive scraper initialized: urls=1, max_items=100, include_terms=1, exclude_terms=0
2026-05-17 23:18:56,653 - TorahAnytimeScraper - INFO - File downloader initialized with 1 terms, prefix 'TA', and file type organization enabled
2026-05-17 23:18:56,653 - TorahAnytimeScraper - INFO - TorahAnytime scraper initialized: limit=400, offset=0, project_id=1, include_terms=1, exclude_terms=0
2026-05-17 23:18:56,655 - BatorahScraper - INFO - File downloader initialized with 1 terms, prefix 'BT', and file type organization enabled
2026-05-17 23:18:56,655 - BatorahScraper - INFO - Batorah scraper initialized: api_urls=2, page_size=100, include_terms=1, exclude_terms=0
2026-05-17 23:18:56,656 - KolHalashonScraper - INFO - Kol Halashon scraper initialized: base_url=https://www2.kolhalashon.com, max_shiurim=50, fetch_all_speakers=True, headless=False, exclude_filters=1, include_filters=0
2026-05-17 23:18:56,656 - __main__ - INFO - Kol Halashon scraper initialized and enabled
2026-05-17 23:18:56,656 - __main__ - INFO - Running 1 scrapers concurrently...
2026-05-17 23:18:56,656 - KolHalashonScraper - INFO - Starting Kol Halashon scraping
2026-05-17 23:18:56,697 - KolHalashonScraper - WARNING - Could not match parasha 'none' to database parsha_list
2026-05-17 23:18:56,697 - KolHalashonScraper - ERROR - Could not determine current parsha, continuing with other search URLs
2026-05-17 23:18:56,698 - KolHalashonScraper - INFO - Processing parsha URL (id=1)
2026-05-17 23:18:56,698 - KolHalashonScraper - INFO - Processing 8 total URLs (1 parsha + 7 search)
2026-05-17 23:18:56,698 - KolHalashonScraper - WARNING - Could not match parasha 'none' to database parsha_list
2026-05-17 23:18:56,698 - KolHalashonScraper - WARNING - Could not resolve parsha_info for URL placeholder replacement
2026-05-17 23:18:56,698 - KolHalashonScraper - WARNING - No daf schedule found for today - cannot replace daf placeholders
2026-05-17 23:18:56,698 - KolHalashonScraper - INFO - Scraping all 5 URLs with unified method...
2026-05-17 23:18:56,698 - KolHalashonScraper - INFO - Starting multi-URL browser scrape with 5 URLs, max 10 pages per URL (download_inline=False)
2026-05-17 23:18:56,698 - KolHalashonScraper - INFO - Running browser in non-headless mode (Cloudflare bypass)
2026-05-17 23:18:56,698 - KolHalashonScraper - WARNING - undetected-chromedriver not available; falling back to standard Chrome
2026-05-17 23:18:57,632 - KolHalashonScraper - INFO - Browser created for multi-URL scrape
2026-05-17 23:18:57,632 - KolHalashonScraper - INFO - Processing URL (id=3): https://www.kolhalashon.com/en/regularSite/searchResults/100/49/Meilah?urlFilters=3:3%7C20:13%7C|Harav Avraham Loifer
2026-05-17 23:18:58,205 - KolHalashonScraper - INFO - Login required โ attempting to log in
2026-05-17 23:18:58,318 - KolHalashonScraper - INFO - [DELAY] Waiting 1.0s after login button click
2026-05-17 23:19:01,367 - KolHalashonScraper - INFO - Submitted login form with stored credentials
2026-05-17 23:19:31,507 - KolHalashonScraper - WARNING - Login not confirmed after 30 seconds
2026-05-17 23:19:32,557 - KolHalashonScraper - INFO - Page loaded with 1 shiur containers after 0s
2026-05-17 23:19:32,557 - KolHalashonScraper - INFO - [DELAY] Waiting 60.0s for page content to stabilize (shiur_found_wait)
2026-05-17 23:20:38,598 - KolHalashonScraper - INFO - Processing page 1 of URL (pagination shows: page 1 of 1, has_next=False)
2026-05-17 23:20:38,598 - KolHalashonScraper - INFO - [DELAY] Waiting 6.0s for DOM extraction (dom_extraction_wait)
2026-05-17 23:20:44,607 - KolHalashonScraper - INFO - Extracted 0 shiurim from DOM
2026-05-17 23:20:44,608 - KolHalashonScraper - WARNING - Page 1: No shiurim extracted
2026-05-17 23:20:44,612 - KolHalashonScraper - INFO - No more pages available after page 1
2026-05-17 23:20:44,612 - KolHalashonScraper - INFO - URL complete: 0 shiurim extracted from 1 page(s), 0 new total so far
2026-05-17 23:20:45,633 - KolHalashonScraper - INFO - DOM filter debug: {'displayAllSections': 0, 'filterContents': 0, 'filterOptions': 0, 'totalCheckboxes': 0, 'type2Checkboxes': 0}
2026-05-17 23:20:45,644 - KolHalashonScraper - INFO - Extracted 0 speakers from DOM filter checkboxes
2026-05-17 23:20:45,644 - KolHalashonScraper - INFO - [DELAY] Waiting 1.0s between URLs (shiur_url_delay)
2026-05-17 23:20:46,644 - KolHalashonScraper - INFO - Processing URL (id=4): https://www.kolhalashon.com/en/regularSite/searchResults/100/49/Meilah?urlFilters=3:3%7C20:13%7C|Harav Avraham Shlomo Yavo Zt"l
2026-05-17 23:20:46,870 - KolHalashonScraper - INFO - Login required โ attempting to log in
2026-05-17 23:20:46,934 - KolHalashonScraper - INFO - [DELAY] Waiting 1.0s after login button click
2026-05-17 23:20:49,973 - KolHalashonScraper - INFO - Submitted login form with stored credentials
2026-05-17 23:21:20,113 - KolHalashonScraper - WARNING - Login not confirmed after 30 seconds
2026-05-17 23:21:21,082 - KolHalashonScraper - INFO - Page loaded with 1 shiur containers after 0s
2026-05-17 23:21:21,082 - KolHalashonScraper - INFO - [DELAY] Waiting 60.0s for page content to stabilize (shiur_found_wait)
2026-05-17 23:22:27,118 - KolHalashonScraper - INFO - Processing page 1 of URL (pagination shows: page 1 of 1, has_next=False)
2026-05-17 23:22:27,118 - KolHalashonScraper - INFO - [DELAY] Waiting 6.0s for DOM extraction (dom_extraction_wait)
2026-05-17 23:22:33,129 - KolHalashonScraper - INFO - Extracted 0 shiurim from DOM
2026-05-17 23:22:33,129 - KolHalashonScraper - WARNING - Page 1: No shiurim extracted
2026-05-17 23:22:33,137 - KolHalashonScraper - INFO - No more pages available after page 1
2026-05-17 23:22:33,137 - KolHalashonScraper - INFO - URL complete: 0 shiurim extracted from 1 page(s), 0 new total so far
2026-05-17 23:22:34,160 - KolHalashonScraper - INFO - DOM filter debug: {'displayAllSections': 0, 'filterContents': 0, 'filterOptions': 0, 'totalCheckboxes': 0, 'type2Checkboxes': 0}
2026-05-17 23:22:34,167 - KolHalashonScraper - INFO - Extracted 0 speakers from DOM filter checkboxes
2026-05-17 23:22:34,167 - KolHalashonScraper - INFO - [DELAY] Waiting 1.0s between URLs (shiur_url_delay)
2026-05-17 23:22:35,167 - KolHalashonScraper - INFO - Processing URL (id=5): https://www.kolhalashon.com/en/regularSite/searchResults/100/49/Meilah?urlFilters=3:3%7C20:13%7C|Harav Elazar Nissan Rubin
2026-05-17 23:22:35,416 - KolHalashonScraper - INFO - Login required โ attempting to log in
2026-05-17 23:22:35,492 - KolHalashonScraper - INFO - [DELAY] Waiting 1.0s after login button click
2026-05-17 23:22:38,533 - KolHalashonScraper - INFO - Submitted login form with stored credentials
2026-05-17 23:23:08,680 - KolHalashonScraper - WARNING - Login not confirmed after 30 seconds
2026-05-17 23:23:09,670 - KolHalashonScraper - INFO - Page loaded with 1 shiur containers after 0s
2026-05-17 23:23:09,670 - KolHalashonScraper - INFO - [DELAY] Waiting 60.0s for page content to stabilize (shiur_found_wait)
2026-05-17 23:26:15,878 - social_media_monitor.utils.logging_config - INFO - Posts log file: /mnt/data/openclaw-shared/home/shared/Social_Media_Monitor/logs/social_media_monitor_posts_2026-05-17.log
2026-05-17 23:26:15,878 - social_media_monitor.utils.logging_config - INFO - Downloads log file: /mnt/data/openclaw-shared/home/shared/Social_Media_Monitor/logs/social_media_monitor_downloads_2026-05-17.log
2026-05-17 23:26:15,878 - social_media_monitor.utils.logging_config - INFO - Logging configured with level INFO, log file: /mnt/data/openclaw-shared/home/shared/Social_Media_Monitor/logs/social_media_monitor_2026-05-17.log
2026-05-17 23:26:15,878 - __main__ - INFO - === Social Media Monitor v2.0 (Async) Started ===
2026-05-17 23:26:15,903 - social_media_monitor.config.settings - INFO - Configuration loaded from /home/openclaw/Social_Media_Monitor/config.yaml
2026-05-17 23:26:15,903 - __main__ - INFO - Configuration loaded and validated successfully
2026-05-17 23:26:15,905 - social_media_monitor.database.connection - INFO - Database connection pool initialized with 5 connections
2026-05-17 23:26:15,905 - social_media_monitor.database.connection - INFO - add_speaker_filters column already exists
2026-05-17 23:26:15,922 - social_media_monitor.database.connection - INFO - Database setup completed successfully
2026-05-17 23:26:15,922 - __main__ - INFO - Database initialized at: /mnt/data/openclaw-shared/home/shared/Social_Media_Monitor/social_media_monitor.db
2026-05-17 23:26:15,923 - social_media_monitor.scrapers.reddit_scraper - INFO - Reddit API initialized successfully
2026-05-17 23:26:15,924 - RSSFeedScraper - INFO - RSS scraper initialized: feeds=2, html_urls=0 (0 with custom selectors), include_terms=1, exclude_terms=0
2026-05-17 23:26:15,926 - Yiddish24Scraper - INFO - File downloader initialized with 1 terms, prefix 'Y2', and file type organization enabled
2026-05-17 23:26:15,926 - Yiddish24Scraper - INFO - Yiddish24 scraper initialized: data_id='latest', page_limit=100, total_pages=10, debug_html=True, include_terms=1, exclude_terms=0
2026-05-17 23:26:15,943 - AhBlickLiveScraper - INFO - AhBlickLive scraper initialized: urls=1, max_items=100, include_terms=1, exclude_terms=0
2026-05-17 23:26:15,948 - TorahAnytimeScraper - INFO - File downloader initialized with 1 terms, prefix 'TA', and file type organization enabled
2026-05-17 23:26:15,948 - TorahAnytimeScraper - INFO - TorahAnytime scraper initialized: limit=400, offset=0, project_id=1, include_terms=1, exclude_terms=0
2026-05-17 23:26:15,950 - BatorahScraper - INFO - File downloader initialized with 1 terms, prefix 'BT', and file type organization enabled
2026-05-17 23:26:15,950 - BatorahScraper - INFO - Batorah scraper initialized: api_urls=2, page_size=100, include_terms=1, exclude_terms=0
2026-05-17 23:26:15,950 - KolHalashonScraper - INFO - Kol Halashon scraper initialized: base_url=https://www2.kolhalashon.com, max_shiurim=50, fetch_all_speakers=True, headless=False, exclude_filters=1, include_filters=0
2026-05-17 23:26:15,950 - __main__ - INFO - Kol Halashon scraper initialized and enabled
2026-05-17 23:26:15,950 - __main__ - INFO - Running 1 scrapers concurrently...
2026-05-17 23:26:15,950 - KolHalashonScraper - INFO - Starting Kol Halashon scraping
2026-05-17 23:26:15,980 - KolHalashonScraper - WARNING - Could not match parasha 'none' to database parsha_list
2026-05-17 23:26:15,980 - KolHalashonScraper - ERROR - Could not determine current parsha, continuing with other search URLs
2026-05-17 23:26:15,980 - KolHalashonScraper - INFO - Processing parsha URL (id=1)
2026-05-17 23:26:15,980 - KolHalashonScraper - INFO - Processing 3 total URLs (1 parsha + 2 search)
2026-05-17 23:26:15,980 - KolHalashonScraper - WARNING - Could not match parasha 'none' to database parsha_list
2026-05-17 23:26:15,980 - KolHalashonScraper - WARNING - Could not resolve parsha_info for URL placeholder replacement
2026-05-17 23:26:15,980 - KolHalashonScraper - WARNING - No daf schedule found for today - cannot replace daf placeholders
2026-05-17 23:26:15,981 - KolHalashonScraper - WARNING - No valid URLs after placeholder replacement
2026-05-17 23:26:15,981 - __main__ - INFO - Scraper kolhalashon completed: 0 results
2026-05-17 23:26:15,981 - __main__ - INFO - No new results found across all enabled sources
2026-05-17 23:26:15,981 - __main__ - INFO - Cache stats: 0 items in memory, 0 files on disk
2026-05-17 23:26:15,981 - __main__ - INFO - Monitor complete. No new results found across all sources.
2026-05-17 23:26:15,981 - __main__ - INFO - === Social Media Monitor v2.0 Complete ===
2026-05-17 23:26:15,981 - social_media_monitor.database.connection - INFO - Closing all database connections
2026-05-17 23:26:15,994 - social_media_monitor.database.connection - INFO - Closed 5 database connections
2026-05-17 23:26:15,994 - __main__ - INFO - Closed all database connections
2026-05-17 23:26:15,994 - social_media_monitor.database.connection - INFO - Closing all database connections
2026-05-17 23:26:15,994 - social_media_monitor.database.connection - INFO - Closed 0 database connections
2026-05-17 23:46:17,570 - social_media_monitor.utils.logging_config - INFO - Posts log file: /mnt/data/openclaw-shared/home/shared/Social_Media_Monitor/logs/social_media_monitor_posts_2026-05-17.log
2026-05-17 23:46:17,570 - social_media_monitor.utils.logging_config - INFO - Downloads log file: /mnt/data/openclaw-shared/home/shared/Social_Media_Monitor/logs/social_media_monitor_downloads_2026-05-17.log
2026-05-17 23:46:17,570 - social_media_monitor.utils.logging_config - INFO - Logging configured with level INFO, log file: /mnt/data/openclaw-shared/home/shared/Social_Media_Monitor/logs/social_media_monitor_2026-05-17.log
2026-05-17 23:46:17,570 - __main__ - INFO - === Social Media Monitor v2.0 (Async) Started ===
2026-05-17 23:46:17,595 - social_media_monitor.config.settings - INFO - Configuration loaded from /home/openclaw/Social_Media_Monitor/config.yaml
2026-05-17 23:46:17,595 - __main__ - INFO - Configuration loaded and validated successfully
2026-05-17 23:46:17,597 - social_media_monitor.database.connection - INFO - Database connection pool initialized with 5 connections
2026-05-17 23:46:17,598 - social_media_monitor.database.connection - INFO - add_speaker_filters column already exists
2026-05-17 23:46:17,613 - social_media_monitor.database.connection - INFO - Database setup completed successfully
2026-05-17 23:46:17,613 - __main__ - INFO - Database initialized at: /mnt/data/openclaw-shared/home/shared/Social_Media_Monitor/social_media_monitor.db
2026-05-17 23:46:17,655 - social_media_monitor.scrapers.reddit_scraper - INFO - Reddit API initialized successfully
2026-05-17 23:46:17,656 - RSSFeedScraper - INFO - RSS scraper initialized: feeds=2, html_urls=0 (0 with custom selectors), include_terms=1, exclude_terms=0
2026-05-17 23:46:17,673 - Yiddish24Scraper - INFO - File downloader initialized with 1 terms, prefix 'Y2', and file type organization enabled
2026-05-17 23:46:17,673 - Yiddish24Scraper - INFO - Yiddish24 scraper initialized: data_id='latest', page_limit=100, total_pages=10, debug_html=True, include_terms=1, exclude_terms=0
2026-05-17 23:46:17,676 - AhBlickLiveScraper - INFO - AhBlickLive scraper initialized: urls=1, max_items=100, include_terms=1, exclude_terms=0
2026-05-17 23:46:17,680 - TorahAnytimeScraper - INFO - File downloader initialized with 1 terms, prefix 'TA', and file type organization enabled
2026-05-17 23:46:17,680 - TorahAnytimeScraper - INFO - TorahAnytime scraper initialized: limit=400, offset=0, project_id=1, include_terms=1, exclude_terms=0
2026-05-17 23:46:17,682 - BatorahScraper - INFO - File downloader initialized with 1 terms, prefix 'BT', and file type organization enabled
2026-05-17 23:46:17,682 - BatorahScraper - INFO - Batorah scraper initialized: api_urls=2, page_size=100, include_terms=1, exclude_terms=0
2026-05-17 23:46:17,682 - KolHalashonScraper - INFO - Kol Halashon scraper initialized: base_url=https://www2.kolhalashon.com, max_shiurim=50, fetch_all_speakers=True, headless=False, exclude_filters=1, include_filters=0
2026-05-17 23:46:17,682 - __main__ - INFO - Kol Halashon scraper initialized and enabled
2026-05-17 23:46:17,682 - __main__ - INFO - Running 1 scrapers concurrently...
2026-05-17 23:46:17,682 - KolHalashonScraper - INFO - Starting Kol Halashon scraping
2026-05-17 23:46:17,710 - KolHalashonScraper - WARNING - No parsha returned from hdate library
2026-05-17 23:46:17,710 - KolHalashonScraper - ERROR - Could not determine current parsha, continuing with other search URLs
2026-05-17 23:46:17,710 - KolHalashonScraper - INFO - Processing parsha URL (id=1)
2026-05-17 23:46:17,710 - KolHalashonScraper - INFO - Processing 3 total URLs (1 parsha + 2 search)
2026-05-17 23:46:17,710 - KolHalashonScraper - WARNING - No parsha returned from hdate library
2026-05-17 23:46:17,710 - KolHalashonScraper - WARNING - Could not resolve parsha_info for URL placeholder replacement
2026-05-17 23:46:17,711 - KolHalashonScraper - WARNING - No daf schedule found for today - cannot replace daf placeholders
2026-05-17 23:46:17,711 - KolHalashonScraper - WARNING - No valid URLs after placeholder replacement
2026-05-17 23:46:17,711 - __main__ - INFO - Scraper kolhalashon completed: 0 results
2026-05-17 23:46:17,711 - __main__ - INFO - No new results found across all enabled sources
2026-05-17 23:46:17,711 - __main__ - INFO - Cache stats: 0 items in memory, 0 files on disk
2026-05-17 23:46:17,711 - __main__ - INFO - Monitor complete. No new results found across all sources.
2026-05-17 23:46:17,711 - __main__ - INFO - === Social Media Monitor v2.0 Complete ===
2026-05-17 23:46:17,711 - social_media_monitor.database.connection - INFO - Closing all database connections
2026-05-17 23:46:17,724 - social_media_monitor.database.connection - INFO - Closed 5 database connections
2026-05-17 23:46:17,724 - __main__ - INFO - Closed all database connections
2026-05-17 23:46:17,725 - social_media_monitor.database.connection - INFO - Closing all database connections
2026-05-17 23:46:17,725 - social_media_monitor.database.connection - INFO - Closed 0 database connections
2026-05-17 23:48:28,922 - social_media_monitor.utils.logging_config - INFO - Posts log file: /mnt/data/openclaw-shared/home/shared/Social_Media_Monitor/logs/social_media_monitor_posts_2026-05-17.log
2026-05-17 23:48:28,922 - social_media_monitor.utils.logging_config - INFO - Downloads log file: /mnt/data/openclaw-shared/home/shared/Social_Media_Monitor/logs/social_media_monitor_downloads_2026-05-17.log
2026-05-17 23:48:28,922 - social_media_monitor.utils.logging_config - INFO - Logging configured with level INFO, log file: /mnt/data/openclaw-shared/home/shared/Social_Media_Monitor/logs/social_media_monitor_2026-05-17.log
2026-05-17 23:48:28,922 - __main__ - INFO - === Social Media Monitor v2.0 (Async) Started ===
2026-05-17 23:48:28,947 - social_media_monitor.config.settings - INFO - Configuration loaded from /home/openclaw/Social_Media_Monitor/config.yaml
2026-05-17 23:48:28,947 - __main__ - INFO - Configuration loaded and validated successfully
2026-05-17 23:48:28,949 - social_media_monitor.database.connection - INFO - Database connection pool initialized with 5 connections
2026-05-17 23:48:28,949 - social_media_monitor.database.connection - INFO - add_speaker_filters column already exists
2026-05-17 23:48:28,962 - social_media_monitor.database.connection - INFO - Database setup completed successfully
2026-05-17 23:48:28,962 - __main__ - INFO - Database initialized at: /mnt/data/openclaw-shared/home/shared/Social_Media_Monitor/social_media_monitor.db
2026-05-17 23:48:28,963 - social_media_monitor.scrapers.reddit_scraper - INFO - Reddit API initialized successfully
2026-05-17 23:48:28,964 - RSSFeedScraper - INFO - RSS scraper initialized: feeds=2, html_urls=0 (0 with custom selectors), include_terms=1, exclude_terms=0
2026-05-17 23:48:28,966 - Yiddish24Scraper - INFO - File downloader initialized with 1 terms, prefix 'Y2', and file type organization enabled
2026-05-17 23:48:28,966 - Yiddish24Scraper - INFO - Yiddish24 scraper initialized: data_id='latest', page_limit=100, total_pages=10, debug_html=True, include_terms=1, exclude_terms=0
2026-05-17 23:48:28,983 - AhBlickLiveScraper - INFO - AhBlickLive scraper initialized: urls=1, max_items=100, include_terms=1, exclude_terms=0
2026-05-17 23:48:28,987 - TorahAnytimeScraper - INFO - File downloader initialized with 1 terms, prefix 'TA', and file type organization enabled
2026-05-17 23:48:28,987 - TorahAnytimeScraper - INFO - TorahAnytime scraper initialized: limit=400, offset=0, project_id=1, include_terms=1, exclude_terms=0
2026-05-17 23:48:28,989 - BatorahScraper - INFO - File downloader initialized with 1 terms, prefix 'BT', and file type organization enabled
2026-05-17 23:48:28,989 - BatorahScraper - INFO - Batorah scraper initialized: api_urls=2, page_size=100, include_terms=1, exclude_terms=0
2026-05-17 23:48:28,990 - KolHalashonScraper - INFO - Kol Halashon scraper initialized: base_url=https://www2.kolhalashon.com, max_shiurim=50, fetch_all_speakers=True, headless=False, exclude_filters=1, include_filters=0
2026-05-17 23:48:28,990 - __main__ - INFO - Kol Halashon scraper initialized and enabled
2026-05-17 23:48:28,990 - __main__ - INFO - Running 1 scrapers concurrently...
2026-05-17 23:48:28,990 - KolHalashonScraper - INFO - Starting Kol Halashon scraping
2026-05-17 23:48:29,019 - KolHalashonScraper - INFO - Using fallback parsha for 2026-05-23: Naso (ื ืฉื)
2026-05-17 23:48:29,019 - KolHalashonScraper - INFO - Current parsha: Naso (ื ืฉื)
2026-05-17 23:48:29,019 - KolHalashonScraper - INFO - Processing parsha URL (id=1)
2026-05-17 23:48:29,019 - KolHalashonScraper - INFO - Processing 3 total URLs (1 parsha + 2 search)
2026-05-17 23:48:29,019 - KolHalashonScraper - INFO - Replacing parsha placeholders with id=35, name=Naso, in_humash=2
2026-05-17 23:48:29,019 - KolHalashonScraper - WARNING - No daf schedule found for today - cannot replace daf placeholders
2026-05-17 23:48:29,019 - KolHalashonScraper - INFO - Scraping all 2 URLs with unified method...
2026-05-17 23:48:29,020 - KolHalashonScraper - INFO - Starting multi-URL browser scrape with 2 URLs, max 10 pages per URL (download_inline=False)
2026-05-17 23:48:29,020 - KolHalashonScraper - INFO - Running browser in non-headless mode (Cloudflare bypass)
2026-05-17 23:48:29,020 - KolHalashonScraper - INFO - Using undetected-chromedriver for Cloudflare bypass
2026-05-17 23:48:29,369 - undetected_chromedriver.patcher - INFO - patching driver executable /home/openclaw/.local/share/undetected_chromedriver/undetected_chromedriver
2026-05-17 23:48:29,982 - KolHalashonScraper - INFO - Browser created for multi-URL scrape
2026-05-17 23:48:29,982 - KolHalashonScraper - INFO - Processing URL (id=1): https://www2.kolhalashon.com/regularSite/searchResults/1/35/Naso?urlFilters=3:3%7C{speakersfilter}
2026-05-17 23:48:29,983 - KolHalashonScraper - INFO - Replaced speakersfilter placeholder with 10 speaker filter(s)
2026-05-17 23:49:34,236 - social_media_monitor.utils.logging_config - INFO - Posts log file: /mnt/data/openclaw-shared/home/shared/Social_Media_Monitor/logs/social_media_monitor_posts_2026-05-17.log
2026-05-17 23:49:34,236 - social_media_monitor.utils.logging_config - INFO - Downloads log file: /mnt/data/openclaw-shared/home/shared/Social_Media_Monitor/logs/social_media_monitor_downloads_2026-05-17.log
2026-05-17 23:49:34,236 - social_media_monitor.utils.logging_config - INFO - Logging configured with level INFO, log file: /mnt/data/openclaw-shared/home/shared/Social_Media_Monitor/logs/social_media_monitor_2026-05-17.log
2026-05-17 23:49:34,236 - __main__ - INFO - === Social Media Monitor v2.0 (Async) Started ===
2026-05-17 23:49:34,261 - social_media_monitor.config.settings - INFO - Configuration loaded from /home/openclaw/Social_Media_Monitor/config.yaml
2026-05-17 23:49:34,261 - __main__ - INFO - Configuration loaded and validated successfully
2026-05-17 23:49:34,263 - social_media_monitor.database.connection - INFO - Database connection pool initialized with 5 connections
2026-05-17 23:49:34,276 - social_media_monitor.database.connection - INFO - add_speaker_filters column already exists
2026-05-17 23:49:34,289 - social_media_monitor.database.connection - INFO - Database setup completed successfully
2026-05-17 23:49:34,289 - __main__ - INFO - Database initialized at: /mnt/data/openclaw-shared/home/shared/Social_Media_Monitor/social_media_monitor.db
2026-05-17 23:49:34,290 - social_media_monitor.scrapers.reddit_scraper - INFO - Reddit API initialized successfully
2026-05-17 23:49:34,291 - RSSFeedScraper - INFO - RSS scraper initialized: feeds=2, html_urls=0 (0 with custom selectors), include_terms=1, exclude_terms=0
2026-05-17 23:49:34,293 - Yiddish24Scraper - INFO - File downloader initialized with 1 terms, prefix 'Y2', and file type organization enabled
2026-05-17 23:49:34,293 - Yiddish24Scraper - INFO - Yiddish24 scraper initialized: data_id='latest', page_limit=100, total_pages=10, debug_html=True, include_terms=1, exclude_terms=0
2026-05-17 23:49:34,305 - AhBlickLiveScraper - INFO - AhBlickLive scraper initialized: urls=1, max_items=100, include_terms=1, exclude_terms=0
2026-05-17 23:49:34,309 - TorahAnytimeScraper - INFO - File downloader initialized with 1 terms, prefix 'TA', and file type organization enabled
2026-05-17 23:49:34,309 - TorahAnytimeScraper - INFO - TorahAnytime scraper initialized: limit=400, offset=0, project_id=1, include_terms=1, exclude_terms=0
2026-05-17 23:49:34,311 - BatorahScraper - INFO - File downloader initialized with 1 terms, prefix 'BT', and file type organization enabled
2026-05-17 23:49:34,311 - BatorahScraper - INFO - Batorah scraper initialized: api_urls=2, page_size=100, include_terms=1, exclude_terms=0
2026-05-17 23:49:34,311 - KolHalashonScraper - INFO - Kol Halashon scraper initialized: base_url=https://www2.kolhalashon.com, max_shiurim=50, fetch_all_speakers=True, headless=False, exclude_filters=1, include_filters=0
2026-05-17 23:49:34,311 - __main__ - INFO - Kol Halashon scraper initialized and enabled
2026-05-17 23:49:34,311 - __main__ - INFO - Running 1 scrapers concurrently...
2026-05-17 23:49:34,311 - KolHalashonScraper - INFO - Starting Kol Halashon scraping
2026-05-17 23:49:34,341 - KolHalashonScraper - INFO - Using fallback parsha for 2026-05-23: Naso (ื ืฉื)
2026-05-17 23:49:34,341 - KolHalashonScraper - INFO - Current parsha: Naso (ื ืฉื)
2026-05-17 23:49:34,341 - KolHalashonScraper - INFO - Processing parsha URL (id=1)
2026-05-17 23:49:34,341 - KolHalashonScraper - INFO - Processing 3 total URLs (1 parsha + 2 search)
2026-05-17 23:49:34,341 - KolHalashonScraper - INFO - Replacing parsha placeholders with id=35, name=Naso, in_humash=2
2026-05-17 23:49:34,342 - KolHalashonScraper - WARNING - No daf schedule found for today - cannot replace daf placeholders
2026-05-17 23:49:34,342 - KolHalashonScraper - INFO - Scraping all 2 URLs with unified method...
2026-05-17 23:49:34,342 - KolHalashonScraper - INFO - Starting multi-URL browser scrape with 2 URLs, max 10 pages per URL (download_inline=False)
2026-05-17 23:49:34,342 - KolHalashonScraper - INFO - Running browser in non-headless mode (Cloudflare bypass)
2026-05-17 23:49:34,342 - KolHalashonScraper - INFO - Using undetected-chromedriver for Cloudflare bypass
2026-05-17 23:49:34,575 - undetected_chromedriver.patcher - INFO - patching driver executable /home/openclaw/.local/share/undetected_chromedriver/undetected_chromedriver
2026-05-17 23:49:35,215 - KolHalashonScraper - INFO - Browser created for multi-URL scrape
2026-05-17 23:49:35,215 - KolHalashonScraper - INFO - Processing URL (id=1): https://www2.kolhalashon.com/regularSite/searchResults/1/35/Naso?urlFilters=3:3%7C{speakersfilter}
2026-05-17 23:49:35,215 - KolHalashonScraper - INFO - Replaced speakersfilter placeholder with 10 speaker filter(s)
2026-05-17 23:51:37,754 - KolHalashonScraper - WARNING - No shiur containers found after 120s for URL: https://www2.kolhalashon.com/regularSite/searchResults/1/35/Naso?urlFilters=3:3%...
2026-05-17 23:51:37,778 - KolHalashonScraper - WARNING - Page title: Just a moment..., URL: https://www2.kolhalashon.com/regularSite/searchResults/1/35/Naso?urlFilters=3:3%7C2:7098%7C2:999121%7C2:952143%7C2:7651%7C2:985996%7C2:7643%7C2:7124%7C2:934613%7C2:2393%7C2:7031, body length: 6088
2026-05-17 23:51:37,786 - KolHalashonScraper - WARNING - Page content sample: www2.kolhalashon.com
Performing security verification
This website uses a security service to protect against malicious bots. This page is displayed while the website verifies you are not a bot.
Ray
2026-05-17 23:51:37,786 - KolHalashonScraper - INFO - Processing URL (id=2): https://www2.kolhalashon.com/regularSite/searchResults/6/-1/-1/-1/-1/New%20Shiurei%20Torah?urlFilters=3:3%7C14:35%7C
2026-05-17 23:53:39,977 - KolHalashonScraper - WARNING - No shiur containers found after 120s for URL: https://www2.kolhalashon.com/regularSite/searchResults/6/-1/-1/-1/-1/New%20Shiur...
2026-05-17 23:53:40,002 - KolHalashonScraper - WARNING - Page title: Just a moment..., URL: https://www2.kolhalashon.com/regularSite/searchResults/6/-1/-1/-1/-1/New%20Shiurei%20Torah?urlFilters=3:3%7C14:35%7C, body length: 5724
2026-05-17 23:53:40,010 - KolHalashonScraper - WARNING - Page content sample: www2.kolhalashon.com
Performing security verification
This website uses a security service to protect against malicious bots. This page is displayed while the website verifies you are not a bot.
Ray
2026-05-17 23:53:40,010 - KolHalashonScraper - INFO - Multi-URL scrape complete: 0 new shiurim from 0 URLs (0 total pages), 0 total checked
2026-05-17 23:53:40,010 - KolHalashonScraper - INFO - Starting standalone downloads for all URLs...
2026-05-17 23:53:40,011 - KolHalashonScraper - INFO - Using fallback parsha for 2026-05-23: Naso (ื ืฉื)
2026-05-17 23:53:40,011 - KolHalashonScraper - INFO - Download directory: /mnt/data/openclaw-shared/home/shared/Social_Media_Monitor/downloads/Audio
2026-05-17 23:53:40,012 - social_media_monitor.scrapers.kolhalashon_scraper - INFO - Found 46 undownloaded shiurim (no speaker grouping)
2026-05-17 23:53:40,013 - KolHalashonScraper - INFO - Selected 28 shiurim using session-based one-per-speaker (from 46 total undownloaded)
2026-05-17 23:53:40,013 - KolHalashonScraper - INFO - Starting download of 28 shiurim (one_per_speaker=True)
2026-05-17 23:53:40,013 - KolHalashonScraper - INFO - Using temporary Chrome profile: /tmp/khl_profile_kg06kw9b
2026-05-17 23:53:40,013 - KolHalashonScraper - INFO - Running browser in non-headless mode (Cloudflare bypass)
2026-05-17 23:53:40,013 - KolHalashonScraper - INFO - Browser download directory set to: /mnt/data/openclaw-shared/home/shared/Social_Media_Monitor/downloads/Audio
2026-05-17 23:53:40,013 - KolHalashonScraper - INFO - Using undetected-chromedriver for Cloudflare bypass
2026-05-17 23:53:40,221 - undetected_chromedriver.patcher - INFO - patching driver executable /home/openclaw/.local/share/undetected_chromedriver/undetected_chromedriver
2026-05-17 23:53:40,852 - KolHalashonScraper - INFO - Browser created for downloads
2026-05-17 23:53:40,931 - KolHalashonScraper - INFO - Navigating to: https://www2.kolhalashon.com/regularSite/searchResults/1/35/Naso?urlFilters=3:3%...
2026-05-17 23:54:18,863 - KolHalashonScraper - INFO - Processing page 1 (pagination shows: page 1 of 77, has_next=True)
2026-05-17 23:54:18,875 - KolHalashonScraper - INFO - Page 1 has 24 shiur items
2026-05-17 23:54:18,876 - KolHalashonScraper - INFO - [p1:0] Skip (not_requested_or_already_downloaded): 1:48:21 Harav Akiva Erlanger ืคืจืฉืช ื ืฉื ืืจืืฉ ืืฉืืืขืืช, Parashat by Unknown (ID: 42527680)
2026-05-17 23:54:18,876 - KolHalashonScraper - INFO - [p1:1] Skip (not_requested_or_already_downloaded): 2:04 Harav Hilel Verthaimer ืืื ืืืืข ืืงืจืืื ืฉื ืืืืคืช ืืฉืืืจืืจ by Unknown (ID: 42453439)
2026-05-17 23:54:18,876 - KolHalashonScraper - INFO - [p1:2] Skip (not_requested_or_already_downloaded): 36:55 Harav Meir Dov Rechnitzer ืฉืืขืืจ ืคืจืฉืช ื ืฉื ืกืืืื ืืฉืืื ืข by Unknown (ID: 42526679)
2026-05-17 23:54:18,876 - KolHalashonScraper - INFO - [p1:3] Skip (not_requested_or_already_downloaded): 1:24:18 Harav Aharon Toissig ืืืืื ืืืฉืืืฉ ืืืืจื ืื ืื ืชืง, Par by Unknown (ID: 42453148)
2026-05-17 23:54:18,876 - KolHalashonScraper - INFO - [p1:4] Skip (not_requested_or_already_downloaded): 41:48 Harav Shimon Spitzer ืืื ื ืคืจืฉืช ื ืฉื ืชืฉืค"ื, Parashat No by Unknown (ID: 42453129)
2026-05-17 23:54:18,876 - KolHalashonScraper - INFO - [p1:5] Skip (not_requested_or_already_downloaded): 41:20 ืืฉืืืช ืืืืจ ืขืื ืืื ืืื ืืื ื ืฉื ืจืืฉ ืืืืฉ ืกืืื, Parashat N by Unknown (ID: 42526599)
2026-05-17 23:54:18,876 - KolHalashonScraper - INFO - [p1:6] Skip (not_requested_or_already_downloaded): 42:44 Harav Shmuel Arye Shapira ืคืจืฉืช ืืฉืืืข ืคืจืฉืช ื ืฉื ืขื ืืื ื ื by Unknown (ID: 42526398)
2026-05-17 23:54:18,876 - KolHalashonScraper - INFO - [p1:7] Skip (not_requested_or_already_downloaded): 1:07:22 Harav Menashe Yisrael Reizman ืืื ืโ ืคืจืฉืช ื ืฉื ืจ"ื ืกื by Unknown (ID: 42526325)
2026-05-17 23:54:18,876 - KolHalashonScraper - INFO - [p1:8] Skip (not_requested_or_already_downloaded): 1:19:53 Harav Shmuel Shmelke Leifer Zt"l ืืจืฉื ืืขื ืื ืืืืื ืืช by Unknown (ID: 42452357)
2026-05-17 23:54:18,876 - KolHalashonScraper - INFO - [p1:9] Skip (not_requested_or_already_downloaded): 1:05:22 Rav Schnair Zalman Weiss ืคืจืฉืช ื ืฉื ืืื ื ืืื ืืฉืืืขืืช, by Unknown (ID: 42525025)
2026-05-17 23:54:18,877 - KolHalashonScraper - INFO - [p1:10] Skip (not_requested_or_already_downloaded): 23:33 Harav Shlomo Luria ืคืจืฉืช ื ืฉื, Parashat Noso Harav Shlom by Unknown (ID: 42446656)
2026-05-17 23:54:18,877 - KolHalashonScraper - INFO - [p1:11] Skip (not_requested_or_already_downloaded): 7:39 Harav Sholom Ungar ืืขืจ ืคืกืืง ืืืื ืืื ืคืจืฉืช ื ืฉื, Masechet by Unknown (ID: 42506567)
2026-05-17 23:54:18,877 - KolHalashonScraper - INFO - [p1:12] Skip (not_requested_or_already_downloaded): 42:34 Rav Yoel Katz ื ืฉืืืงื ืคืื ืื ืืืืืขืืืืข ืคืจืฉื ื ืฉื, Paras by Unknown (ID: 42503152)
2026-05-17 23:54:18,877 - KolHalashonScraper - INFO - [p1:13] Skip (not_requested_or_already_downloaded): 41:30 Harav Baruch Nussbaum ืคืจืฉืช ื ืฉื ืกืืฃ ืคืจืง ื', Parashat No by Unknown (ID: 42494240)
2026-05-17 23:54:18,877 - KolHalashonScraper - INFO - [p1:14] Skip (not_requested_or_already_downloaded): 25:33 Dirshu Amud Yomi Vein ืคืจืฉืช ื ืฉื ืฉืืขืืจ ื', Parashat Noso by Unknown (ID: 42490910)
2026-05-17 23:54:18,877 - KolHalashonScraper - INFO - [p1:15] Skip (not_requested_or_already_downloaded): 2:40:38 Harav Avrohom Mordechay Goldblat ืื ืืืืืขืืืืข ืคืจืฉื ืค by Unknown (ID: 42474432)
2026-05-17 23:54:18,877 - KolHalashonScraper - INFO - [p1:16] Skip (not_requested_or_already_downloaded): 33:19 Harav Shlomo Sharf ืคืจืฉืช ื ืฉื ืฉืืขืืจ ืืกืคืจ ืืจืืข, Parashat by Unknown (ID: 42396946)
2026-05-17 23:54:18,877 - KolHalashonScraper - INFO - [p1:17] Skip (not_requested_or_already_downloaded): 44:21 Harav Mendel Shaffran ืคืจืฉืช ื ืฉื ืื ืืืืืืืข ืคืืืืช ืืืง ื, by Unknown (ID: 42394029)
2026-05-17 23:54:18,877 - KolHalashonScraper - INFO - [p1:18] Skip (not_requested_or_already_downloaded): 27:46 Harav Baruch Leifer ืคืจืฉื ื ืฉื, Parashat Noso, Sefer Sid by Unknown (ID: 42352701)
2026-05-17 23:54:18,877 - KolHalashonScraper - INFO - [p1:19] Skip (not_requested_or_already_downloaded): 2:38 ืืจื ืืืื ืืืืจ ืื ืื ืืื ืชืืจื, Parashat Noso, Shavuos ืืื by Unknown (ID: 42312910)
2026-05-17 23:54:18,877 - KolHalashonScraper - INFO - [p1:20] Skip (not_requested_or_already_downloaded): 50:57 Harav Shimon Ozer Bransdorfer ืฉืืขืืจ ื"ื ืคืจืฉืช ื ืฉื, ื ืืืจ by Unknown (ID: 42248586)
2026-05-17 23:54:18,877 - KolHalashonScraper - INFO - [p1:21] Skip (not_requested_or_already_downloaded): 18:16 Harav Mordechai Weiss ืคืจืฉืช ื ืฉื, Parashat Noso Shiurim by Unknown (ID: 42247268)
2026-05-17 23:54:18,877 - KolHalashonScraper - INFO - [p1:22] Skip (not_requested_or_already_downloaded): 56:31 Harav Chaim Halperin ืคืจืฉืช ื ืฉื. ืคืืจืืืืก ื ืฉื ืืขื ืืจืื ืืข by Unknown (ID: 42246390)
2026-05-17 23:54:18,877 - KolHalashonScraper - INFO - [p1:23] Skip (not_requested_or_already_downloaded): 45:24 Harav Shmuel Shish ืฉืืขืืจ ืืืจ ืืืืื ืืงืืืฉ ืคืจืฉืช ื ืฉื, Par by Unknown (ID: 42222980)
2026-05-17 23:54:18,886 - KolHalashonScraper - INFO - Navigating to page 2...
2026-05-17 23:54:18,957 - KolHalashonScraper - INFO - [DELAY] Waiting 10s for page to reload after clicking next button
2026-05-17 23:54:33,009 - KolHalashonScraper - INFO - Processing page 2 (pagination shows: page 2 of 77, has_next=True)
2026-05-17 23:54:33,020 - KolHalashonScraper - INFO - Page 2 has 24 shiur items
2026-05-17 23:54:33,020 - KolHalashonScraper - INFO - [p2:0] Skip (not_requested_or_already_downloaded): 9:59 Harav Sholom Ahron Ehrenfeld ืืื ืืืืจืื ืฉืชืื ืขืฉืจื ืคืขืืื by Unknown (ID: 33399851)
2026-05-17 23:54:33,020 - KolHalashonScraper - INFO - [p2:1] Skip (not_requested_or_already_downloaded): 1:59:51 Harav Nisan Dovid Kiwack ืฉืืขืืจ ืืืื ืคืจืฉืช ื ืฉื, Parash by Unknown (ID: 42054208)
2026-05-17 23:54:33,020 - KolHalashonScraper - INFO - [p2:2] Skip (not_requested_or_already_downloaded): 35:33 Harav Baruch Gefner ืฉืืขืืจ ืืืจ ืืืืื ื ืฉื, Parashat Noso by Unknown (ID: 41825827)
2026-05-17 23:54:33,020 - KolHalashonScraper - INFO - [p2:3] Skip (not_requested_or_already_downloaded): 51:24 Harav Avraham Tzvi Kluger ืฉืืื ื ื ืฉื ืชืฉืคื, Parashat No by Unknown (ID: 41814117)
2026-05-17 23:54:33,020 - KolHalashonScraper - INFO - [p2:4] Skip (not_requested_or_already_downloaded): 43:45 Harav Alter Horowitz Parashah Naso, Parashat Noso ืืืื by Unknown (ID: 41827621)
2026-05-17 23:54:33,020 - KolHalashonScraper - INFO - [p2:5] Skip (not_requested_or_already_downloaded): 33:40 Harav Yitzchok Zalmen Gips ื ืฉื ืชืฉืคื, Parashat Noso Shi by Unknown (ID: 41756382)
2026-05-17 23:54:33,021 - KolHalashonScraper - INFO - [p2:6] Skip (not_requested_or_already_downloaded): Harav Yochanan Friend 198ืฉื ื ื ืืืืื ืงืฆื ื ืฉื ืคื ืืืฉืืง, Paras by Unknown (ID: 41755685)
2026-05-17 23:54:33,021 - KolHalashonScraper - INFO - [p2:7] Skip (not_requested_or_already_downloaded): 21:24 Harav Moshe Chaim Cohen ื ืฉื-ืื ืืชืจืืืืืช ืืืื ืืฉืืื ืืฉื by Unknown (ID: 41753870)
2026-05-17 23:54:33,021 - KolHalashonScraper - INFO - [p2:8] Skip (not_requested_or_already_downloaded): 15:06 Harav Mendel Heimlich - Kurade Ruv ืืืจื ืชืืจื ืฉืืฉ ืกืขืืื by Unknown (ID: 41752297)
2026-05-17 23:54:33,021 - KolHalashonScraper - INFO - [p2:9] Skip (not_requested_or_already_downloaded): 15:05 Harav Elazar Nissan Rubin Parashat Noso, Sefer Ohr Hac by Unknown (ID: 41749768)
2026-05-17 23:54:33,021 - KolHalashonScraper - INFO - [p2:10] Skip (not_requested_or_already_downloaded): Harav Zvi Yishai Deman ืืฉืช ืืืฉ ืืืื ืืขืื, Masechet Sotta, Pa by Unknown (ID: 41749587)
2026-05-17 23:54:33,021 - KolHalashonScraper - INFO - [p2:11] Skip (not_requested_or_already_downloaded): 1:29:14 Harav Mordechai Zilber - Toldos Yehuda Stochin ืืืจืช by Unknown (ID: 41797485)
2026-05-17 23:54:33,021 - KolHalashonScraper - INFO - [p2:12] Skip (not_requested_or_already_downloaded): 10:23 Harav Daniel Chaim Alter ืขืืชืง ืฉื ืฉืืขืืจ ืืืจืฉ''ื ืคืจืฉืช ื ืฉ by Unknown (ID: 41748282)
2026-05-17 23:54:33,021 - KolHalashonScraper - INFO - [p2:13] Skip (not_requested_or_already_downloaded): 1:04:36 Harav Yitzchok Meir Morgenstern ื ืฉื ืชืฉืค"ื, Parashat by Unknown (ID: 41794503)
2026-05-17 23:54:33,021 - KolHalashonScraper - INFO - [p2:14] Skip (not_requested_or_already_downloaded): 13:13 Harav Avraham Yehuda Hacoen Rabinowitz ืคืจืฉืช ื ืฉื ืชืฉืค"ื, by Unknown (ID: 41747637)
2026-05-17 23:54:33,021 - KolHalashonScraper - INFO - [p2:15] Skip (not_requested_or_already_downloaded): 0:27 Harav Levi Folger ืื ืืขื ืืขืื ืืื ืืขื, Parashat Noso PA by Unknown (ID: 41747791)
2026-05-17 23:54:33,021 - KolHalashonScraper - INFO - [p2:16] Skip (not_requested_or_already_downloaded): 13:25 Harav Moshe Duvid Grozalsky ืคืจืฉืช ื ืฉื ืชืฉืคื ืืขื ืื ืืืืืช by Unknown (ID: 41747599)
2026-05-17 23:54:33,021 - KolHalashonScraper - INFO - [p2:17] Skip (not_requested_or_already_downloaded): 33:24 Harav Binyomin Finkel ืืืฆืฉ ืคืจืฉืช ื ืฉื - ืกืคืจ ืืฉืื ืคืจืง ื ืค by Unknown (ID: 41751133)
2026-05-17 23:54:33,021 - KolHalashonScraper - INFO - [p2:18] Skip (not_requested_or_already_downloaded): 46:27 Harav Yitschok Moshe Erlanger Parashat Noso Shiurim & by Unknown (ID: 41794188)
2026-05-17 23:54:33,021 - KolHalashonScraper - INFO - [p2:19] Skip (not_requested_or_already_downloaded): 40:13 Harav Zeev Ernster ืคืจืฉืช ื ืฉื ืชืฉืค"ื, Parashat Noso ืคืจืฉืช by Unknown (ID: 41748208)
2026-05-17 23:54:33,021 - KolHalashonScraper - INFO - [p2:20] Skip (not_requested_or_already_downloaded): 20:16 Harav Avraham Yechezkel Friedman ืคืจืฉืช ื ืฉื, Parashat No by Unknown (ID: 41750144)
2026-05-17 23:54:33,022 - KolHalashonScraper - INFO - [p2:21] Skip (not_requested_or_already_downloaded): 23:36 Harav Chaim Shlomo Miller Parashat Noso Parshas Hashav by Unknown (ID: 41747695)
2026-05-17 23:54:33,022 - KolHalashonScraper - INFO - [p2:22] Skip (not_requested_or_already_downloaded): 1:10:01 Harav Eliyahu Shorr ืขืจื ืฉืืช ืงืืืฉ ื ืฉื ืืขื ืื ืืื ืืชืฉืื by Unknown (ID: 41747584)
2026-05-17 23:54:33,022 - KolHalashonScraper - INFO - [p2:23] Skip (not_requested_or_already_downloaded): 43:15 Rav Sholom Dovid Folger Parashat Noso Shiurim from Ame by Unknown (ID: 41747582)
2026-05-17 23:54:33,029 - KolHalashonScraper - INFO - Navigating to page 3...
2026-05-17 23:54:33,083 - KolHalashonScraper - INFO - [DELAY] Waiting 10s for page to reload after clicking next button
2026-05-17 23:54:47,125 - KolHalashonScraper - INFO - Processing page 3 (pagination shows: page 3 of 77, has_next=True)
2026-05-17 23:54:47,134 - KolHalashonScraper - INFO - Page 3 has 24 shiur items
2026-05-17 23:54:47,134 - KolHalashonScraper - INFO - [p3:0] Skip (not_requested_or_already_downloaded): 1:07 Harav Chaim Mordechai Pollack Parashat Noso, Sefer Zera by Unknown (ID: 41794068)
2026-05-17 23:54:47,134 - KolHalashonScraper - INFO - [p3:1] Skip (not_requested_or_already_downloaded): 40:48 Harav Yechiel Mayer Katz ืืขืืื, ืืื ืืืจ, ืืจืืช ืืื ืื, P by Unknown (ID: 41747445)
2026-05-17 23:54:47,134 - KolHalashonScraper - INFO - [p3:2] Skip (not_requested_or_already_downloaded): 26:08 Harav Ezriel Yaffe Parashat Noso Shiurim in Parshas Ha by Unknown (ID: 41793927)
2026-05-17 23:54:47,134 - KolHalashonScraper - INFO - [p3:3] Skip (not_requested_or_already_downloaded): 3:09 Harav Ben Zion Sneh ืชืจืื ืืช ืืืืง ืืืื - ืคืจืฉืช ื ืฉื - ืืืื by Unknown (ID: 41747406)
2026-05-17 23:54:47,134 - KolHalashonScraper - INFO - [p3:4] Skip (not_requested_or_already_downloaded): 4:50 Harav Shmuel Gaffner ืืืฉื, Parashat Noso Naso ืืืืืฉ10 S by Unknown (ID: 41793883)
2026-05-17 23:54:47,134 - KolHalashonScraper - INFO - [p3:5] Skip (not_requested_or_already_downloaded): 43:18 Harav Yitzchak Aharon Goldhirsh ืคืจืฉืช ื ืฉื ืชืฉืค"ื, Parash by Unknown (ID: 41747363)
2026-05-17 23:54:47,134 - KolHalashonScraper - INFO - [p3:6] Skip (not_requested_or_already_downloaded): 11:35 Harav Avraham Zvi Rotman Parashat Noso Shiurim in Pars by Unknown (ID: 41793765)
2026-05-17 23:54:47,134 - KolHalashonScraper - INFO - [p3:7] Skip (not_requested_or_already_downloaded): 12:41 Harav Chaim Stark ืฉืืช ืืืืจ ืื ืืฉืืืขืืช-ืื ืืืืื ืืื ืงืื by Unknown (ID: 41793586)
2026-05-17 23:54:47,134 - KolHalashonScraper - INFO - [p3:8] Skip (not_requested_or_already_downloaded): 6:57 Harav Moshe Lantsiski 01 ืืฉืื ืืฉืจืื ื ืฉื ืืืช ื, Parashat by Unknown (ID: 41747169)
2026-05-17 23:54:47,135 - KolHalashonScraper - INFO - [p3:9] Skip (not_requested_or_already_downloaded): 1:00:27 Harav Meir Feldbrand Parashat Noso Shiurim in Parsha by Unknown (ID: 41793537)
2026-05-17 23:54:47,135 - KolHalashonScraper - INFO - [p3:10] Skip (not_requested_or_already_downloaded): 26:23 Minchas Yitzchak Yeshiva Bnei Brak Parashat Noso Shiur by Unknown (ID: 41793532)
2026-05-17 23:54:47,135 - KolHalashonScraper - INFO - [p3:11] Skip (not_requested_or_already_downloaded): 28:12 Harav Moshe David Katz ืืืืจ ืืืืืช: ืืื ืืื ืืชืืื ืืงืจืื by Unknown (ID: 41747113)
2026-05-17 23:54:47,135 - KolHalashonScraper - INFO - [p3:12] Skip (not_requested_or_already_downloaded): 33:24 Rav Meshilem Bauer ื ืฉื ืืืจ ืฉืืืขืืช ืช ืฉ ืค ื, Parashat No by Unknown (ID: 41747109)
2026-05-17 23:54:47,135 - KolHalashonScraper - INFO - [p3:13] Skip (not_requested_or_already_downloaded): 1:05:07 Harav Shneir Zalmen Weiss ืคืจืฉืช ื ืฉื, Parashat Noso Pa by Unknown (ID: 41747074)
2026-05-17 23:54:47,135 - KolHalashonScraper - INFO - [p3:14] Skip (not_requested_or_already_downloaded): 1:19 ืืจื ืืฉืจืื ืืื ืืื ืืืืืจื ืืืืื -ื ืกืืื ืชืฉืค''ื, Parashat by Unknown (ID: 41743566)
2026-05-17 23:54:47,135 - KolHalashonScraper - INFO - [p3:15] Skip (not_requested_or_already_downloaded): 50:40 Harav Mosha Fraind ืฉืืขืืจ ืืืจ ืืืืื ืคืจืฉืช ื ืฉื ืชืฉืค"ื, Par by Unknown (ID: 41747069)
2026-05-17 23:54:47,135 - KolHalashonScraper - INFO - [p3:16] Skip (not_requested_or_already_downloaded): 11:39 Harav Petachya Klein ืื ื ื ืฉื ืืืื ืงืืืื -ื ืกืืื ืฉืืืฆืืื by Unknown (ID: 41793350)
2026-05-17 23:54:47,135 - KolHalashonScraper - INFO - [p3:17] Skip (not_requested_or_already_downloaded): 54:38 Irgun Shiurai Torah ืืจื ืืืขืืจ ื ืืกื ืจืืืื - ืืืืฉ ืจืฉ''ื by Unknown (ID: 41747059)
2026-05-17 23:54:47,135 - KolHalashonScraper - INFO - [p3:18] Skip (not_requested_or_already_downloaded): 11:13 Harav Moshe Donnebaum The One-Liner LIfe Changing Rash by Unknown (ID: 41746982)
2026-05-17 23:54:47,135 - KolHalashonScraper - INFO - [p3:19] Skip (not_requested_or_already_downloaded): 39:24 Harav Meilech Landau Shiur 137 Leil Shishi Nusoi, Para by Unknown (ID: 41746957)
2026-05-17 23:54:47,135 - KolHalashonScraper - INFO - [p3:20] Skip (not_requested_or_already_downloaded): 1:19:31 Harav Yisroel Dovid Schlesinger ืึตืื ืฉืึดืฉึผืึดื - ืคึผึทืจ by Unknown (ID: 41747179)
2026-05-17 23:54:47,135 - KolHalashonScraper - INFO - [p3:21] Skip (not_requested_or_already_downloaded): 1:26:08 Harav Elimelech Biderman ืืื ืฉืืฉื ืคืจืฉืช ื ืฉื, Parashat by Unknown (ID: 41793118)
2026-05-17 23:54:47,135 - KolHalashonScraper - INFO - [p3:22] Skip (not_requested_or_already_downloaded): 37:30 Harav Shlomo Frishwasser ืคืจืฉืช ื ืฉื ืคืจืง ื ืคืกืืง ื, Parash by Unknown (ID: 41793082)
2026-05-17 23:54:47,135 - KolHalashonScraper - INFO - [p3:23] Skip (not_requested_or_already_downloaded): 30:26 Harav Yitzchak Mordechai Lebron 1075 ืืจืืช ืืื ืื, Paras by Unknown (ID: 41746892)
2026-05-17 23:54:47,141 - KolHalashonScraper - INFO - Navigating to page 4...
2026-05-17 23:54:47,175 - KolHalashonScraper - INFO - [DELAY] Waiting 10s for page to reload after clicking next button
Live tail active โ waiting for new lines...