← Logs

social_media_monitor_2026-09-17.log

2026-09-17 20:10:28,824 - __main__ - INFO - === Social Media Monitor v2.0 (Async) Started ===

2026-09-17 20:30:01,735 - social_media_monitor.utils.logging_config - INFO - Posts log file: /mnt/data/openclaw-shared/home/shared/Social_Media_Monitor/logs/social_media_monitor_posts_2026-09-17.log

2026-09-17 20:30:01,736 - social_media_monitor.utils.logging_config - INFO - Downloads log file: /mnt/data/openclaw-shared/home/shared/Social_Media_Monitor/logs/social_media_monitor_downloads_2026-09-17.log

2026-09-17 20:30:01,736 - social_media_monitor.utils.logging_config - INFO - Logging configured with level INFO, log file: /mnt/data/openclaw-shared/home/shared/Social_Media_Monitor/logs/social_media_monitor_2026-09-17.log

2026-09-17 20:30:01,736 - __main__ - INFO - === Social Media Monitor v2.0 (Async) Started ===

2026-09-17 20:30:01,771 - social_media_monitor.config.settings - INFO - Configuration loaded from /home/openclaw/Social_Media_Monitor/config.yaml

2026-09-17 20:30:01,771 - __main__ - INFO - Configuration loaded and validated successfully

2026-09-17 20:30:01,773 - social_media_monitor.database.connection - INFO - Database connection pool initialized with 5 connections

2026-09-17 20:30:01,775 - social_media_monitor.database.connection - INFO - add_speaker_filters column already exists

2026-09-17 20:30:01,789 - social_media_monitor.database.connection - INFO - Database setup completed successfully

2026-09-17 20:30:01,789 - __main__ - INFO - Database initialized at: /mnt/data/openclaw-shared/home/shared/Social_Media_Monitor/social_media_monitor.db

2026-09-17 20:30:01,793 - social_media_monitor.scrapers.reddit_scraper - INFO - Reddit API initialized successfully (UA='SocialMediaMonitor/1.0')

2026-09-17 20:30:01,794 - __main__ - INFO - GitHub scraper initialized and enabled

2026-09-17 20:30:01,794 - __main__ - INFO - GitHub feed scraper initialized and enabled

2026-09-17 20:30:01,794 - RSSFeedScraper - INFO - RSS scraper initialized: feeds=1, html_urls=0 (0 with custom selectors), include_terms=0, exclude_terms=0

2026-09-17 20:30:01,798 - Yiddish24Scraper - INFO - File downloader initialized with 55 terms, prefix 'Y2', and file type organization enabled

2026-09-17 20:30:01,798 - Yiddish24Scraper - INFO - Yiddish24 scraper initialized: data_id='latest', page_limit=100, total_pages=10, debug_html=True, include_terms=55, exclude_terms=0

2026-09-17 20:30:01,802 - AhBlickLiveScraper - INFO - AhBlickLive scraper initialized: urls=1, max_items=100, include_terms=49, exclude_terms=0

2026-09-17 20:30:01,808 - TorahAnytimeScraper - INFO - File downloader initialized with 53 terms, prefix 'TA', and file type organization enabled

2026-09-17 20:30:01,808 - TorahAnytimeScraper - INFO - TorahAnytime scraper initialized: limit=400, offset=0, project_id=1, include_terms=53, exclude_terms=13

2026-09-17 20:30:01,812 - BatorahScraper - INFO - File downloader initialized with 19 terms, prefix 'BT', and file type organization enabled

2026-09-17 20:30:01,812 - BatorahScraper - INFO - Batorah scraper initialized: api_urls=2, page_size=100, include_terms=19, exclude_terms=2

2026-09-17 20:30:01,812 - KolHalashonScraper - INFO - Kol Halashon scraper initialized: base_url=https://www2.kolhalashon.com, max_shiurim=50, fetch_all_speakers=True, headless=True, exclude_filters=1, include_filters=0

2026-09-17 20:30:01,815 - AllDafScraper - INFO - All Daf scraper initialized: enabled=True sections=['Daf Yomi Shiurim', 'Supplemental Clips', 'Reference Materials'] skip_terms=0 include_terms=0

2026-09-17 20:30:01,816 - __main__ - INFO - YouTube Intel scraper initialized and enabled

2026-09-17 20:30:01,816 - __main__ - INFO - Running 3 scrapers concurrently...

2026-09-17 20:30:01,816 - GitHubScraper - INFO - Searching GitHub for terms: ['"Hermes Agent"', '"grok bot OR grokbot"', '"muse NOT music NOT museum"', '"prime agent"', 'paperclip']

2026-09-17 20:30:02,241 - httpx - INFO - HTTP Request: GET https://api.github.com/user "HTTP/1.1 200 OK"

2026-09-17 20:30:02,242 - GitHubFeedScraper - INFO - GitHub auth preflight passed for abeperl; token scopes: not reported (fine-grained token)

2026-09-17 20:30:03,068 - httpx - INFO - HTTP Request: GET https://api.github.com/search/repositories?q=%22Hermes+Agent%22+in%3Aname%2Cdescription%2Creadme+stars%3A%3E150&sort=updated&order=desc&per_page=20 "HTTP/1.1 200 OK"

2026-09-17 20:30:03,168 - GitHubScraper - INFO - Found 615 GitHub repositories for term: "Hermes Agent"

2026-09-17 20:30:03,358 - httpx - INFO - HTTP Request: GET https://api.github.com/users/abeperl/received_events?per_page=100&page=1 "HTTP/1.1 200 OK"

2026-09-17 20:30:03,725 - httpx - INFO - HTTP Request: GET https://api.github.com/search/repositories?q=%22grok+bot+OR+grokbot%22+in%3Aname%2Cdescription%2Creadme+stars%3A%3E150&sort=updated&order=desc&per_page=20 "HTTP/1.1 200 OK"

2026-09-17 20:30:03,726 - GitHubScraper - INFO - Found 0 GitHub repositories for term: "grok bot OR grokbot"

2026-09-17 20:30:04,300 - httpx - INFO - HTTP Request: GET https://api.github.com/users/abeperl/received_events?per_page=100&page=2 "HTTP/1.1 200 OK"

2026-09-17 20:30:04,679 - httpx - INFO - HTTP Request: GET https://api.github.com/search/repositories?q=%22muse+NOT+music+NOT+museum%22+in%3Aname%2Cdescription%2Creadme+stars%3A%3E150&sort=updated&order=desc&per_page=20 "HTTP/1.1 200 OK"

2026-09-17 20:30:04,680 - GitHubScraper - INFO - Found 0 GitHub repositories for term: "muse NOT music NOT museum"

2026-09-17 20:30:05,306 - httpx - INFO - HTTP Request: GET https://api.github.com/users/abeperl/received_events?per_page=100&page=3 "HTTP/1.1 200 OK"

2026-09-17 20:30:06,253 - httpx - INFO - HTTP Request: GET https://api.github.com/search/repositories?q=%22prime+agent%22+in%3Aname%2Cdescription%2Creadme+stars%3A%3E150&sort=updated&order=desc&per_page=20 "HTTP/1.1 200 OK"

2026-09-17 20:30:06,256 - GitHubScraper - INFO - Found 41 GitHub repositories for term: "prime agent"

2026-09-17 20:30:07,346 - httpx - INFO - HTTP Request: GET https://api.github.com/search/repositories?q=paperclip+in%3Aname%2Cdescription%2Creadme+stars%3A%3E150&sort=updated&order=desc&per_page=20 "HTTP/1.1 200 OK"

2026-09-17 20:30:07,379 - GitHubScraper - INFO - Found 261 GitHub repositories for term: paperclip

2026-09-17 20:30:07,526 - GitHubScraper - INFO - Total GitHub results found: 0

2026-09-17 20:30:08,484 - httpx - INFO - HTTP Request: POST https://api.github.com/graphql "HTTP/1.1 200 OK"

2026-09-17 20:30:08,852 - httpx - INFO - HTTP Request: GET https://api.github.com/repos/yehoshuayaakov/mishnayos-signup/events?per_page=10 "HTTP/1.1 304 Not Modified"

2026-09-17 20:30:08,950 - httpx - INFO - HTTP Request: GET https://api.github.com/repos/efiles-star/ShasTracker/events?per_page=10 "HTTP/1.1 304 Not Modified"

2026-09-17 20:30:08,965 - httpx - INFO - HTTP Request: GET https://api.github.com/repos/Biswaspr1022/KodeshMode/events?per_page=10 "HTTP/1.1 304 Not Modified"

2026-09-17 20:30:08,996 - httpx - INFO - HTTP Request: GET https://api.github.com/repos/akivayevdayev-debug/Shelah_app/events?per_page=10 "HTTP/1.1 304 Not Modified"

2026-09-17 20:30:09,302 - httpx - INFO - HTTP Request: GET https://api.github.com/repos/nnanach/torah-bot/events?per_page=10 "HTTP/1.1 304 Not Modified"

2026-09-17 20:30:09,368 - httpx - INFO - HTTP Request: GET https://api.github.com/repos/Oranburg/havruta/events?per_page=10 "HTTP/1.1 304 Not Modified"

2026-09-17 20:30:09,373 - httpx - INFO - HTTP Request: GET https://api.github.com/repos/hitchin999/YidCal/events?per_page=10 "HTTP/1.1 200 OK"

2026-09-17 20:30:09,393 - httpx - INFO - HTTP Request: GET https://api.github.com/repos/azernjj/talmudai/events?per_page=10 "HTTP/1.1 304 Not Modified"

2026-09-17 20:30:09,707 - httpx - INFO - HTTP Request: GET https://api.github.com/repos/rabbishimon613-lang/chabad-tracker/events?per_page=10 "HTTP/1.1 304 Not Modified"

2026-09-17 20:30:09,724 - httpx - INFO - HTTP Request: GET https://api.github.com/repos/ethanpil/daily-zmanim/events?per_page=10 "HTTP/1.1 304 Not Modified"

2026-09-17 20:30:09,776 - httpx - INFO - HTTP Request: GET https://api.github.com/repos/adunnmobilenotary-gif/halacha-search/events?per_page=10 "HTTP/1.1 304 Not Modified"

2026-09-17 20:30:09,803 - httpx - INFO - HTTP Request: GET https://api.github.com/repos/Chabadrt/chabad-cms-backend/events?per_page=10 "HTTP/1.1 304 Not Modified"

2026-09-17 20:30:10,067 - httpx - INFO - HTTP Request: GET https://api.github.com/repos/IsraelCIL/SiddurAmIsrael/events?per_page=10 "HTTP/1.1 304 Not Modified"

2026-09-17 20:30:10,096 - httpx - INFO - HTTP Request: GET https://api.github.com/repos/ronrefael/kav-siddur-site/events?per_page=10 "HTTP/1.1 304 Not Modified"

2026-09-17 20:30:10,166 - httpx - INFO - HTTP Request: GET https://api.github.com/repos/kdroidFilter/Zayit/events?per_page=10 "HTTP/1.1 200 OK"

2026-09-17 20:30:10,185 - httpx - INFO - HTTP Request: GET https://api.github.com/repos/ElikoGrins/Halacha-bot/events?per_page=10 "HTTP/1.1 304 Not Modified"

2026-09-17 20:30:10,429 - httpx - INFO - HTTP Request: GET https://api.github.com/repos/EzraBrand/talmud-nlp-indexer/events?per_page=10 "HTTP/1.1 304 Not Modified"

2026-09-17 20:30:10,570 - httpx - INFO - HTTP Request: GET https://api.github.com/repos/brainheart/tanakh-contabulate/events?per_page=10 "HTTP/1.1 304 Not Modified"

2026-09-17 20:30:10,730 - httpx - INFO - HTTP Request: GET https://api.github.com/repos/Esoterics611/maayan/events?per_page=10 "HTTP/1.1 304 Not Modified"

2026-09-17 20:30:10,750 - httpx - INFO - HTTP Request: GET https://api.github.com/repos/sikmenedowai-a11y/savetube-stream-kit/events?per_page=10 "HTTP/1.1 200 OK"

2026-09-17 20:30:10,758 - httpx - INFO - HTTP Request: GET https://api.github.com/repos/Hbleiberg/Hebrew_Blender/events?per_page=10 "HTTP/1.1 200 OK"

2026-09-17 20:30:11,054 - httpx - INFO - HTTP Request: GET https://api.github.com/repos/stenagmois-byte/SimHebrew-Concordance/events?per_page=10 "HTTP/1.1 304 Not Modified"

2026-09-17 20:30:11,135 - httpx - INFO - HTTP Request: GET https://api.github.com/repos/wuisabel-gif/Aleph-Bet/events?per_page=10 "HTTP/1.1 304 Not Modified"

2026-09-17 20:30:11,150 - httpx - INFO - HTTP Request: GET https://api.github.com/repos/naorbrown/daf-yomi-history-bot/events?per_page=10 "HTTP/1.1 304 Not Modified"

2026-09-17 20:30:11,151 - httpx - INFO - HTTP Request: GET https://api.github.com/repos/jepick/talmud-map/events?per_page=10 "HTTP/1.1 304 Not Modified"

2026-09-17 20:30:11,551 - httpx - INFO - HTTP Request: GET https://api.github.com/repos/Asifah-Analytics-C-A-V-E/asifah-analytics/events?per_page=10 "HTTP/1.1 304 Not Modified"

2026-09-17 20:30:11,655 - httpx - INFO - HTTP Request: GET https://api.github.com/repos/shloimie-beep/bnei-neviim-academy/events?per_page=10 "HTTP/1.1 304 Not Modified"

2026-09-17 20:30:11,683 - httpx - INFO - HTTP Request: GET https://api.github.com/repos/paperlesspaper/integrations/events?per_page=10 "HTTP/1.1 200 OK"

2026-09-17 20:30:11,843 - httpx - INFO - HTTP Request: GET https://api.github.com/repos/complytime/complytime-collector-components/events?per_page=10 "HTTP/1.1 304 Not Modified"

2026-09-17 20:30:11,934 - httpx - INFO - HTTP Request: GET https://api.github.com/repos/DavidPatlas-AI/tehillim-russian-site/events?per_page=10 "HTTP/1.1 304 Not Modified"

2026-09-17 20:30:12,043 - httpx - INFO - HTTP Request: GET https://api.github.com/repos/dniasoff/learning-tracker/events?per_page=10 "HTTP/1.1 304 Not Modified"

2026-09-17 20:30:12,289 - httpx - INFO - HTTP Request: GET https://api.github.com/repos/hytric/NLP_term_project/events?per_page=10 "HTTP/1.1 304 Not Modified"

2026-09-17 20:30:12,332 - httpx - INFO - HTTP Request: GET https://api.github.com/repos/yossi-weinberger/ten10/events?per_page=10 "HTTP/1.1 304 Not Modified"

2026-09-17 20:30:12,410 - httpx - INFO - HTTP Request: GET https://api.github.com/repos/RonTurrentine/KJV-Strongs-EBook/events?per_page=10 "HTTP/1.1 304 Not Modified"

2026-09-17 20:30:12,428 - httpx - INFO - HTTP Request: GET https://api.github.com/repos/dmo18/MySugya/events?per_page=10 "HTTP/1.1 200 OK"

2026-09-17 20:30:12,655 - httpx - INFO - HTTP Request: GET https://api.github.com/repos/zarbinco/zarbin-seo/events?per_page=10 "HTTP/1.1 304 Not Modified"

2026-09-17 20:30:12,846 - httpx - INFO - HTTP Request: GET https://api.github.com/repos/v97969181-commits/master-pdf-editor-legacy-release/events?per_page=10 "HTTP/1.1 200 OK"

2026-09-17 20:30:12,902 - httpx - INFO - HTTP Request: GET https://api.github.com/repos/duvarakesh06/FAB-Subtitler-12.1-Offline-Release/events?per_page=10 "HTTP/1.1 200 OK"

2026-09-17 20:30:12,939 - httpx - INFO - HTTP Request: GET https://api.github.com/repos/elmadanimohammed78-del/daily-flow-landing/events?per_page=10 "HTTP/1.1 304 Not Modified"

2026-09-17 20:30:13,002 - httpx - INFO - HTTP Request: GET https://api.github.com/repos/AUX440/liquidtext-ultimate-patch-kit/events?per_page=10 "HTTP/1.1 200 OK"

2026-09-17 20:30:13,310 - httpx - INFO - HTTP Request: GET https://api.github.com/repos/apusaha0011/MenuSideKick/events?per_page=10 "HTTP/1.1 304 Not Modified"

2026-09-17 20:30:13,328 - httpx - INFO - HTTP Request: GET https://api.github.com/repos/rafistrauss/commons-announcements/events?per_page=10 "HTTP/1.1 304 Not Modified"

2026-09-17 20:30:13,438 - httpx - INFO - HTTP Request: GET https://api.github.com/repos/Roger0129/bitwarden-2024-8-0-patchless-install/events?per_page=10 "HTTP/1.1 200 OK"

2026-09-17 20:30:13,527 - httpx - INFO - HTTP Request: GET https://api.github.com/repos/Aljademaratas/readiris-product-code-generator/events?per_page=10 "HTTP/1.1 200 OK"

2026-09-17 20:30:13,655 - httpx - INFO - HTTP Request: GET https://api.github.com/repos/chfle/chfle/events?per_page=10 "HTTP/1.1 304 Not Modified"

2026-09-17 20:30:13,836 - httpx - INFO - HTTP Request: GET https://api.github.com/repos/revanite-io/pvtr-gcp-cloud-storage/events?per_page=10 "HTTP/1.1 304 Not Modified"

2026-09-17 20:30:13,868 - httpx - INFO - HTTP Request: GET https://api.github.com/repos/semiagenticRob/100Reps/events?per_page=10 "HTTP/1.1 304 Not Modified"

2026-09-17 20:30:13,935 - httpx - INFO - HTTP Request: GET https://api.github.com/repos/benemanuel/status/events?per_page=10 "HTTP/1.1 304 Not Modified"

2026-09-17 20:30:14,035 - httpx - INFO - HTTP Request: GET https://api.github.com/repos/Jossifresben/translation-alignment/events?per_page=10 "HTTP/1.1 304 Not Modified"

2026-09-17 20:30:14,233 - httpx - INFO - HTTP Request: GET https://api.github.com/repos/egkristi/ToponymiaEuropaea/events?per_page=10 "HTTP/1.1 304 Not Modified"

2026-09-17 20:30:14,315 - httpx - INFO - HTTP Request: GET https://api.github.com/repos/KosherJava/zmanim/events?per_page=10 "HTTP/1.1 304 Not Modified"

2026-09-17 20:30:14,338 - httpx - INFO - HTTP Request: GET https://api.github.com/repos/Israel-Kli/shabbat-timetable/events?per_page=10 "HTTP/1.1 304 Not Modified"

2026-09-17 20:30:14,361 - httpx - INFO - HTTP Request: GET https://api.github.com/repos/hebcal/hebcal/events?per_page=10 "HTTP/1.1 304 Not Modified"

2026-09-17 20:30:14,616 - httpx - INFO - HTTP Request: GET https://api.github.com/repos/woodpeckerbros/watch-reminder/events?per_page=10 "HTTP/1.1 304 Not Modified"

2026-09-17 20:30:14,690 - httpx - INFO - HTTP Request: GET https://api.github.com/repos/svenalot-ui/sidur-native/events?per_page=10 "HTTP/1.1 304 Not Modified"

2026-09-17 20:30:14,693 - httpx - INFO - HTTP Request: GET https://api.github.com/repos/hebcal/learning/events?per_page=10 "HTTP/1.1 304 Not Modified"

2026-09-17 20:30:14,797 - httpx - INFO - HTTP Request: GET https://api.github.com/repos/hebcal/hebcal-go/events?per_page=10 "HTTP/1.1 304 Not Modified"

2026-09-17 20:30:15,078 - httpx - INFO - HTTP Request: GET https://api.github.com/repos/Sefaria/ai-chatbot/events?per_page=10 "HTTP/1.1 200 OK"

2026-09-17 20:30:15,142 - httpx - INFO - HTTP Request: GET https://api.github.com/repos/joelcw/intsayt/events?per_page=10 "HTTP/1.1 200 OK"

2026-09-17 20:30:15,236 - httpx - INFO - HTTP Request: GET https://api.github.com/repos/levidao-eth/Torah/events?per_page=10 "HTTP/1.1 304 Not Modified"

2026-09-17 20:30:15,267 - httpx - INFO - HTTP Request: GET https://api.github.com/repos/complytime/complytime-policies/events?per_page=10 "HTTP/1.1 304 Not Modified"

2026-09-17 20:30:15,483 - httpx - INFO - HTTP Request: GET https://api.github.com/repos/AdamYairLee/torah-korean-translation/events?per_page=10 "HTTP/1.1 304 Not Modified"

2026-09-17 20:30:15,535 - httpx - INFO - HTTP Request: GET https://api.github.com/repos/zakdev26/sefaria-links/events?per_page=10 "HTTP/1.1 304 Not Modified"

2026-09-17 20:30:15,537 - httpx - INFO - HTTP Request: GET https://api.github.com/repos/ainihonuragawa-lang/TalmudFinance/events?per_page=10 "HTTP/1.1 304 Not Modified"

2026-09-17 20:30:15,683 - httpx - INFO - HTTP Request: GET https://api.github.com/repos/vaisisrael/gemara-benichuta/events?per_page=10 "HTTP/1.1 304 Not Modified"

2026-09-17 20:30:15,761 - YouTubeIntelScraper - INFO - YouTube Intel found 22 candidates and 3 new videos

2026-09-17 20:30:15,779 - httpx - INFO - HTTP Request: GET https://api.github.com/repos/Yiddish-Dialect-Dictionary/-brukhe-machine/events?per_page=10 "HTTP/1.1 304 Not Modified"

2026-09-17 20:30:15,912 - httpx - INFO - HTTP Request: GET https://api.github.com/repos/SnirRadomsky/daf-yomi/events?per_page=10 "HTTP/1.1 304 Not Modified"

2026-09-17 20:30:15,949 - httpx - INFO - HTTP Request: GET https://api.github.com/repos/elconde/kosher-races/events?per_page=10 "HTTP/1.1 304 Not Modified"

2026-09-17 20:30:15,985 - httpx - INFO - HTTP Request: GET https://api.github.com/repos/jacopovirga/kindle-zmanim/events?per_page=10 "HTTP/1.1 304 Not Modified"

2026-09-17 20:30:16,173 - httpx - INFO - HTTP Request: GET https://api.github.com/repos/Selecr-Surplus/minyan-finder/events?per_page=10 "HTTP/1.1 304 Not Modified"

2026-09-17 20:30:16,182 - httpx - INFO - HTTP Request: GET https://api.github.com/repos/robmoffat/FOSS-Contribution-Gemara/events?per_page=10 "HTTP/1.1 304 Not Modified"

2026-09-17 20:30:16,273 - httpx - INFO - HTTP Request: GET https://api.github.com/repos/theOnionsAreWatching/Shtigletz/events?per_page=10 "HTTP/1.1 304 Not Modified"

2026-09-17 20:30:16,345 - httpx - INFO - HTTP Request: GET https://api.github.com/repos/aharonium/opensiddur.org/events?per_page=10 "HTTP/1.1 304 Not Modified"

2026-09-17 20:30:16,545 - httpx - INFO - HTTP Request: GET https://api.github.com/repos/naorbrown/peninei-halacha-yomi-bot/events?per_page=10 "HTTP/1.1 304 Not Modified"

2026-09-17 20:30:16,553 - httpx - INFO - HTTP Request: GET https://api.github.com/repos/mig2/mishnah-style/events?per_page=10 "HTTP/1.1 304 Not Modified"

2026-09-17 20:30:16,649 - httpx - INFO - HTTP Request: GET https://api.github.com/repos/vistaTeam/satmar/events?per_page=10 "HTTP/1.1 304 Not Modified"

2026-09-17 20:30:16,734 - httpx - INFO - HTTP Request: GET https://api.github.com/repos/skilead770/stoliskikosher/events?per_page=10 "HTTP/1.1 304 Not Modified"

2026-09-17 20:30:16,973 - httpx - INFO - HTTP Request: GET https://api.github.com/repos/mak-sandboxuser/chabad-react-and-supabase/events?per_page=10 "HTTP/1.1 304 Not Modified"

2026-09-17 20:30:17,015 - httpx - INFO - HTTP Request: GET https://api.github.com/repos/bnq9705-creator/siddur/events?per_page=10 "HTTP/1.1 304 Not Modified"

2026-09-17 20:30:17,022 - httpx - INFO - HTTP Request: GET https://api.github.com/repos/a0556770172-png/koshermat/events?per_page=10 "HTTP/1.1 304 Not Modified"

2026-09-17 20:30:17,343 - httpx - INFO - HTTP Request: GET https://api.github.com/repos/yeled-tov/filtertube-android/events?per_page=10 "HTTP/1.1 200 OK"

2026-09-17 20:30:17,368 - httpx - INFO - HTTP Request: GET https://api.github.com/repos/Kosher-T/fannexus/events?per_page=10 "HTTP/1.1 200 OK"

2026-09-17 20:30:17,373 - httpx - INFO - HTTP Request: GET https://api.github.com/repos/yosefmalul52-lang/costa-rica-kosher-retreat/events?per_page=10 "HTTP/1.1 304 Not Modified"

2026-09-17 20:30:17,568 - httpx - INFO - HTTP Request: GET https://api.github.com/repos/RavBogard/shirei-tfilah/events?per_page=10 "HTTP/1.1 304 Not Modified"

2026-09-17 20:30:17,644 - httpx - INFO - HTTP Request: GET https://api.github.com/repos/AviMish10/tanach-map/events?per_page=10 "HTTP/1.1 304 Not Modified"

2026-09-17 20:30:17,697 - httpx - INFO - HTTP Request: GET https://api.github.com/repos/shliachflow/chabad-southern-delaware-camp/events?per_page=10 "HTTP/1.1 304 Not Modified"

2026-09-17 20:30:17,725 - httpx - INFO - HTTP Request: GET https://api.github.com/repos/ReuvenB-S/kosher-browser-data/events?per_page=10 "HTTP/1.1 304 Not Modified"

2026-09-17 20:30:17,918 - httpx - INFO - HTTP Request: GET https://api.github.com/repos/TanachatGKRB/TanachatGKRB.github.io/events?per_page=10 "HTTP/1.1 304 Not Modified"

2026-09-17 20:30:18,054 - httpx - INFO - HTTP Request: GET https://api.github.com/repos/TamamTamar/chabad-yafo/events?per_page=10 "HTTP/1.1 304 Not Modified"

2026-09-17 20:30:18,076 - httpx - INFO - HTTP Request: GET https://api.github.com/repos/TG-CONSULTANT/tanachok-stock-FG1/events?per_page=10 "HTTP/1.1 304 Not Modified"

2026-09-17 20:30:18,096 - httpx - INFO - HTTP Request: GET https://api.github.com/repos/DeepSynchronicity/chabadofhurlingham/events?per_page=10 "HTTP/1.1 304 Not Modified"

2026-09-17 20:30:18,210 - httpx - INFO - HTTP Request: GET https://api.github.com/repos/semiagenticRob/digital-siddur/events?per_page=10 "HTTP/1.1 304 Not Modified"

2026-09-17 20:30:18,398 - httpx - INFO - HTTP Request: GET https://api.github.com/repos/orelzion/daily-tanach/events?per_page=10 "HTTP/1.1 304 Not Modified"

2026-09-17 20:30:18,441 - httpx - INFO - HTTP Request: GET https://api.github.com/repos/TG-CONSULTANT/tanachok-stock-mat/events?per_page=10 "HTTP/1.1 304 Not Modified"

2026-09-17 20:30:18,531 - httpx - INFO - HTTP Request: GET https://api.github.com/repos/Frish-Gebakn/kol-satmar-website/events?per_page=10 "HTTP/1.1 304 Not Modified"

2026-09-17 20:30:18,589 - httpx - INFO - HTTP Request: GET https://api.github.com/repos/satmarthubby-blip/Satmarthubby/events?per_page=10 "HTTP/1.1 304 Not Modified"

2026-09-17 20:30:18,743 - httpx - INFO - HTTP Request: GET https://api.github.com/repos/akshatasatmar/WeatherServiceApi/events?per_page=10 "HTTP/1.1 304 Not Modified"

2026-09-17 20:30:18,798 - httpx - INFO - HTTP Request: GET https://api.github.com/repos/elchunenfreund/satmar-matzos-app/events?per_page=10 "HTTP/1.1 304 Not Modified"

2026-09-17 20:30:18,934 - httpx - INFO - HTTP Request: GET https://api.github.com/repos/satmaribianca05-eng/AFSD-Satmari-Bianca/events?per_page=10 "HTTP/1.1 304 Not Modified"

2026-09-17 20:30:19,075 - httpx - INFO - HTTP Request: GET https://api.github.com/repos/Sefaria/Sefaria-Export/events?per_page=10 "HTTP/1.1 304 Not Modified"

2026-09-17 20:30:19,190 - GitHubFeedScraper - INFO - GitHub feed healthy: login=abeperl fetched=300 account_new=0 list_repos=100 list_new=0 private_new=0 total_new=0

2026-09-17 20:30:19,191 - __main__ - INFO - Scraper github completed: 0 results

2026-09-17 20:30:19,192 - __main__ - INFO - Scraper github_feed completed: 0 results

2026-09-17 20:30:19,192 - __main__ - INFO - Scraper youtube_intel completed: 3 results

2026-09-17 20:30:20,708 - social_media_monitor.email.sender - INFO - Email sent successfully at 2026-09-17 20:30:20.708362

2026-09-17 20:30:20,708 - __main__ - INFO - Email sent successfully

2026-09-17 20:30:20,708 - __main__ - INFO - Cache stats: 0 items in memory, 0 files on disk

2026-09-17 20:30:20,709 - __main__ - INFO - Monitor complete. Found 3 total results: 3 new youtube intel results.

2026-09-17 20:30:20,709 - __main__ - INFO - === Social Media Monitor v2.0 Complete ===

2026-09-17 20:30:20,709 - social_media_monitor.database.connection - INFO - Closing all database connections

2026-09-17 20:30:20,733 - social_media_monitor.database.connection - INFO - Closed 5 database connections

2026-09-17 20:30:20,734 - __main__ - INFO - Closed all database connections

2026-09-17 20:30:20,736 - social_media_monitor.database.connection - INFO - Closing all database connections

2026-09-17 20:30:20,736 - social_media_monitor.database.connection - INFO - Closed 0 database connections

2026-09-17 21:00:02,287 - social_media_monitor.utils.logging_config - INFO - Posts log file: /mnt/data/openclaw-shared/home/shared/Social_Media_Monitor/logs/social_media_monitor_posts_2026-09-17.log

2026-09-17 21:00:02,287 - social_media_monitor.utils.logging_config - INFO - Downloads log file: /mnt/data/openclaw-shared/home/shared/Social_Media_Monitor/logs/social_media_monitor_downloads_2026-09-17.log

2026-09-17 21:00:02,287 - social_media_monitor.utils.logging_config - INFO - Logging configured with level INFO, log file: /mnt/data/openclaw-shared/home/shared/Social_Media_Monitor/logs/social_media_monitor_2026-09-17.log

2026-09-17 21:00:02,287 - __main__ - INFO - === Social Media Monitor v2.0 (Async) Started ===

2026-09-17 21:00:02,317 - social_media_monitor.config.settings - INFO - Configuration loaded from /home/openclaw/Social_Media_Monitor/config.yaml

2026-09-17 21:00:02,318 - __main__ - INFO - Configuration loaded and validated successfully

2026-09-17 21:00:02,320 - social_media_monitor.database.connection - INFO - Database connection pool initialized with 5 connections

2026-09-17 21:00:02,321 - social_media_monitor.database.connection - INFO - add_speaker_filters column already exists

2026-09-17 21:00:02,346 - social_media_monitor.database.connection - INFO - Database setup completed successfully

2026-09-17 21:00:02,346 - __main__ - INFO - Database initialized at: /mnt/data/openclaw-shared/home/shared/Social_Media_Monitor/social_media_monitor.db

2026-09-17 21:00:02,349 - social_media_monitor.scrapers.reddit_scraper - INFO - Reddit API initialized successfully (UA='SocialMediaMonitor/1.0')

2026-09-17 21:00:02,350 - RSSFeedScraper - INFO - RSS scraper initialized: feeds=1, html_urls=0 (0 with custom selectors), include_terms=0, exclude_terms=0

2026-09-17 21:00:02,350 - __main__ - INFO - RSS scraper initialized and enabled

2026-09-17 21:00:02,352 - Yiddish24Scraper - INFO - File downloader initialized with 55 terms, prefix 'Y2', and file type organization enabled

2026-09-17 21:00:02,352 - Yiddish24Scraper - INFO - Yiddish24 scraper initialized: data_id='latest', page_limit=100, total_pages=10, debug_html=True, include_terms=55, exclude_terms=0

2026-09-17 21:00:02,352 - __main__ - INFO - Yiddish24 scraper initialized and enabled

2026-09-17 21:00:02,355 - AhBlickLiveScraper - INFO - AhBlickLive scraper initialized: urls=1, max_items=100, include_terms=49, exclude_terms=0

2026-09-17 21:00:02,359 - TorahAnytimeScraper - INFO - File downloader initialized with 53 terms, prefix 'TA', and file type organization enabled

2026-09-17 21:00:02,359 - TorahAnytimeScraper - INFO - TorahAnytime scraper initialized: limit=400, offset=0, project_id=1, include_terms=53, exclude_terms=13

2026-09-17 21:00:02,361 - BatorahScraper - INFO - File downloader initialized with 19 terms, prefix 'BT', and file type organization enabled

2026-09-17 21:00:02,361 - BatorahScraper - INFO - Batorah scraper initialized: api_urls=2, page_size=100, include_terms=19, exclude_terms=2

2026-09-17 21:00:02,362 - KolHalashonScraper - INFO - Kol Halashon scraper initialized: base_url=https://www2.kolhalashon.com, max_shiurim=50, fetch_all_speakers=True, headless=True, exclude_filters=1, include_filters=0

2026-09-17 21:00:02,364 - AllDafScraper - INFO - All Daf scraper initialized: enabled=True sections=['Daf Yomi Shiurim', 'Supplemental Clips', 'Reference Materials'] skip_terms=0 include_terms=0

2026-09-17 21:00:02,364 - __main__ - INFO - Ivelt scraper initialized and enabled

2026-09-17 21:00:02,364 - __main__ - INFO - Running 3 scrapers concurrently...

2026-09-17 21:00:02,364 - Yiddish24Scraper - INFO - Starting Yiddish24 scraping (max 10 pages, limit 100 items)

2026-09-17 21:00:02,470 - Yiddish24Scraper - INFO - Fetching Yiddish24 page 1 with data_id='latest', page_limit=100

2026-09-17 21:00:02,475 - IveltScraper - INFO - Loaded 181 include, 4 exclude filters from DB topic_filters

2026-09-17 21:00:02,475 - IveltScraper - INFO - Fetching ivelt active topics from https://www.ivelt.com/forum/search.php?search_id=active_topics...

2026-09-17 21:00:02,477 - RSSFeedScraper - INFO - Fetching RSS feed: https://www.theyeshivaworld.com/feed

2026-09-17 21:00:02,836 - RSSFeedScraper - INFO - Total RSS/HTML results: 0

2026-09-17 21:00:04,305 - httpx - INFO - HTTP Request: POST https://www.yiddish24.com/ajax/cat_pagination.php "HTTP/1.1 200 OK"

2026-09-17 21:00:04,526 - Yiddish24Scraper - INFO - Received response with 497043 characters

2026-09-17 21:00:04,529 - Yiddish24Scraper - INFO - Extracted HTML from JSON result field (315867 chars)

2026-09-17 21:00:04,532 - Yiddish24Scraper - INFO - Saved HTML debug file to: /mnt/data/openclaw-shared/home/shared/Social_Media_Monitor/debug/yiddish24_page1_response.html

2026-09-17 21:00:04,612 - Yiddish24Scraper - INFO - Found 100 potential article elements on page 1

2026-09-17 21:00:04,654 - Yiddish24Scraper - INFO - Page 1 extraction: 100 successful, 0 failed out of 100 elements

2026-09-17 21:00:04,654 - Yiddish24Scraper - INFO - Parsed 100 articles from HTML on page 1

2026-09-17 21:00:04,686 - Yiddish24Scraper - INFO - Term 'אינטערעסאנטס -' matched in DESCRIPTION: 'אינטערעסאנטס - פראדעצורן דעם אטאם באמבע'

2026-09-17 21:00:04,686 - Yiddish24Scraper - INFO - Downloading 1 media file(s) for article: הכל בכל

2026-09-17 21:00:05,206 - social_media_monitor.utils.file_downloader - INFO - Starting download from: https://cloudfront.yiddish24.com/_________________260_260917030027.mp3

2026-09-17 21:00:07,074 - social_media_monitor.utils.file_downloader - INFO - Successfully downloaded: Y2__________________260_260917030027.mp3 (33.2 MB)

2026-09-17 21:00:07,074 - social_media_monitor.utils.file_downloader - INFO - Organized audio file to: /mnt/data/openclaw-shared/home/shared/Social_Media_Monitor/downloads/Audio (audio/mpeg)

2026-09-17 21:00:07,094 - social_media_monitor.utils.media_tagger - INFO - Tagged Y2__________________260_260917030027.mp3: title, artist, album, comment, source_url, genre, publisher

2026-09-17 21:00:07,094 - Yiddish24Scraper - INFO - Renamed downloaded file to include match term: Y2__________________260_260917030027_אינטערעסאנטס_-.mp3

2026-09-17 21:00:07,828 - Yiddish24Scraper - INFO - Page 1 summary: 3 new out of 100 total articles

2026-09-17 21:00:07,828 - Yiddish24Scraper - INFO - Page 1: 3/100 new items

2026-09-17 21:00:07,828 - Yiddish24Scraper - INFO - Low new item ratio (3.0%), stopping pagination

2026-09-17 21:00:07,829 - Yiddish24Scraper - INFO - Total Yiddish24 results found: 3

2026-09-17 21:00:14,657 - IveltScraper - INFO - Found 50 active topics on ivelt

2026-09-17 21:00:32,596 - IveltScraper - INFO - ivelt scrape complete. Checked 4 topics.

2026-09-17 21:00:32,596 - IveltScraper - INFO - All forums scrape complete. Found 20 new posts.

2026-09-17 21:00:32,596 - __main__ - INFO - Scraper rss completed: 0 results

2026-09-17 21:00:32,596 - __main__ - INFO - Scraper yiddish24 completed: 3 results

2026-09-17 21:00:32,596 - __main__ - INFO - Scraper ivelt completed: 20 results

2026-09-17 21:00:33,508 - social_media_monitor.email.sender - INFO - Email sent successfully at 2026-09-17 21:00:33.508338

2026-09-17 21:00:33,508 - __main__ - INFO - Email sent successfully

2026-09-17 21:00:33,508 - __main__ - INFO - Cache stats: 0 items in memory, 0 files on disk

2026-09-17 21:00:33,509 - __main__ - INFO - Monitor complete. Found 23 total results: 3 new yiddish24 results, 20 new ivelt results.

2026-09-17 21:00:33,509 - __main__ - INFO - === Social Media Monitor v2.0 Complete ===

2026-09-17 21:00:33,509 - social_media_monitor.database.connection - INFO - Closing all database connections

2026-09-17 21:00:33,582 - social_media_monitor.database.connection - INFO - Closed 5 database connections

2026-09-17 21:00:33,582 - __main__ - INFO - Closed all database connections

2026-09-17 21:00:33,583 - social_media_monitor.database.connection - INFO - Closing all database connections

2026-09-17 21:00:33,584 - social_media_monitor.database.connection - INFO - Closed 0 database connections

2026-09-17 22:00:02,337 - social_media_monitor.utils.logging_config - INFO - Posts log file: /mnt/data/openclaw-shared/home/shared/Social_Media_Monitor/logs/social_media_monitor_posts_2026-09-17.log

2026-09-17 22:00:02,338 - social_media_monitor.utils.logging_config - INFO - Downloads log file: /mnt/data/openclaw-shared/home/shared/Social_Media_Monitor/logs/social_media_monitor_downloads_2026-09-17.log

2026-09-17 22:00:02,338 - social_media_monitor.utils.logging_config - INFO - Logging configured with level INFO, log file: /mnt/data/openclaw-shared/home/shared/Social_Media_Monitor/logs/social_media_monitor_2026-09-17.log

2026-09-17 22:00:02,338 - __main__ - INFO - === Social Media Monitor v2.0 (Async) Started ===

2026-09-17 22:00:02,374 - social_media_monitor.config.settings - INFO - Configuration loaded from /home/openclaw/Social_Media_Monitor/config.yaml

2026-09-17 22:00:02,374 - __main__ - INFO - Configuration loaded and validated successfully

2026-09-17 22:00:02,376 - social_media_monitor.database.connection - INFO - Database connection pool initialized with 5 connections

2026-09-17 22:00:02,377 - social_media_monitor.database.connection - INFO - add_speaker_filters column already exists

2026-09-17 22:00:02,389 - social_media_monitor.database.connection - INFO - Database setup completed successfully

2026-09-17 22:00:02,389 - __main__ - INFO - Database initialized at: /mnt/data/openclaw-shared/home/shared/Social_Media_Monitor/social_media_monitor.db

2026-09-17 22:00:02,443 - social_media_monitor.scrapers.reddit_scraper - INFO - Reddit API initialized successfully (UA='SocialMediaMonitor/1.0')

2026-09-17 22:00:02,445 - RSSFeedScraper - INFO - RSS scraper initialized: feeds=1, html_urls=0 (0 with custom selectors), include_terms=0, exclude_terms=0

2026-09-17 22:00:02,445 - __main__ - INFO - RSS scraper initialized and enabled

2026-09-17 22:00:02,454 - Yiddish24Scraper - INFO - File downloader initialized with 55 terms, prefix 'Y2', and file type organization enabled

2026-09-17 22:00:02,454 - Yiddish24Scraper - INFO - Yiddish24 scraper initialized: data_id='latest', page_limit=100, total_pages=10, debug_html=True, include_terms=55, exclude_terms=0

2026-09-17 22:00:02,454 - __main__ - INFO - Yiddish24 scraper initialized and enabled

2026-09-17 22:00:02,461 - AhBlickLiveScraper - INFO - AhBlickLive scraper initialized: urls=1, max_items=100, include_terms=49, exclude_terms=0

2026-09-17 22:00:02,467 - TorahAnytimeScraper - INFO - File downloader initialized with 53 terms, prefix 'TA', and file type organization enabled

2026-09-17 22:00:02,467 - TorahAnytimeScraper - INFO - TorahAnytime scraper initialized: limit=400, offset=0, project_id=1, include_terms=53, exclude_terms=13

2026-09-17 22:00:02,470 - BatorahScraper - INFO - File downloader initialized with 19 terms, prefix 'BT', and file type organization enabled

2026-09-17 22:00:02,470 - BatorahScraper - INFO - Batorah scraper initialized: api_urls=2, page_size=100, include_terms=19, exclude_terms=2

2026-09-17 22:00:02,470 - KolHalashonScraper - INFO - Kol Halashon scraper initialized: base_url=https://www2.kolhalashon.com, max_shiurim=50, fetch_all_speakers=True, headless=True, exclude_filters=1, include_filters=0

2026-09-17 22:00:02,472 - AllDafScraper - INFO - All Daf scraper initialized: enabled=True sections=['Daf Yomi Shiurim', 'Supplemental Clips', 'Reference Materials'] skip_terms=0 include_terms=0

2026-09-17 22:00:02,472 - __main__ - INFO - Ivelt scraper initialized and enabled

2026-09-17 22:00:02,473 - __main__ - INFO - Running 3 scrapers concurrently...

2026-09-17 22:00:02,473 - Yiddish24Scraper - INFO - Starting Yiddish24 scraping (max 10 pages, limit 100 items)

2026-09-17 22:00:02,569 - Yiddish24Scraper - INFO - Fetching Yiddish24 page 1 with data_id='latest', page_limit=100

2026-09-17 22:00:02,576 - IveltScraper - INFO - Loaded 181 include, 4 exclude filters from DB topic_filters

2026-09-17 22:00:02,576 - IveltScraper - INFO - Fetching ivelt active topics from https://www.ivelt.com/forum/search.php?search_id=active_topics...

2026-09-17 22:00:02,579 - RSSFeedScraper - INFO - Fetching RSS feed: https://www.theyeshivaworld.com/feed

2026-09-17 22:00:02,822 - RSSFeedScraper - INFO - Total RSS/HTML results: 0

2026-09-17 22:00:03,226 - httpx - INFO - HTTP Request: POST https://www.yiddish24.com/ajax/cat_pagination.php "HTTP/1.1 200 OK"

2026-09-17 22:00:03,448 - Yiddish24Scraper - INFO - Received response with 498709 characters

2026-09-17 22:00:03,452 - Yiddish24Scraper - INFO - Extracted HTML from JSON result field (313694 chars)

2026-09-17 22:00:03,454 - Yiddish24Scraper - INFO - Saved HTML debug file to: /mnt/data/openclaw-shared/home/shared/Social_Media_Monitor/debug/yiddish24_page1_response.html

2026-09-17 22:00:03,545 - Yiddish24Scraper - INFO - Found 100 potential article elements on page 1

2026-09-17 22:00:03,595 - Yiddish24Scraper - INFO - Page 1 extraction: 100 successful, 0 failed out of 100 elements

2026-09-17 22:00:03,595 - Yiddish24Scraper - INFO - Parsed 100 articles from HTML on page 1

2026-09-17 22:00:04,797 - Yiddish24Scraper - INFO - Page 1 summary: 3 new out of 100 total articles

2026-09-17 22:00:04,797 - Yiddish24Scraper - INFO - Page 1: 3/100 new items

2026-09-17 22:00:04,797 - Yiddish24Scraper - INFO - Low new item ratio (3.0%), stopping pagination

2026-09-17 22:00:04,798 - Yiddish24Scraper - INFO - Total Yiddish24 results found: 3

2026-09-17 22:00:14,587 - IveltScraper - INFO - Found 50 active topics on ivelt

2026-09-17 22:00:32,527 - IveltScraper - INFO - ivelt scrape complete. Checked 3 topics.

2026-09-17 22:00:32,527 - IveltScraper - INFO - All forums scrape complete. Found 8 new posts.

2026-09-17 22:00:32,528 - __main__ - INFO - Scraper rss completed: 0 results

2026-09-17 22:00:32,528 - __main__ - INFO - Scraper yiddish24 completed: 3 results

2026-09-17 22:00:32,528 - __main__ - INFO - Scraper ivelt completed: 8 results

2026-09-17 22:00:33,530 - social_media_monitor.email.sender - INFO - Email sent successfully at 2026-09-17 22:00:33.530709

2026-09-17 22:00:33,530 - __main__ - INFO - Email sent successfully

2026-09-17 22:00:33,531 - __main__ - INFO - Cache stats: 0 items in memory, 0 files on disk

2026-09-17 22:00:33,531 - __main__ - INFO - Monitor complete. Found 11 total results: 3 new yiddish24 results, 8 new ivelt results.

2026-09-17 22:00:33,531 - __main__ - INFO - === Social Media Monitor v2.0 Complete ===

2026-09-17 22:00:33,531 - social_media_monitor.database.connection - INFO - Closing all database connections

2026-09-17 22:00:33,591 - social_media_monitor.database.connection - INFO - Closed 5 database connections

2026-09-17 22:00:33,591 - __main__ - INFO - Closed all database connections

2026-09-17 22:00:33,593 - social_media_monitor.database.connection - INFO - Closing all database connections

2026-09-17 22:00:33,593 - social_media_monitor.database.connection - INFO - Closed 0 database connections

2026-09-17 22:30:02,532 - social_media_monitor.utils.logging_config - INFO - Posts log file: /mnt/data/openclaw-shared/home/shared/Social_Media_Monitor/logs/social_media_monitor_posts_2026-09-17.log

2026-09-17 22:30:02,532 - social_media_monitor.utils.logging_config - INFO - Downloads log file: /mnt/data/openclaw-shared/home/shared/Social_Media_Monitor/logs/social_media_monitor_downloads_2026-09-17.log

2026-09-17 22:30:02,532 - social_media_monitor.utils.logging_config - INFO - Logging configured with level INFO, log file: /mnt/data/openclaw-shared/home/shared/Social_Media_Monitor/logs/social_media_monitor_2026-09-17.log

2026-09-17 22:30:02,532 - __main__ - INFO - === Social Media Monitor v2.0 (Async) Started ===

2026-09-17 22:30:02,561 - social_media_monitor.config.settings - INFO - Configuration loaded from /home/openclaw/Social_Media_Monitor/config.yaml

2026-09-17 22:30:02,561 - __main__ - INFO - Configuration loaded and validated successfully

2026-09-17 22:30:02,563 - social_media_monitor.database.connection - INFO - Database connection pool initialized with 5 connections

2026-09-17 22:30:02,563 - social_media_monitor.database.connection - INFO - add_speaker_filters column already exists

2026-09-17 22:30:02,579 - social_media_monitor.database.connection - INFO - Database setup completed successfully

2026-09-17 22:30:02,579 - __main__ - INFO - Database initialized at: /mnt/data/openclaw-shared/home/shared/Social_Media_Monitor/social_media_monitor.db

2026-09-17 22:30:02,583 - social_media_monitor.scrapers.reddit_scraper - INFO - Reddit API initialized successfully (UA='SocialMediaMonitor/1.0')

2026-09-17 22:30:02,585 - __main__ - INFO - GitHub scraper initialized and enabled

2026-09-17 22:30:02,585 - __main__ - INFO - GitHub feed scraper initialized and enabled

2026-09-17 22:30:02,585 - RSSFeedScraper - INFO - RSS scraper initialized: feeds=1, html_urls=0 (0 with custom selectors), include_terms=0, exclude_terms=0

2026-09-17 22:30:02,590 - Yiddish24Scraper - INFO - File downloader initialized with 55 terms, prefix 'Y2', and file type organization enabled

2026-09-17 22:30:02,590 - Yiddish24Scraper - INFO - Yiddish24 scraper initialized: data_id='latest', page_limit=100, total_pages=10, debug_html=True, include_terms=55, exclude_terms=0

2026-09-17 22:30:02,596 - AhBlickLiveScraper - INFO - AhBlickLive scraper initialized: urls=1, max_items=100, include_terms=49, exclude_terms=0

2026-09-17 22:30:02,601 - TorahAnytimeScraper - INFO - File downloader initialized with 53 terms, prefix 'TA', and file type organization enabled

2026-09-17 22:30:02,601 - TorahAnytimeScraper - INFO - TorahAnytime scraper initialized: limit=400, offset=0, project_id=1, include_terms=53, exclude_terms=13

2026-09-17 22:30:02,603 - BatorahScraper - INFO - File downloader initialized with 19 terms, prefix 'BT', and file type organization enabled

2026-09-17 22:30:02,603 - BatorahScraper - INFO - Batorah scraper initialized: api_urls=2, page_size=100, include_terms=19, exclude_terms=2

2026-09-17 22:30:02,604 - KolHalashonScraper - INFO - Kol Halashon scraper initialized: base_url=https://www2.kolhalashon.com, max_shiurim=50, fetch_all_speakers=True, headless=True, exclude_filters=1, include_filters=0

2026-09-17 22:30:02,606 - AllDafScraper - INFO - All Daf scraper initialized: enabled=True sections=['Daf Yomi Shiurim', 'Supplemental Clips', 'Reference Materials'] skip_terms=0 include_terms=0

2026-09-17 22:30:02,606 - __main__ - INFO - YouTube Intel scraper initialized and enabled

2026-09-17 22:30:02,606 - __main__ - INFO - Running 3 scrapers concurrently...

2026-09-17 22:30:02,606 - GitHubScraper - INFO - Searching GitHub for terms: ['"Hermes Agent"', '"grok bot OR grokbot"', '"muse NOT music NOT museum"', '"prime agent"', 'paperclip']

2026-09-17 22:30:03,032 - httpx - INFO - HTTP Request: GET https://api.github.com/user "HTTP/1.1 200 OK"

2026-09-17 22:30:03,033 - GitHubFeedScraper - INFO - GitHub auth preflight passed for abeperl; token scopes: not reported (fine-grained token)

2026-09-17 22:30:03,763 - httpx - INFO - HTTP Request: GET https://api.github.com/search/repositories?q=%22Hermes+Agent%22+in%3Aname%2Cdescription%2Creadme+stars%3A%3E150&sort=updated&order=desc&per_page=20 "HTTP/1.1 200 OK"

2026-09-17 22:30:03,789 - GitHubScraper - INFO - Found 616 GitHub repositories for term: "Hermes Agent"

2026-09-17 22:30:04,179 - httpx - INFO - HTTP Request: GET https://api.github.com/users/abeperl/received_events?per_page=100&page=1 "HTTP/1.1 200 OK"

2026-09-17 22:30:04,277 - httpx - INFO - HTTP Request: GET https://api.github.com/search/repositories?q=%22grok+bot+OR+grokbot%22+in%3Aname%2Cdescription%2Creadme+stars%3A%3E150&sort=updated&order=desc&per_page=20 "HTTP/1.1 200 OK"

2026-09-17 22:30:04,278 - GitHubScraper - INFO - Found 0 GitHub repositories for term: "grok bot OR grokbot"

2026-09-17 22:30:05,309 - httpx - INFO - HTTP Request: GET https://api.github.com/search/repositories?q=%22muse+NOT+music+NOT+museum%22+in%3Aname%2Cdescription%2Creadme+stars%3A%3E150&sort=updated&order=desc&per_page=20 "HTTP/1.1 200 OK"

2026-09-17 22:30:05,310 - GitHubScraper - INFO - Found 0 GitHub repositories for term: "muse NOT music NOT museum"

2026-09-17 22:30:05,408 - httpx - INFO - HTTP Request: GET https://api.github.com/users/abeperl/received_events?per_page=100&page=2 "HTTP/1.1 200 OK"

2026-09-17 22:30:06,256 - httpx - INFO - HTTP Request: GET https://api.github.com/users/abeperl/received_events?per_page=100&page=3 "HTTP/1.1 200 OK"

2026-09-17 22:30:06,812 - httpx - INFO - HTTP Request: GET https://api.github.com/search/repositories?q=%22prime+agent%22+in%3Aname%2Cdescription%2Creadme+stars%3A%3E150&sort=updated&order=desc&per_page=20 "HTTP/1.1 200 OK"

2026-09-17 22:30:06,815 - GitHubScraper - INFO - Found 41 GitHub repositories for term: "prime agent"

2026-09-17 22:30:08,074 - httpx - INFO - HTTP Request: GET https://api.github.com/search/repositories?q=paperclip+in%3Aname%2Cdescription%2Creadme+stars%3A%3E150&sort=updated&order=desc&per_page=20 "HTTP/1.1 200 OK"

2026-09-17 22:30:08,077 - GitHubScraper - INFO - Found 261 GitHub repositories for term: paperclip

2026-09-17 22:30:08,243 - GitHubScraper - INFO - Total GitHub results found: 0

2026-09-17 22:30:08,933 - httpx - INFO - HTTP Request: POST https://api.github.com/graphql "HTTP/1.1 200 OK"

2026-09-17 22:30:09,316 - httpx - INFO - HTTP Request: GET https://api.github.com/repos/akivayevdayev-debug/Shelah_app/events?per_page=10 "HTTP/1.1 304 Not Modified"

2026-09-17 22:30:09,319 - httpx - INFO - HTTP Request: GET https://api.github.com/repos/yehoshuayaakov/mishnayos-signup/events?per_page=10 "HTTP/1.1 304 Not Modified"

2026-09-17 22:30:09,388 - httpx - INFO - HTTP Request: GET https://api.github.com/repos/Biswaspr1022/KodeshMode/events?per_page=10 "HTTP/1.1 304 Not Modified"

2026-09-17 22:30:09,396 - httpx - INFO - HTTP Request: GET https://api.github.com/repos/efiles-star/ShasTracker/events?per_page=10 "HTTP/1.1 304 Not Modified"

2026-09-17 22:30:09,610 - httpx - INFO - HTTP Request: GET https://api.github.com/repos/nnanach/torah-bot/events?per_page=10 "HTTP/1.1 304 Not Modified"

2026-09-17 22:30:09,699 - httpx - INFO - HTTP Request: GET https://api.github.com/repos/azernjj/talmudai/events?per_page=10 "HTTP/1.1 304 Not Modified"

2026-09-17 22:30:09,754 - httpx - INFO - HTTP Request: GET https://api.github.com/repos/hitchin999/YidCal/events?per_page=10 "HTTP/1.1 200 OK"

2026-09-17 22:30:09,817 - httpx - INFO - HTTP Request: GET https://api.github.com/repos/Oranburg/havruta/events?per_page=10 "HTTP/1.1 304 Not Modified"

2026-09-17 22:30:10,037 - httpx - INFO - HTTP Request: GET https://api.github.com/repos/ethanpil/daily-zmanim/events?per_page=10 "HTTP/1.1 304 Not Modified"

2026-09-17 22:30:10,064 - httpx - INFO - HTTP Request: GET https://api.github.com/repos/rabbishimon613-lang/chabad-tracker/events?per_page=10 "HTTP/1.1 304 Not Modified"

2026-09-17 22:30:10,085 - httpx - INFO - HTTP Request: GET https://api.github.com/repos/adunnmobilenotary-gif/halacha-search/events?per_page=10 "HTTP/1.1 304 Not Modified"

2026-09-17 22:30:10,238 - httpx - INFO - HTTP Request: GET https://api.github.com/repos/Chabadrt/chabad-cms-backend/events?per_page=10 "HTTP/1.1 304 Not Modified"

2026-09-17 22:30:10,400 - httpx - INFO - HTTP Request: GET https://api.github.com/repos/IsraelCIL/SiddurAmIsrael/events?per_page=10 "HTTP/1.1 304 Not Modified"

2026-09-17 22:30:10,436 - httpx - INFO - HTTP Request: GET https://api.github.com/repos/ronrefael/kav-siddur-site/events?per_page=10 "HTTP/1.1 304 Not Modified"

2026-09-17 22:30:10,462 - httpx - INFO - HTTP Request: GET https://api.github.com/repos/ElikoGrins/Halacha-bot/events?per_page=10 "HTTP/1.1 304 Not Modified"

2026-09-17 22:30:10,621 - httpx - INFO - HTTP Request: GET https://api.github.com/repos/kdroidFilter/Zayit/events?per_page=10 "HTTP/1.1 304 Not Modified"

2026-09-17 22:30:10,819 - httpx - INFO - HTTP Request: GET https://api.github.com/repos/EzraBrand/talmud-nlp-indexer/events?per_page=10 "HTTP/1.1 304 Not Modified"

2026-09-17 22:30:10,911 - httpx - INFO - HTTP Request: GET https://api.github.com/repos/brainheart/tanakh-contabulate/events?per_page=10 "HTTP/1.1 304 Not Modified"

2026-09-17 22:30:10,941 - httpx - INFO - HTTP Request: GET https://api.github.com/repos/sikmenedowai-a11y/savetube-stream-kit/events?per_page=10 "HTTP/1.1 200 OK"

2026-09-17 22:30:11,007 - httpx - INFO - HTTP Request: GET https://api.github.com/repos/Hbleiberg/Hebrew_Blender/events?per_page=10 "HTTP/1.1 304 Not Modified"

2026-09-17 22:30:11,231 - httpx - INFO - HTTP Request: GET https://api.github.com/repos/Esoterics611/maayan/events?per_page=10 "HTTP/1.1 304 Not Modified"

2026-09-17 22:30:11,314 - httpx - INFO - HTTP Request: GET https://api.github.com/repos/stenagmois-byte/SimHebrew-Concordance/events?per_page=10 "HTTP/1.1 304 Not Modified"

2026-09-17 22:30:11,353 - httpx - INFO - HTTP Request: GET https://api.github.com/repos/naorbrown/daf-yomi-history-bot/events?per_page=10 "HTTP/1.1 304 Not Modified"

2026-09-17 22:30:11,375 - httpx - INFO - HTTP Request: GET https://api.github.com/repos/wuisabel-gif/Aleph-Bet/events?per_page=10 "HTTP/1.1 304 Not Modified"

2026-09-17 22:30:11,597 - httpx - INFO - HTTP Request: GET https://api.github.com/repos/jepick/talmud-map/events?per_page=10 "HTTP/1.1 304 Not Modified"

2026-09-17 22:30:11,917 - httpx - INFO - HTTP Request: GET https://api.github.com/repos/shloimie-beep/bnei-neviim-academy/events?per_page=10 "HTTP/1.1 304 Not Modified"

2026-09-17 22:30:11,953 - httpx - INFO - HTTP Request: GET https://api.github.com/repos/paperlesspaper/integrations/events?per_page=10 "HTTP/1.1 304 Not Modified"

2026-09-17 22:30:12,003 - httpx - INFO - HTTP Request: GET https://api.github.com/repos/Asifah-Analytics-C-A-V-E/asifah-analytics/events?per_page=10 "HTTP/1.1 304 Not Modified"

2026-09-17 22:30:12,037 - httpx - INFO - HTTP Request: GET https://api.github.com/repos/complytime/complytime-collector-components/events?per_page=10 "HTTP/1.1 304 Not Modified"

2026-09-17 22:30:12,275 - httpx - INFO - HTTP Request: GET https://api.github.com/repos/dniasoff/learning-tracker/events?per_page=10 "HTTP/1.1 304 Not Modified"

2026-09-17 22:30:12,290 - httpx - INFO - HTTP Request: GET https://api.github.com/repos/DavidPatlas-AI/tehillim-russian-site/events?per_page=10 "HTTP/1.1 304 Not Modified"

2026-09-17 22:30:12,509 - httpx - INFO - HTTP Request: GET https://api.github.com/repos/dmo18/MySugya/events?per_page=10 "HTTP/1.1 200 OK"

2026-09-17 22:30:12,573 - httpx - INFO - HTTP Request: GET https://api.github.com/repos/RonTurrentine/KJV-Strongs-EBook/events?per_page=10 "HTTP/1.1 304 Not Modified"

2026-09-17 22:30:12,585 - httpx - INFO - HTTP Request: GET https://api.github.com/repos/yossi-weinberger/ten10/events?per_page=10 "HTTP/1.1 304 Not Modified"

2026-09-17 22:30:12,611 - httpx - INFO - HTTP Request: GET https://api.github.com/repos/hytric/NLP_term_project/events?per_page=10 "HTTP/1.1 304 Not Modified"

2026-09-17 22:30:12,793 - httpx - INFO - HTTP Request: GET https://api.github.com/repos/zarbinco/zarbin-seo/events?per_page=10 "HTTP/1.1 304 Not Modified"

2026-09-17 22:30:13,023 - httpx - INFO - HTTP Request: GET https://api.github.com/repos/AUX440/liquidtext-ultimate-patch-kit/events?per_page=10 "HTTP/1.1 200 OK"

2026-09-17 22:30:13,075 - httpx - INFO - HTTP Request: GET https://api.github.com/repos/duvarakesh06/FAB-Subtitler-12.1-Offline-Release/events?per_page=10 "HTTP/1.1 304 Not Modified"

2026-09-17 22:30:13,121 - httpx - INFO - HTTP Request: GET https://api.github.com/repos/v97969181-commits/master-pdf-editor-legacy-release/events?per_page=10 "HTTP/1.1 200 OK"

2026-09-17 22:30:13,343 - httpx - INFO - HTTP Request: GET https://api.github.com/repos/elmadanimohammed78-del/daily-flow-landing/events?per_page=10 "HTTP/1.1 304 Not Modified"

2026-09-17 22:30:13,440 - httpx - INFO - HTTP Request: GET https://api.github.com/repos/rafistrauss/commons-announcements/events?per_page=10 "HTTP/1.1 304 Not Modified"

2026-09-17 22:30:13,564 - httpx - INFO - HTTP Request: GET https://api.github.com/repos/Roger0129/bitwarden-2024-8-0-patchless-install/events?per_page=10 "HTTP/1.1 304 Not Modified"

2026-09-17 22:30:13,603 - httpx - INFO - HTTP Request: GET https://api.github.com/repos/Aljademaratas/readiris-product-code-generator/events?per_page=10 "HTTP/1.1 200 OK"

2026-09-17 22:30:13,712 - httpx - INFO - HTTP Request: GET https://api.github.com/repos/apusaha0011/MenuSideKick/events?per_page=10 "HTTP/1.1 304 Not Modified"

2026-09-17 22:30:13,830 - httpx - INFO - HTTP Request: GET https://api.github.com/repos/chfle/chfle/events?per_page=10 "HTTP/1.1 200 OK"

2026-09-17 22:30:14,021 - httpx - INFO - HTTP Request: GET https://api.github.com/repos/benemanuel/status/events?per_page=10 "HTTP/1.1 304 Not Modified"

2026-09-17 22:30:14,069 - httpx - INFO - HTTP Request: GET https://api.github.com/repos/semiagenticRob/100Reps/events?per_page=10 "HTTP/1.1 304 Not Modified"

2026-09-17 22:30:14,133 - httpx - INFO - HTTP Request: GET https://api.github.com/repos/revanite-io/pvtr-gcp-cloud-storage/events?per_page=10 "HTTP/1.1 200 OK"

2026-09-17 22:30:14,154 - httpx - INFO - HTTP Request: GET https://api.github.com/repos/Jossifresben/translation-alignment/events?per_page=10 "HTTP/1.1 304 Not Modified"

2026-09-17 22:30:14,310 - httpx - INFO - HTTP Request: GET https://api.github.com/repos/egkristi/ToponymiaEuropaea/events?per_page=10 "HTTP/1.1 304 Not Modified"

2026-09-17 22:30:14,518 - httpx - INFO - HTTP Request: GET https://api.github.com/repos/hebcal/hebcal/events?per_page=10 "HTTP/1.1 304 Not Modified"

2026-09-17 22:30:14,528 - httpx - INFO - HTTP Request: GET https://api.github.com/repos/Israel-Kli/shabbat-timetable/events?per_page=10 "HTTP/1.1 304 Not Modified"

2026-09-17 22:30:14,536 - httpx - INFO - HTTP Request: GET https://api.github.com/repos/KosherJava/zmanim/events?per_page=10 "HTTP/1.1 304 Not Modified"

2026-09-17 22:30:14,746 - httpx - INFO - HTTP Request: GET https://api.github.com/repos/woodpeckerbros/watch-reminder/events?per_page=10 "HTTP/1.1 200 OK"

2026-09-17 22:30:14,807 - httpx - INFO - HTTP Request: GET https://api.github.com/repos/hebcal/learning/events?per_page=10 "HTTP/1.1 304 Not Modified"

2026-09-17 22:30:14,890 - httpx - INFO - HTTP Request: GET https://api.github.com/repos/svenalot-ui/sidur-native/events?per_page=10 "HTTP/1.1 304 Not Modified"

2026-09-17 22:30:15,024 - httpx - INFO - HTTP Request: GET https://api.github.com/repos/hebcal/hebcal-go/events?per_page=10 "HTTP/1.1 304 Not Modified"

2026-09-17 22:30:15,171 - YouTubeIntelScraper - INFO - YouTube Intel found 22 candidates and 2 new videos

2026-09-17 22:30:15,362 - httpx - INFO - HTTP Request: GET https://api.github.com/repos/Sefaria/ai-chatbot/events?per_page=10 "HTTP/1.1 200 OK"

2026-09-17 22:30:15,369 - httpx - INFO - HTTP Request: GET https://api.github.com/repos/levidao-eth/Torah/events?per_page=10 "HTTP/1.1 304 Not Modified"

2026-09-17 22:30:15,410 - httpx - INFO - HTTP Request: GET https://api.github.com/repos/joelcw/intsayt/events?per_page=10 "HTTP/1.1 304 Not Modified"

2026-09-17 22:30:15,757 - httpx - INFO - HTTP Request: GET https://api.github.com/repos/zakdev26/sefaria-links/events?per_page=10 "HTTP/1.1 304 Not Modified"

2026-09-17 22:30:15,785 - httpx - INFO - HTTP Request: GET https://api.github.com/repos/ainihonuragawa-lang/TalmudFinance/events?per_page=10 "HTTP/1.1 304 Not Modified"

2026-09-17 22:30:15,791 - httpx - INFO - HTTP Request: GET https://api.github.com/repos/complytime/complytime-policies/events?per_page=10 "HTTP/1.1 304 Not Modified"

2026-09-17 22:30:15,943 - httpx - INFO - HTTP Request: GET https://api.github.com/repos/AdamYairLee/torah-korean-translation/events?per_page=10 "HTTP/1.1 304 Not Modified"

2026-09-17 22:30:16,054 - httpx - INFO - HTTP Request: GET https://api.github.com/repos/Yiddish-Dialect-Dictionary/-brukhe-machine/events?per_page=10 "HTTP/1.1 304 Not Modified"

2026-09-17 22:30:16,149 - httpx - INFO - HTTP Request: GET https://api.github.com/repos/SnirRadomsky/daf-yomi/events?per_page=10 "HTTP/1.1 304 Not Modified"

2026-09-17 22:30:16,189 - httpx - INFO - HTTP Request: GET https://api.github.com/repos/vaisisrael/gemara-benichuta/events?per_page=10 "HTTP/1.1 304 Not Modified"

2026-09-17 22:30:16,271 - httpx - INFO - HTTP Request: GET https://api.github.com/repos/elconde/kosher-races/events?per_page=10 "HTTP/1.1 304 Not Modified"

2026-09-17 22:30:16,441 - httpx - INFO - HTTP Request: GET https://api.github.com/repos/jacopovirga/kindle-zmanim/events?per_page=10 "HTTP/1.1 304 Not Modified"

2026-09-17 22:30:16,557 - httpx - INFO - HTTP Request: GET https://api.github.com/repos/robmoffat/FOSS-Contribution-Gemara/events?per_page=10 "HTTP/1.1 304 Not Modified"

2026-09-17 22:30:16,684 - httpx - INFO - HTTP Request: GET https://api.github.com/repos/aharonium/opensiddur.org/events?per_page=10 "HTTP/1.1 304 Not Modified"

2026-09-17 22:30:16,716 - httpx - INFO - HTTP Request: GET https://api.github.com/repos/Selecr-Surplus/minyan-finder/events?per_page=10 "HTTP/1.1 304 Not Modified"

2026-09-17 22:30:16,891 - httpx - INFO - HTTP Request: GET https://api.github.com/repos/naorbrown/peninei-halacha-yomi-bot/events?per_page=10 "HTTP/1.1 304 Not Modified"

2026-09-17 22:30:16,898 - httpx - INFO - HTTP Request: GET https://api.github.com/repos/theOnionsAreWatching/Shtigletz/events?per_page=10 "HTTP/1.1 304 Not Modified"

2026-09-17 22:30:17,010 - httpx - INFO - HTTP Request: GET https://api.github.com/repos/vistaTeam/satmar/events?per_page=10 "HTTP/1.1 304 Not Modified"

2026-09-17 22:30:17,022 - httpx - INFO - HTTP Request: GET https://api.github.com/repos/mig2/mishnah-style/events?per_page=10 "HTTP/1.1 304 Not Modified"

2026-09-17 22:30:17,283 - httpx - INFO - HTTP Request: GET https://api.github.com/repos/skilead770/stoliskikosher/events?per_page=10 "HTTP/1.1 304 Not Modified"

2026-09-17 22:30:17,327 - httpx - INFO - HTTP Request: GET https://api.github.com/repos/a0556770172-png/koshermat/events?per_page=10 "HTTP/1.1 304 Not Modified"

2026-09-17 22:30:17,378 - httpx - INFO - HTTP Request: GET https://api.github.com/repos/mak-sandboxuser/chabad-react-and-supabase/events?per_page=10 "HTTP/1.1 304 Not Modified"

2026-09-17 22:30:17,427 - httpx - INFO - HTTP Request: GET https://api.github.com/repos/yeled-tov/filtertube-android/events?per_page=10 "HTTP/1.1 304 Not Modified"

2026-09-17 22:30:17,650 - httpx - INFO - HTTP Request: GET https://api.github.com/repos/bnq9705-creator/siddur/events?per_page=10 "HTTP/1.1 304 Not Modified"

2026-09-17 22:30:17,662 - httpx - INFO - HTTP Request: GET https://api.github.com/repos/Kosher-T/fannexus/events?per_page=10 "HTTP/1.1 304 Not Modified"

2026-09-17 22:30:17,781 - httpx - INFO - HTTP Request: GET https://api.github.com/repos/yosefmalul52-lang/costa-rica-kosher-retreat/events?per_page=10 "HTTP/1.1 304 Not Modified"

2026-09-17 22:30:17,921 - httpx - INFO - HTTP Request: GET https://api.github.com/repos/RavBogard/shirei-tfilah/events?per_page=10 "HTTP/1.1 304 Not Modified"

2026-09-17 22:30:17,995 - httpx - INFO - HTTP Request: GET https://api.github.com/repos/ReuvenB-S/kosher-browser-data/events?per_page=10 "HTTP/1.1 304 Not Modified"

2026-09-17 22:30:18,036 - httpx - INFO - HTTP Request: GET https://api.github.com/repos/shliachflow/chabad-southern-delaware-camp/events?per_page=10 "HTTP/1.1 304 Not Modified"

2026-09-17 22:30:18,149 - httpx - INFO - HTTP Request: GET https://api.github.com/repos/AviMish10/tanach-map/events?per_page=10 "HTTP/1.1 304 Not Modified"

2026-09-17 22:30:18,218 - httpx - INFO - HTTP Request: GET https://api.github.com/repos/TanachatGKRB/TanachatGKRB.github.io/events?per_page=10 "HTTP/1.1 304 Not Modified"

2026-09-17 22:30:18,399 - httpx - INFO - HTTP Request: GET https://api.github.com/repos/TamamTamar/chabad-yafo/events?per_page=10 "HTTP/1.1 304 Not Modified"

2026-09-17 22:30:18,433 - httpx - INFO - HTTP Request: GET https://api.github.com/repos/DeepSynchronicity/chabadofhurlingham/events?per_page=10 "HTTP/1.1 304 Not Modified"

2026-09-17 22:30:18,482 - httpx - INFO - HTTP Request: GET https://api.github.com/repos/TG-CONSULTANT/tanachok-stock-FG1/events?per_page=10 "HTTP/1.1 304 Not Modified"

2026-09-17 22:30:18,492 - httpx - INFO - HTTP Request: GET https://api.github.com/repos/semiagenticRob/digital-siddur/events?per_page=10 "HTTP/1.1 304 Not Modified"

2026-09-17 22:30:18,731 - httpx - INFO - HTTP Request: GET https://api.github.com/repos/orelzion/daily-tanach/events?per_page=10 "HTTP/1.1 304 Not Modified"

2026-09-17 22:30:18,766 - httpx - INFO - HTTP Request: GET https://api.github.com/repos/TG-CONSULTANT/tanachok-stock-mat/events?per_page=10 "HTTP/1.1 304 Not Modified"

2026-09-17 22:30:18,809 - httpx - INFO - HTTP Request: GET https://api.github.com/repos/Frish-Gebakn/kol-satmar-website/events?per_page=10 "HTTP/1.1 304 Not Modified"

2026-09-17 22:30:18,831 - httpx - INFO - HTTP Request: GET https://api.github.com/repos/satmarthubby-blip/Satmarthubby/events?per_page=10 "HTTP/1.1 304 Not Modified"

2026-09-17 22:30:19,095 - httpx - INFO - HTTP Request: GET https://api.github.com/repos/akshatasatmar/WeatherServiceApi/events?per_page=10 "HTTP/1.1 304 Not Modified"

2026-09-17 22:30:19,119 - httpx - INFO - HTTP Request: GET https://api.github.com/repos/elchunenfreund/satmar-matzos-app/events?per_page=10 "HTTP/1.1 304 Not Modified"

2026-09-17 22:30:19,148 - httpx - INFO - HTTP Request: GET https://api.github.com/repos/satmaribianca05-eng/AFSD-Satmari-Bianca/events?per_page=10 "HTTP/1.1 304 Not Modified"

2026-09-17 22:30:19,240 - httpx - INFO - HTTP Request: GET https://api.github.com/repos/Sefaria/Sefaria-Export/events?per_page=10 "HTTP/1.1 304 Not Modified"

2026-09-17 22:30:19,307 - GitHubFeedScraper - INFO - GitHub feed healthy: login=abeperl fetched=300 account_new=0 list_repos=100 list_new=1 private_new=0 total_new=1

2026-09-17 22:30:19,309 - __main__ - INFO - Scraper github completed: 0 results

2026-09-17 22:30:19,309 - __main__ - INFO - Scraper github_feed completed: 1 results

2026-09-17 22:30:19,309 - __main__ - INFO - Scraper youtube_intel completed: 2 results

2026-09-17 22:30:19,916 - social_media_monitor.email.sender - INFO - Email sent successfully at 2026-09-17 22:30:19.916513

2026-09-17 22:30:19,916 - __main__ - INFO - Email sent successfully

2026-09-17 22:30:19,928 - __main__ - INFO - GitHub feed delivery acknowledged: 1 stored, 0 failed

2026-09-17 22:30:19,929 - __main__ - INFO - Cache stats: 0 items in memory, 0 files on disk

2026-09-17 22:30:19,929 - __main__ - INFO - Monitor complete. Found 3 total results: 1 new github feed results, 2 new youtube intel results.

2026-09-17 22:30:19,929 - __main__ - INFO - === Social Media Monitor v2.0 Complete ===

2026-09-17 22:30:19,929 - social_media_monitor.database.connection - INFO - Closing all database connections

2026-09-17 22:30:19,957 - social_media_monitor.database.connection - INFO - Closed 5 database connections

2026-09-17 22:30:19,957 - __main__ - INFO - Closed all database connections

2026-09-17 22:30:19,960 - social_media_monitor.database.connection - INFO - Closing all database connections

2026-09-17 22:30:19,960 - social_media_monitor.database.connection - INFO - Closed 0 database connections

2026-09-17 23:00:01,983 - social_media_monitor.utils.logging_config - INFO - Posts log file: /mnt/data/openclaw-shared/home/shared/Social_Media_Monitor/logs/social_media_monitor_posts_2026-09-17.log

2026-09-17 23:00:01,984 - social_media_monitor.utils.logging_config - INFO - Downloads log file: /mnt/data/openclaw-shared/home/shared/Social_Media_Monitor/logs/social_media_monitor_downloads_2026-09-17.log

2026-09-17 23:00:01,984 - social_media_monitor.utils.logging_config - INFO - Logging configured with level INFO, log file: /mnt/data/openclaw-shared/home/shared/Social_Media_Monitor/logs/social_media_monitor_2026-09-17.log

2026-09-17 23:00:01,984 - __main__ - INFO - === Social Media Monitor v2.0 (Async) Started ===

2026-09-17 23:00:02,013 - social_media_monitor.config.settings - INFO - Configuration loaded from /home/openclaw/Social_Media_Monitor/config.yaml

2026-09-17 23:00:02,013 - __main__ - INFO - Configuration loaded and validated successfully

2026-09-17 23:00:02,015 - social_media_monitor.database.connection - INFO - Database connection pool initialized with 5 connections

2026-09-17 23:00:02,016 - social_media_monitor.database.connection - INFO - add_speaker_filters column already exists

2026-09-17 23:00:02,036 - social_media_monitor.database.connection - INFO - Database setup completed successfully

2026-09-17 23:00:02,036 - __main__ - INFO - Database initialized at: /mnt/data/openclaw-shared/home/shared/Social_Media_Monitor/social_media_monitor.db

2026-09-17 23:00:02,040 - social_media_monitor.scrapers.reddit_scraper - INFO - Reddit API initialized successfully (UA='SocialMediaMonitor/1.0')

2026-09-17 23:00:02,041 - RSSFeedScraper - INFO - RSS scraper initialized: feeds=1, html_urls=0 (0 with custom selectors), include_terms=0, exclude_terms=0

2026-09-17 23:00:02,041 - __main__ - INFO - RSS scraper initialized and enabled

2026-09-17 23:00:02,045 - Yiddish24Scraper - INFO - File downloader initialized with 55 terms, prefix 'Y2', and file type organization enabled

2026-09-17 23:00:02,045 - Yiddish24Scraper - INFO - Yiddish24 scraper initialized: data_id='latest', page_limit=100, total_pages=10, debug_html=True, include_terms=55, exclude_terms=0

2026-09-17 23:00:02,045 - __main__ - INFO - Yiddish24 scraper initialized and enabled

2026-09-17 23:00:02,050 - AhBlickLiveScraper - INFO - AhBlickLive scraper initialized: urls=1, max_items=100, include_terms=49, exclude_terms=0

2026-09-17 23:00:02,057 - TorahAnytimeScraper - INFO - File downloader initialized with 53 terms, prefix 'TA', and file type organization enabled

2026-09-17 23:00:02,058 - TorahAnytimeScraper - INFO - TorahAnytime scraper initialized: limit=400, offset=0, project_id=1, include_terms=53, exclude_terms=13

2026-09-17 23:00:02,061 - BatorahScraper - INFO - File downloader initialized with 19 terms, prefix 'BT', and file type organization enabled

2026-09-17 23:00:02,061 - BatorahScraper - INFO - Batorah scraper initialized: api_urls=2, page_size=100, include_terms=19, exclude_terms=2

2026-09-17 23:00:02,062 - KolHalashonScraper - INFO - Kol Halashon scraper initialized: base_url=https://www2.kolhalashon.com, max_shiurim=50, fetch_all_speakers=True, headless=True, exclude_filters=1, include_filters=0

2026-09-17 23:00:02,065 - AllDafScraper - INFO - All Daf scraper initialized: enabled=True sections=['Daf Yomi Shiurim', 'Supplemental Clips', 'Reference Materials'] skip_terms=0 include_terms=0

2026-09-17 23:00:02,066 - __main__ - INFO - Ivelt scraper initialized and enabled

2026-09-17 23:00:02,066 - __main__ - INFO - Running 3 scrapers concurrently...

2026-09-17 23:00:02,066 - Yiddish24Scraper - INFO - Starting Yiddish24 scraping (max 10 pages, limit 100 items)

2026-09-17 23:00:02,181 - Yiddish24Scraper - INFO - Fetching Yiddish24 page 1 with data_id='latest', page_limit=100

2026-09-17 23:00:02,185 - IveltScraper - INFO - Loaded 181 include, 4 exclude filters from DB topic_filters

2026-09-17 23:00:02,185 - IveltScraper - INFO - Fetching ivelt active topics from https://www.ivelt.com/forum/search.php?search_id=active_topics...

2026-09-17 23:00:02,186 - RSSFeedScraper - INFO - Fetching RSS feed: https://www.theyeshivaworld.com/feed

2026-09-17 23:00:02,586 - RSSFeedScraper - INFO - Total RSS/HTML results: 0

2026-09-17 23:00:03,023 - httpx - INFO - HTTP Request: POST https://www.yiddish24.com/ajax/cat_pagination.php "HTTP/1.1 200 OK"

2026-09-17 23:00:03,241 - Yiddish24Scraper - INFO - Received response with 512895 characters

2026-09-17 23:00:03,243 - Yiddish24Scraper - INFO - Extracted HTML from JSON result field (315451 chars)

2026-09-17 23:00:03,245 - Yiddish24Scraper - INFO - Saved HTML debug file to: /mnt/data/openclaw-shared/home/shared/Social_Media_Monitor/debug/yiddish24_page1_response.html

2026-09-17 23:00:03,346 - Yiddish24Scraper - INFO - Found 100 potential article elements on page 1

2026-09-17 23:00:03,409 - Yiddish24Scraper - INFO - Page 1 extraction: 100 successful, 0 failed out of 100 elements

2026-09-17 23:00:03,409 - Yiddish24Scraper - INFO - Parsed 100 articles from HTML on page 1

2026-09-17 23:00:04,164 - Yiddish24Scraper - INFO - Page 1 summary: 9 new out of 100 total articles

2026-09-17 23:00:04,164 - Yiddish24Scraper - INFO - Page 1: 9/100 new items

2026-09-17 23:00:04,165 - Yiddish24Scraper - INFO - Low new item ratio (9.0%), stopping pagination

2026-09-17 23:00:04,165 - Yiddish24Scraper - INFO - Total Yiddish24 results found: 9

2026-09-17 23:00:14,415 - IveltScraper - INFO - Found 50 active topics on ivelt

2026-09-17 23:00:21,341 - IveltScraper - INFO - Filtering post containing exclude term: '*ביטע מאכט זיכער בלויז צו באריכטן קראנטע נייעס*'

2026-09-17 23:00:26,013 - IveltScraper - INFO - ivelt scrape complete. Checked 3 topics.

2026-09-17 23:00:26,013 - IveltScraper - INFO - All forums scrape complete. Found 13 new posts.

2026-09-17 23:00:26,014 - __main__ - INFO - Scraper rss completed: 0 results

2026-09-17 23:00:26,014 - __main__ - INFO - Scraper yiddish24 completed: 9 results

2026-09-17 23:00:26,014 - __main__ - INFO - Scraper ivelt completed: 13 results

2026-09-17 23:00:26,835 - social_media_monitor.email.sender - INFO - Email sent successfully at 2026-09-17 23:00:26.835282

2026-09-17 23:00:26,835 - __main__ - INFO - Email sent successfully

2026-09-17 23:00:26,835 - __main__ - INFO - Cache stats: 0 items in memory, 0 files on disk

2026-09-17 23:00:26,836 - __main__ - INFO - Monitor complete. Found 22 total results: 9 new yiddish24 results, 13 new ivelt results.

2026-09-17 23:00:26,836 - __main__ - INFO - === Social Media Monitor v2.0 Complete ===

2026-09-17 23:00:26,836 - social_media_monitor.database.connection - INFO - Closing all database connections

2026-09-17 23:00:26,909 - social_media_monitor.database.connection - INFO - Closed 5 database connections

2026-09-17 23:00:26,909 - __main__ - INFO - Closed all database connections

2026-09-17 23:00:26,911 - social_media_monitor.database.connection - INFO - Closing all database connections

2026-09-17 23:00:26,911 - social_media_monitor.database.connection - INFO - Closed 0 database connections