โ† Logs

social_media_monitor_2026-09-28.log

2026-09-28 20:30:01,955 - __main__ - INFO - === Social Media Monitor v2.0 (Async) Started ===
2026-09-28 20:30:01,983 - social_media_monitor.config.settings - INFO - Configuration loaded from /home/openclaw/Social_Media_Monitor/config.yaml
2026-09-28 20:30:01,983 - __main__ - INFO - Configuration loaded and validated successfully
2026-09-28 20:30:01,985 - social_media_monitor.database.connection - INFO - Database connection pool initialized with 5 connections
2026-09-28 20:30:01,986 - social_media_monitor.database.connection - INFO - add_speaker_filters column already exists
2026-09-28 20:30:01,998 - social_media_monitor.database.connection - INFO - Database setup completed successfully
2026-09-28 20:30:01,998 - __main__ - INFO - Database initialized at: /mnt/data/openclaw-shared/home/shared/Social_Media_Monitor/social_media_monitor.db
2026-09-28 20:30:02,001 - social_media_monitor.scrapers.reddit_scraper - INFO - Reddit API initialized successfully (UA='SocialMediaMonitor/1.0')
2026-09-28 20:30:02,001 - __main__ - INFO - GitHub scraper initialized and enabled
2026-09-28 20:30:02,002 - __main__ - INFO - GitHub feed scraper initialized and enabled
2026-09-28 20:30:02,002 - RSSFeedScraper - INFO - RSS scraper initialized: feeds=1, html_urls=0 (0 with custom selectors), include_terms=0, exclude_terms=0
2026-09-28 20:30:02,006 - Yiddish24Scraper - INFO - File downloader initialized with 55 terms, prefix 'Y2', and file type organization enabled
2026-09-28 20:30:02,006 - Yiddish24Scraper - INFO - Yiddish24 scraper initialized: data_id='latest', page_limit=100, total_pages=10, debug_html=True, include_terms=55, exclude_terms=0
2026-09-28 20:30:02,009 - AhBlickLiveScraper - INFO - AhBlickLive scraper initialized: urls=1, max_items=100, include_terms=49, exclude_terms=0
2026-09-28 20:30:02,014 - TorahAnytimeScraper - INFO - File downloader initialized with 53 terms, prefix 'TA', and file type organization enabled
2026-09-28 20:30:02,014 - TorahAnytimeScraper - INFO - TorahAnytime scraper initialized: limit=400, offset=0, project_id=1, include_terms=53, exclude_terms=13
2026-09-28 20:30:02,016 - BatorahScraper - INFO - File downloader initialized with 19 terms, prefix 'BT', and file type organization enabled
2026-09-28 20:30:02,016 - BatorahScraper - INFO - Batorah scraper initialized: api_urls=2, page_size=100, include_terms=19, exclude_terms=2
2026-09-28 20:30:02,016 - KolHalashonScraper - INFO - Kol Halashon scraper initialized: base_url=https://www2.kolhalashon.com, max_shiurim=50, fetch_all_speakers=True, headless=True, exclude_filters=1, include_filters=0
2026-09-28 20:30:02,018 - AllDafScraper - INFO - All Daf scraper initialized: enabled=True sections=['Daf Yomi Shiurim', 'Supplemental Clips', 'Reference Materials'] skip_terms=0 include_terms=0
2026-09-28 20:30:02,018 - __main__ - INFO - YouTube Intel scraper initialized and enabled
2026-09-28 20:30:02,018 - __main__ - INFO - Running 3 scrapers concurrently...
2026-09-28 20:30:02,019 - GitHubScraper - INFO - Searching GitHub for terms: ['"Hermes Agent"', '"grok bot OR grokbot"', '"muse NOT music NOT museum"', '"prime agent"', 'paperclip']
2026-09-28 20:30:02,424 - httpx - INFO - HTTP Request: GET https://api.github.com/user "HTTP/1.1 200 OK"
2026-09-28 20:30:02,425 - GitHubFeedScraper - INFO - GitHub auth preflight passed for abeperl; token scopes: not reported (fine-grained token)
2026-09-28 20:30:03,129 - httpx - INFO - HTTP Request: GET https://api.github.com/search/repositories?q=%22Hermes+Agent%22+in%3Aname%2Cdescription%2Creadme+stars%3A%3E150&sort=updated&order=desc&per_page=20 "HTTP/1.1 200 OK"
2026-09-28 20:30:03,154 - GitHubScraper - INFO - Found 649 GitHub repositories for term: "Hermes Agent"
2026-09-28 20:30:03,687 - httpx - INFO - HTTP Request: GET https://api.github.com/users/abeperl/received_events?per_page=100&page=1 "HTTP/1.1 200 OK"
2026-09-28 20:30:03,813 - httpx - INFO - HTTP Request: GET https://api.github.com/search/repositories?q=%22grok+bot+OR+grokbot%22+in%3Aname%2Cdescription%2Creadme+stars%3A%3E150&sort=updated&order=desc&per_page=20 "HTTP/1.1 200 OK"
2026-09-28 20:30:03,814 - GitHubScraper - INFO - Found 0 GitHub repositories for term: "grok bot OR grokbot"
2026-09-28 20:30:04,743 - httpx - INFO - HTTP Request: GET https://api.github.com/search/repositories?q=%22muse+NOT+music+NOT+museum%22+in%3Aname%2Cdescription%2Creadme+stars%3A%3E150&sort=updated&order=desc&per_page=20 "HTTP/1.1 200 OK"
2026-09-28 20:30:04,744 - GitHubScraper - INFO - Found 0 GitHub repositories for term: "muse NOT music NOT museum"
2026-09-28 20:30:05,071 - httpx - INFO - HTTP Request: GET https://api.github.com/users/abeperl/received_events?per_page=100&page=2 "HTTP/1.1 200 OK"
2026-09-28 20:30:06,302 - httpx - INFO - HTTP Request: GET https://api.github.com/search/repositories?q=%22prime+agent%22+in%3Aname%2Cdescription%2Creadme+stars%3A%3E150&sort=updated&order=desc&per_page=20 "HTTP/1.1 200 OK"
2026-09-28 20:30:06,306 - GitHubScraper - INFO - Found 44 GitHub repositories for term: "prime agent"
2026-09-28 20:30:06,467 - httpx - INFO - HTTP Request: GET https://api.github.com/users/abeperl/received_events?per_page=100&page=3 "HTTP/1.1 304 Not Modified"
2026-09-28 20:30:06,468 - GitHubFeedScraper - INFO - GitHub account feed page 3 unchanged (ETag)
2026-09-28 20:30:07,596 - httpx - INFO - HTTP Request: GET https://api.github.com/search/repositories?q=paperclip+in%3Aname%2Cdescription%2Creadme+stars%3A%3E150&sort=updated&order=desc&per_page=20 "HTTP/1.1 200 OK"
2026-09-28 20:30:07,693 - GitHubScraper - INFO - Found 266 GitHub repositories for term: paperclip
2026-09-28 20:30:08,080 - GitHubScraper - INFO - Total GitHub results found: 0
2026-09-28 20:30:10,151 - httpx - INFO - HTTP Request: POST https://api.github.com/graphql "HTTP/1.1 200 OK"
2026-09-28 20:30:10,501 - httpx - INFO - HTTP Request: GET https://api.github.com/repos/akivayevdayev-debug/Shelah_app/events?per_page=10 "HTTP/1.1 304 Not Modified"
2026-09-28 20:30:10,519 - httpx - INFO - HTTP Request: GET https://api.github.com/repos/efiles-star/ShasTracker/events?per_page=10 "HTTP/1.1 304 Not Modified"
2026-09-28 20:30:10,519 - httpx - INFO - HTTP Request: GET https://api.github.com/repos/yehoshuayaakov/mishnayos-signup/events?per_page=10 "HTTP/1.1 304 Not Modified"
2026-09-28 20:30:10,707 - httpx - INFO - HTTP Request: GET https://api.github.com/repos/Biswaspr1022/KodeshMode/events?per_page=10 "HTTP/1.1 304 Not Modified"
2026-09-28 20:30:10,919 - httpx - INFO - HTTP Request: GET https://api.github.com/repos/Oranburg/havruta/events?per_page=10 "HTTP/1.1 304 Not Modified"
2026-09-28 20:30:10,943 - httpx - INFO - HTTP Request: GET https://api.github.com/repos/nnanach/torah-bot/events?per_page=10 "HTTP/1.1 304 Not Modified"
2026-09-28 20:30:10,958 - httpx - INFO - HTTP Request: GET https://api.github.com/repos/hitchin999/YidCal/events?per_page=10 "HTTP/1.1 304 Not Modified"
2026-09-28 20:30:11,114 - httpx - INFO - HTTP Request: GET https://api.github.com/repos/azernjj/talmudai/events?per_page=10 "HTTP/1.1 304 Not Modified"
2026-09-28 20:30:11,331 - httpx - INFO - HTTP Request: GET https://api.github.com/repos/rabbishimon613-lang/chabad-tracker/events?per_page=10 "HTTP/1.1 304 Not Modified"
2026-09-28 20:30:11,358 - httpx - INFO - HTTP Request: GET https://api.github.com/repos/ethanpil/daily-zmanim/events?per_page=10 "HTTP/1.1 304 Not Modified"
2026-09-28 20:30:11,366 - httpx - INFO - HTTP Request: GET https://api.github.com/repos/adunnmobilenotary-gif/halacha-search/events?per_page=10 "HTTP/1.1 304 Not Modified"
2026-09-28 20:30:11,536 - httpx - INFO - HTTP Request: GET https://api.github.com/repos/Chabadrt/chabad-cms-backend/events?per_page=10 "HTTP/1.1 304 Not Modified"
2026-09-28 20:30:11,687 - httpx - INFO - HTTP Request: GET https://api.github.com/repos/ElikoGrins/Halacha-bot/events?per_page=10 "HTTP/1.1 304 Not Modified"
2026-09-28 20:30:11,713 - httpx - INFO - HTTP Request: GET https://api.github.com/repos/IsraelCIL/SiddurAmIsrael/events?per_page=10 "HTTP/1.1 304 Not Modified"
2026-09-28 20:30:11,735 - httpx - INFO - HTTP Request: GET https://api.github.com/repos/ronrefael/kav-siddur-site/events?per_page=10 "HTTP/1.1 304 Not Modified"
2026-09-28 20:30:11,977 - httpx - INFO - HTTP Request: GET https://api.github.com/repos/kdroidFilter/Zayit/events?per_page=10 "HTTP/1.1 200 OK"
2026-09-28 20:30:12,066 - httpx - INFO - HTTP Request: GET https://api.github.com/repos/EzraBrand/talmud-nlp-indexer/events?per_page=10 "HTTP/1.1 304 Not Modified"
2026-09-28 20:30:12,074 - httpx - INFO - HTTP Request: GET https://api.github.com/repos/brainheart/tanakh-contabulate/events?per_page=10 "HTTP/1.1 304 Not Modified"
2026-09-28 20:30:12,264 - httpx - INFO - HTTP Request: GET https://api.github.com/repos/Hbleiberg/Hebrew_Blender/events?per_page=10 "HTTP/1.1 200 OK"
2026-09-28 20:30:12,342 - httpx - INFO - HTTP Request: GET https://api.github.com/repos/Esoterics611/maayan/events?per_page=10 "HTTP/1.1 304 Not Modified"
2026-09-28 20:30:12,454 - YouTubeIntelScraper - INFO - YouTube Intel found 30 candidates and 3 new videos
2026-09-28 20:30:12,477 - httpx - INFO - HTTP Request: GET https://api.github.com/repos/naorbrown/daf-yomi-history-bot/events?per_page=10 "HTTP/1.1 304 Not Modified"
2026-09-28 20:30:12,533 - httpx - INFO - HTTP Request: GET https://api.github.com/repos/stenagmois-byte/SimHebrew-Concordance/events?per_page=10 "HTTP/1.1 304 Not Modified"
2026-09-28 20:30:12,591 - httpx - INFO - HTTP Request: GET https://api.github.com/repos/wuisabel-gif/Aleph-Bet/events?per_page=10 "HTTP/1.1 304 Not Modified"
2026-09-28 20:30:12,666 - httpx - INFO - HTTP Request: GET https://api.github.com/repos/jepick/talmud-map/events?per_page=10 "HTTP/1.1 304 Not Modified"
2026-09-28 20:30:12,970 - httpx - INFO - HTTP Request: GET https://api.github.com/repos/shloimie-beep/bnei-neviim-academy/events?per_page=10 "HTTP/1.1 304 Not Modified"
2026-09-28 20:30:13,094 - httpx - INFO - HTTP Request: GET https://api.github.com/repos/paperlesspaper/integrations/events?per_page=10 "HTTP/1.1 200 OK"
2026-09-28 20:30:13,114 - httpx - INFO - HTTP Request: GET https://api.github.com/repos/Asifah-Analytics-C-A-V-E/asifah-analytics/events?per_page=10 "HTTP/1.1 304 Not Modified"
2026-09-28 20:30:13,261 - httpx - INFO - HTTP Request: GET https://api.github.com/repos/complytime/complytime-collector-components/events?per_page=10 "HTTP/1.1 304 Not Modified"
2026-09-28 20:30:13,352 - httpx - INFO - HTTP Request: GET https://api.github.com/repos/DavidPatlas-AI/tehillim-russian-site/events?per_page=10 "HTTP/1.1 304 Not Modified"
2026-09-28 20:30:13,436 - httpx - INFO - HTTP Request: GET https://api.github.com/repos/dniasoff/learning-tracker/events?per_page=10 "HTTP/1.1 200 OK"
2026-09-28 20:30:13,713 - httpx - INFO - HTTP Request: GET https://api.github.com/repos/dmo18/MySugya/events?per_page=10 "HTTP/1.1 304 Not Modified"
2026-09-28 20:30:13,723 - httpx - INFO - HTTP Request: GET https://api.github.com/repos/hytric/NLP_term_project/events?per_page=10 "HTTP/1.1 304 Not Modified"
2026-09-28 20:30:13,794 - httpx - INFO - HTTP Request: GET https://api.github.com/repos/RonTurrentine/KJV-Strongs-EBook/events?per_page=10 "HTTP/1.1 304 Not Modified"
2026-09-28 20:30:13,850 - httpx - INFO - HTTP Request: GET https://api.github.com/repos/yossi-weinberger/ten10/events?per_page=10 "HTTP/1.1 304 Not Modified"
2026-09-28 20:30:14,071 - httpx - INFO - HTTP Request: GET https://api.github.com/repos/zarbinco/zarbin-seo/events?per_page=10 "HTTP/1.1 304 Not Modified"
2026-09-28 20:30:14,262 - httpx - INFO - HTTP Request: GET https://api.github.com/repos/duvarakesh06/FAB-Subtitler-12.1-Offline-Release/events?per_page=10 "HTTP/1.1 200 OK"
2026-09-28 20:30:14,320 - httpx - INFO - HTTP Request: GET https://api.github.com/repos/v97969181-commits/master-pdf-editor-legacy-release/events?per_page=10 "HTTP/1.1 304 Not Modified"
2026-09-28 20:30:14,322 - httpx - INFO - HTTP Request: GET https://api.github.com/repos/AUX440/liquidtext-ultimate-patch-kit/events?per_page=10 "HTTP/1.1 304 Not Modified"
2026-09-28 20:30:14,456 - httpx - INFO - HTTP Request: GET https://api.github.com/repos/elmadanimohammed78-del/daily-flow-landing/events?per_page=10 "HTTP/1.1 304 Not Modified"
2026-09-28 20:30:14,763 - httpx - INFO - HTTP Request: GET https://api.github.com/repos/Roger0129/bitwarden-2024-8-0-patchless-install/events?per_page=10 "HTTP/1.1 200 OK"
2026-09-28 20:30:14,765 - httpx - INFO - HTTP Request: GET https://api.github.com/repos/rafistrauss/commons-announcements/events?per_page=10 "HTTP/1.1 304 Not Modified"
2026-09-28 20:30:14,776 - httpx - INFO - HTTP Request: GET https://api.github.com/repos/Aljademaratas/readiris-product-code-generator/events?per_page=10 "HTTP/1.1 304 Not Modified"
2026-09-28 20:30:14,844 - httpx - INFO - HTTP Request: GET https://api.github.com/repos/apusaha0011/MenuSideKick/events?per_page=10 "HTTP/1.1 304 Not Modified"
2026-09-28 20:30:15,141 - httpx - INFO - HTTP Request: GET https://api.github.com/repos/semiagenticRob/100Reps/events?per_page=10 "HTTP/1.1 304 Not Modified"
2026-09-28 20:30:15,204 - httpx - INFO - HTTP Request: GET https://api.github.com/repos/chfle/chfle/events?per_page=10 "HTTP/1.1 304 Not Modified"
2026-09-28 20:30:15,272 - httpx - INFO - HTTP Request: GET https://api.github.com/repos/benemanuel/status/events?per_page=10 "HTTP/1.1 304 Not Modified"
2026-09-28 20:30:15,433 - httpx - INFO - HTTP Request: GET https://api.github.com/repos/Jossifresben/translation-alignment/events?per_page=10 "HTTP/1.1 304 Not Modified"
2026-09-28 20:30:15,475 - httpx - INFO - HTTP Request: GET https://api.github.com/repos/revanite-io/pvtr-gcp-cloud-storage/events?per_page=10 "HTTP/1.1 304 Not Modified"
2026-09-28 20:30:15,576 - httpx - INFO - HTTP Request: GET https://api.github.com/repos/egkristi/ToponymiaEuropaea/events?per_page=10 "HTTP/1.1 304 Not Modified"
2026-09-28 20:30:15,812 - httpx - INFO - HTTP Request: GET https://api.github.com/repos/hebcal/hebcal/events?per_page=10 "HTTP/1.1 304 Not Modified"
2026-09-28 20:30:15,828 - httpx - INFO - HTTP Request: GET https://api.github.com/repos/KosherJava/zmanim/events?per_page=10 "HTTP/1.1 304 Not Modified"
2026-09-28 20:30:15,904 - httpx - INFO - HTTP Request: GET https://api.github.com/repos/Israel-Kli/shabbat-timetable/events?per_page=10 "HTTP/1.1 304 Not Modified"
2026-09-28 20:30:16,108 - httpx - INFO - HTTP Request: GET https://api.github.com/repos/woodpeckerbros/watch-reminder/events?per_page=10 "HTTP/1.1 304 Not Modified"
2026-09-28 20:30:16,168 - httpx - INFO - HTTP Request: GET https://api.github.com/repos/svenalot-ui/sidur-native/events?per_page=10 "HTTP/1.1 304 Not Modified"
2026-09-28 20:30:16,279 - httpx - INFO - HTTP Request: GET https://api.github.com/repos/hebcal/learning/events?per_page=10 "HTTP/1.1 304 Not Modified"
2026-09-28 20:30:16,747 - httpx - INFO - HTTP Request: GET https://api.github.com/repos/joelcw/intsayt/events?per_page=10 "HTTP/1.1 304 Not Modified"
2026-09-28 20:30:16,777 - httpx - INFO - HTTP Request: GET https://api.github.com/repos/hebcal/hebcal-go/events?per_page=10 "HTTP/1.1 304 Not Modified"
2026-09-28 20:30:16,819 - httpx - INFO - HTTP Request: GET https://api.github.com/repos/Sefaria/ai-chatbot/events?per_page=10 "HTTP/1.1 304 Not Modified"
2026-09-28 20:30:16,849 - httpx - INFO - HTTP Request: GET https://api.github.com/repos/levidao-eth/Torah/events?per_page=10 "HTTP/1.1 304 Not Modified"
2026-09-28 20:30:17,186 - httpx - INFO - HTTP Request: GET https://api.github.com/repos/AdamYairLee/torah-korean-translation/events?per_page=10 "HTTP/1.1 304 Not Modified"
2026-09-28 20:30:17,219 - httpx - INFO - HTTP Request: GET https://api.github.com/repos/ainihonuragawa-lang/TalmudFinance/events?per_page=10 "HTTP/1.1 304 Not Modified"
2026-09-28 20:30:17,299 - httpx - INFO - HTTP Request: GET https://api.github.com/repos/zakdev26/sefaria-links/events?per_page=10 "HTTP/1.1 304 Not Modified"
2026-09-28 20:30:17,369 - httpx - INFO - HTTP Request: GET https://api.github.com/repos/complytime/complytime-policies/events?per_page=10 "HTTP/1.1 304 Not Modified"
2026-09-28 20:30:17,502 - httpx - INFO - HTTP Request: GET https://api.github.com/repos/Yiddish-Dialect-Dictionary/-brukhe-machine/events?per_page=10 "HTTP/1.1 304 Not Modified"
2026-09-28 20:30:17,657 - httpx - INFO - HTTP Request: GET https://api.github.com/repos/vaisisrael/gemara-benichuta/events?per_page=10 "HTTP/1.1 304 Not Modified"
2026-09-28 20:30:17,740 - httpx - INFO - HTTP Request: GET https://api.github.com/repos/SnirRadomsky/daf-yomi/events?per_page=10 "HTTP/1.1 304 Not Modified"
2026-09-28 20:30:17,788 - httpx - INFO - HTTP Request: GET https://api.github.com/repos/elconde/kosher-races/events?per_page=10 "HTTP/1.1 304 Not Modified"
2026-09-28 20:30:17,842 - httpx - INFO - HTTP Request: GET https://api.github.com/repos/jacopovirga/kindle-zmanim/events?per_page=10 "HTTP/1.1 304 Not Modified"
2026-09-28 20:30:17,984 - httpx - INFO - HTTP Request: GET https://api.github.com/repos/Selecr-Surplus/minyan-finder/events?per_page=10 "HTTP/1.1 304 Not Modified"
2026-09-28 20:30:18,147 - httpx - INFO - HTTP Request: GET https://api.github.com/repos/theOnionsAreWatching/Shtigletz/events?per_page=10 "HTTP/1.1 304 Not Modified"
2026-09-28 20:30:18,186 - httpx - INFO - HTTP Request: GET https://api.github.com/repos/aharonium/opensiddur.org/events?per_page=10 "HTTP/1.1 304 Not Modified"
2026-09-28 20:30:18,312 - httpx - INFO - HTTP Request: GET https://api.github.com/repos/robmoffat/FOSS-Contribution-Gemara/events?per_page=10 "HTTP/1.1 304 Not Modified"
2026-09-28 20:30:18,394 - httpx - INFO - HTTP Request: GET https://api.github.com/repos/naorbrown/peninei-halacha-yomi-bot/events?per_page=10 "HTTP/1.1 304 Not Modified"
2026-09-28 20:30:18,501 - httpx - INFO - HTTP Request: GET https://api.github.com/repos/mig2/mishnah-style/events?per_page=10 "HTTP/1.1 304 Not Modified"
2026-09-28 20:30:18,552 - httpx - INFO - HTTP Request: GET https://api.github.com/repos/vistaTeam/satmar/events?per_page=10 "HTTP/1.1 304 Not Modified"
2026-09-28 20:30:18,756 - httpx - INFO - HTTP Request: GET https://api.github.com/repos/skilead770/stoliskikosher/events?per_page=10 "HTTP/1.1 304 Not Modified"
2026-09-28 20:30:18,785 - httpx - INFO - HTTP Request: GET https://api.github.com/repos/mak-sandboxuser/chabad-react-and-supabase/events?per_page=10 "HTTP/1.1 304 Not Modified"
2026-09-28 20:30:18,821 - httpx - INFO - HTTP Request: GET https://api.github.com/repos/a0556770172-png/koshermat/events?per_page=10 "HTTP/1.1 304 Not Modified"
2026-09-28 20:30:18,912 - httpx - INFO - HTTP Request: GET https://api.github.com/repos/bnq9705-creator/siddur/events?per_page=10 "HTTP/1.1 304 Not Modified"
2026-09-28 20:30:19,135 - httpx - INFO - HTTP Request: GET https://api.github.com/repos/Kosher-T/fannexus/events?per_page=10 "HTTP/1.1 304 Not Modified"
2026-09-28 20:30:19,172 - httpx - INFO - HTTP Request: GET https://api.github.com/repos/yosefmalul52-lang/costa-rica-kosher-retreat/events?per_page=10 "HTTP/1.1 304 Not Modified"
2026-09-28 20:30:19,202 - httpx - INFO - HTTP Request: GET https://api.github.com/repos/RavBogard/shirei-tfilah/events?per_page=10 "HTTP/1.1 304 Not Modified"
2026-09-28 20:30:19,307 - httpx - INFO - HTTP Request: GET https://api.github.com/repos/ReuvenB-S/kosher-browser-data/events?per_page=10 "HTTP/1.1 304 Not Modified"
2026-09-28 20:30:19,498 - httpx - INFO - HTTP Request: GET https://api.github.com/repos/shliachflow/chabad-southern-delaware-camp/events?per_page=10 "HTTP/1.1 304 Not Modified"
2026-09-28 20:30:19,500 - httpx - INFO - HTTP Request: GET https://api.github.com/repos/AviMish10/tanach-map/events?per_page=10 "HTTP/1.1 304 Not Modified"
2026-09-28 20:30:19,580 - httpx - INFO - HTTP Request: GET https://api.github.com/repos/TanachatGKRB/TanachatGKRB.github.io/events?per_page=10 "HTTP/1.1 304 Not Modified"
2026-09-28 20:30:19,627 - httpx - INFO - HTTP Request: GET https://api.github.com/repos/TamamTamar/chabad-yafo/events?per_page=10 "HTTP/1.1 304 Not Modified"
2026-09-28 20:30:19,814 - httpx - INFO - HTTP Request: GET https://api.github.com/repos/TG-CONSULTANT/tanachok-stock-FG1/events?per_page=10 "HTTP/1.1 304 Not Modified"
2026-09-28 20:30:19,918 - httpx - INFO - HTTP Request: GET https://api.github.com/repos/DeepSynchronicity/chabadofhurlingham/events?per_page=10 "HTTP/1.1 304 Not Modified"
2026-09-28 20:30:19,970 - httpx - INFO - HTTP Request: GET https://api.github.com/repos/semiagenticRob/digital-siddur/events?per_page=10 "HTTP/1.1 304 Not Modified"
2026-09-28 20:30:19,983 - httpx - INFO - HTTP Request: GET https://api.github.com/repos/orelzion/daily-tanach/events?per_page=10 "HTTP/1.1 304 Not Modified"
2026-09-28 20:30:20,305 - httpx - INFO - HTTP Request: GET https://api.github.com/repos/TG-CONSULTANT/tanachok-stock-mat/events?per_page=10 "HTTP/1.1 304 Not Modified"
2026-09-28 20:30:20,350 - httpx - INFO - HTTP Request: GET https://api.github.com/repos/Frish-Gebakn/kol-satmar-website/events?per_page=10 "HTTP/1.1 304 Not Modified"
2026-09-28 20:30:20,375 - httpx - INFO - HTTP Request: GET https://api.github.com/repos/satmarthubby-blip/Satmarthubby/events?per_page=10 "HTTP/1.1 304 Not Modified"
2026-09-28 20:30:20,456 - httpx - INFO - HTTP Request: GET https://api.github.com/repos/akshatasatmar/WeatherServiceApi/events?per_page=10 "HTTP/1.1 304 Not Modified"
2026-09-28 20:30:20,658 - httpx - INFO - HTTP Request: GET https://api.github.com/repos/elchunenfreund/satmar-matzos-app/events?per_page=10 "HTTP/1.1 304 Not Modified"
2026-09-28 20:30:20,750 - httpx - INFO - HTTP Request: GET https://api.github.com/repos/satmaribianca05-eng/AFSD-Satmari-Bianca/events?per_page=10 "HTTP/1.1 304 Not Modified"
2026-09-28 20:30:20,933 - httpx - INFO - HTTP Request: GET https://api.github.com/repos/Sefaria/Sefaria-Export/events?per_page=10 "HTTP/1.1 304 Not Modified"
2026-09-28 20:30:20,962 - httpx - INFO - HTTP Request: GET https://api.github.com/repos/mritunjaysharma2004/SATMARG-An-Online-Pilgrimage-planner-/events?per_page=10 "HTTP/1.1 304 Not Modified"
2026-09-28 20:30:20,970 - httpx - INFO - HTTP Request: GET https://api.github.com/repos/hebcal/hebcal-es6/events?per_page=10 "HTTP/1.1 304 Not Modified"
2026-09-28 20:30:21,013 - GitHubFeedScraper - INFO - GitHub feed healthy: login=abeperl fetched=200 account_new=0 list_repos=100 list_new=0 private_new=0 total_new=0
2026-09-28 20:30:21,015 - __main__ - INFO - Scraper github completed: 0 results
2026-09-28 20:30:21,015 - __main__ - INFO - Scraper github_feed completed: 0 results
2026-09-28 20:30:21,015 - __main__ - INFO - Scraper youtube_intel completed: 3 results
2026-09-28 20:30:21,655 - social_media_monitor.email.sender - INFO - Email sent successfully at 2026-09-28 20:30:21.655785
2026-09-28 20:30:21,656 - __main__ - INFO - Email sent successfully
2026-09-28 20:30:21,656 - __main__ - INFO - Cache stats: 0 items in memory, 0 files on disk
2026-09-28 20:30:21,656 - __main__ - INFO - Monitor complete. Found 3 total results: 3 new youtube intel results.
2026-09-28 20:30:21,656 - __main__ - INFO - === Social Media Monitor v2.0 Complete ===
2026-09-28 20:30:21,656 - social_media_monitor.database.connection - INFO - Closing all database connections
2026-09-28 20:30:21,688 - social_media_monitor.database.connection - INFO - Closed 5 database connections
2026-09-28 20:30:21,688 - __main__ - INFO - Closed all database connections
2026-09-28 20:30:21,690 - social_media_monitor.database.connection - INFO - Closing all database connections
2026-09-28 20:30:21,691 - social_media_monitor.database.connection - INFO - Closed 0 database connections
2026-09-28 21:00:02,568 - social_media_monitor.utils.logging_config - INFO - Posts log file: /mnt/data/openclaw-shared/home/shared/Social_Media_Monitor/logs/social_media_monitor_posts_2026-09-28.log
2026-09-28 21:00:02,568 - social_media_monitor.utils.logging_config - INFO - Downloads log file: /mnt/data/openclaw-shared/home/shared/Social_Media_Monitor/logs/social_media_monitor_downloads_2026-09-28.log
2026-09-28 21:00:02,568 - social_media_monitor.utils.logging_config - INFO - Logging configured with level INFO, log file: /mnt/data/openclaw-shared/home/shared/Social_Media_Monitor/logs/social_media_monitor_2026-09-28.log
2026-09-28 21:00:02,568 - __main__ - INFO - === Social Media Monitor v2.0 (Async) Started ===
2026-09-28 21:00:02,597 - social_media_monitor.config.settings - INFO - Configuration loaded from /home/openclaw/Social_Media_Monitor/config.yaml
2026-09-28 21:00:02,597 - __main__ - INFO - Configuration loaded and validated successfully
2026-09-28 21:00:02,599 - social_media_monitor.database.connection - INFO - Database connection pool initialized with 5 connections
2026-09-28 21:00:02,600 - social_media_monitor.database.connection - INFO - add_speaker_filters column already exists
2026-09-28 21:00:02,613 - social_media_monitor.database.connection - INFO - Database setup completed successfully
2026-09-28 21:00:02,614 - __main__ - INFO - Database initialized at: /mnt/data/openclaw-shared/home/shared/Social_Media_Monitor/social_media_monitor.db
2026-09-28 21:00:02,666 - social_media_monitor.scrapers.reddit_scraper - INFO - Reddit API initialized successfully (UA='SocialMediaMonitor/1.0')
2026-09-28 21:00:02,669 - RSSFeedScraper - INFO - RSS scraper initialized: feeds=1, html_urls=0 (0 with custom selectors), include_terms=0, exclude_terms=0
2026-09-28 21:00:02,669 - __main__ - INFO - RSS scraper initialized and enabled
2026-09-28 21:00:02,675 - Yiddish24Scraper - INFO - File downloader initialized with 55 terms, prefix 'Y2', and file type organization enabled
2026-09-28 21:00:02,675 - Yiddish24Scraper - INFO - Yiddish24 scraper initialized: data_id='latest', page_limit=100, total_pages=10, debug_html=True, include_terms=55, exclude_terms=0
2026-09-28 21:00:02,675 - __main__ - INFO - Yiddish24 scraper initialized and enabled
2026-09-28 21:00:02,680 - AhBlickLiveScraper - INFO - AhBlickLive scraper initialized: urls=1, max_items=100, include_terms=49, exclude_terms=0
2026-09-28 21:00:02,686 - TorahAnytimeScraper - INFO - File downloader initialized with 53 terms, prefix 'TA', and file type organization enabled
2026-09-28 21:00:02,686 - TorahAnytimeScraper - INFO - TorahAnytime scraper initialized: limit=400, offset=0, project_id=1, include_terms=53, exclude_terms=13
2026-09-28 21:00:02,688 - BatorahScraper - INFO - File downloader initialized with 19 terms, prefix 'BT', and file type organization enabled
2026-09-28 21:00:02,688 - BatorahScraper - INFO - Batorah scraper initialized: api_urls=2, page_size=100, include_terms=19, exclude_terms=2
2026-09-28 21:00:02,688 - KolHalashonScraper - INFO - Kol Halashon scraper initialized: base_url=https://www2.kolhalashon.com, max_shiurim=50, fetch_all_speakers=True, headless=True, exclude_filters=1, include_filters=0
2026-09-28 21:00:02,691 - AllDafScraper - INFO - All Daf scraper initialized: enabled=True sections=['Daf Yomi Shiurim', 'Supplemental Clips', 'Reference Materials'] skip_terms=0 include_terms=0
2026-09-28 21:00:02,691 - __main__ - INFO - Ivelt scraper initialized and enabled
2026-09-28 21:00:02,692 - __main__ - INFO - Running 3 scrapers concurrently...
2026-09-28 21:00:02,692 - Yiddish24Scraper - INFO - Starting Yiddish24 scraping (max 10 pages, limit 100 items)
2026-09-28 21:00:02,778 - Yiddish24Scraper - INFO - Fetching Yiddish24 page 1 with data_id='latest', page_limit=100
2026-09-28 21:00:02,782 - IveltScraper - INFO - Loaded 182 include, 4 exclude filters from DB topic_filters
2026-09-28 21:00:02,782 - IveltScraper - INFO - Fetching ivelt active topics from https://www.ivelt.com/forum/search.php?search_id=active_topics...
2026-09-28 21:00:02,785 - RSSFeedScraper - INFO - Fetching RSS feed: https://www.theyeshivaworld.com/feed
2026-09-28 21:00:03,151 - RSSFeedScraper - INFO - Total RSS/HTML results: 0
2026-09-28 21:00:03,701 - httpx - INFO - HTTP Request: POST https://www.yiddish24.com/ajax/cat_pagination.php "HTTP/1.1 200 OK"
2026-09-28 21:00:03,927 - Yiddish24Scraper - INFO - Received response with 506173 characters
2026-09-28 21:00:03,930 - Yiddish24Scraper - INFO - Extracted HTML from JSON result field (313537 chars)
2026-09-28 21:00:03,933 - Yiddish24Scraper - INFO - Saved HTML debug file to: /mnt/data/openclaw-shared/home/shared/Social_Media_Monitor/debug/yiddish24_page1_response.html
2026-09-28 21:00:04,010 - Yiddish24Scraper - INFO - Found 100 potential article elements on page 1
2026-09-28 21:00:04,048 - Yiddish24Scraper - INFO - Page 1 extraction: 99 successful, 1 failed out of 100 elements
2026-09-28 21:00:04,048 - Yiddish24Scraper - INFO - Parsed 99 articles from HTML on page 1
2026-09-28 21:00:04,077 - Yiddish24Scraper - INFO - Term 'ืžืขืฉื”' matched in DESCRIPTION: 'ืžืขืฉื” ืœื›ื‘ื•ื“ ื—ื•ืœ ื”ืžื•ืขื“ ื“ื•ืจืš ื“ืขืจ ื‘ืืจื™ืžื˜ืขืจ ื‘ืขืœ ืžืกืคืจ ื”ืจื‘ ืžืชืชื™' ื’ืจื™ืŸ'
2026-09-28 21:00:04,077 - Yiddish24Scraper - INFO - Downloading 1 media file(s) for article: ื”ื›ืœ ื‘ื›ืœ
2026-09-28 21:00:04,578 - social_media_monitor.utils.file_downloader - INFO - Starting download from: https://cloudfront.yiddish24.com/18HBS_260_260928040852.mp3
2026-09-28 21:00:05,714 - social_media_monitor.utils.file_downloader - INFO - Successfully downloaded: Y2_18HBS_260_260928040852.mp3 (7.4 MB)
2026-09-28 21:00:05,716 - social_media_monitor.utils.file_downloader - INFO - Organized audio file to: /mnt/data/openclaw-shared/home/shared/Social_Media_Monitor/downloads/Audio (audio/mpeg)
2026-09-28 21:00:05,729 - social_media_monitor.utils.media_tagger - INFO - Tagged Y2_18HBS_260_260928040852.mp3: title, artist, album, comment, source_url, genre, publisher
2026-09-28 21:00:05,729 - Yiddish24Scraper - INFO - Renamed downloaded file to include match term: Y2_18HBS_260_260928040852_ืžืขืฉื”.mp3
2026-09-28 21:00:06,635 - Yiddish24Scraper - INFO - Page 1 summary: 2 new out of 99 total articles
2026-09-28 21:00:06,635 - Yiddish24Scraper - INFO - Page 1: 2/99 new items
2026-09-28 21:00:06,635 - Yiddish24Scraper - INFO - Low new item ratio (2.0%), stopping pagination
2026-09-28 21:00:06,636 - Yiddish24Scraper - INFO - Total Yiddish24 results found: 2
2026-09-28 21:00:17,306 - IveltScraper - INFO - Found 50 active topics on ivelt
2026-09-28 21:00:27,582 - IveltScraper - INFO - Filtering post containing exclude term: '*ื‘ื™ื˜ืข ืžืื›ื˜ ื–ื™ื›ืขืจ ื‘ืœื•ื™ื– ืฆื• ื‘ืืจื™ื›ื˜ืŸ ืงืจืื ื˜ืข ื ื™ื™ืขืก*'
2026-09-28 21:00:48,139 - IveltScraper - INFO - ivelt scrape complete. Checked 5 topics.
2026-09-28 21:00:48,140 - IveltScraper - INFO - All forums scrape complete. Found 45 new posts.
2026-09-28 21:00:48,140 - __main__ - INFO - Scraper rss completed: 0 results
2026-09-28 21:00:48,140 - __main__ - INFO - Scraper yiddish24 completed: 2 results
2026-09-28 21:00:48,140 - __main__ - INFO - Scraper ivelt completed: 45 results
2026-09-28 21:00:49,397 - social_media_monitor.email.sender - INFO - Email sent successfully at 2026-09-28 21:00:49.397908
2026-09-28 21:00:49,398 - __main__ - INFO - Email sent successfully
2026-09-28 21:00:49,398 - __main__ - INFO - Cache stats: 0 items in memory, 0 files on disk
2026-09-28 21:00:49,398 - __main__ - INFO - Monitor complete. Found 47 total results: 2 new yiddish24 results, 45 new ivelt results.
2026-09-28 21:00:49,398 - __main__ - INFO - === Social Media Monitor v2.0 Complete ===
2026-09-28 21:00:49,398 - social_media_monitor.database.connection - INFO - Closing all database connections
2026-09-28 21:00:49,505 - social_media_monitor.database.connection - INFO - Closed 5 database connections
2026-09-28 21:00:49,505 - __main__ - INFO - Closed all database connections
2026-09-28 21:00:49,508 - social_media_monitor.database.connection - INFO - Closing all database connections
2026-09-28 21:00:49,508 - social_media_monitor.database.connection - INFO - Closed 0 database connections
2026-09-28 22:00:02,423 - social_media_monitor.utils.logging_config - INFO - Posts log file: /mnt/data/openclaw-shared/home/shared/Social_Media_Monitor/logs/social_media_monitor_posts_2026-09-28.log
2026-09-28 22:00:02,423 - social_media_monitor.utils.logging_config - INFO - Downloads log file: /mnt/data/openclaw-shared/home/shared/Social_Media_Monitor/logs/social_media_monitor_downloads_2026-09-28.log
2026-09-28 22:00:02,423 - social_media_monitor.utils.logging_config - INFO - Logging configured with level INFO, log file: /mnt/data/openclaw-shared/home/shared/Social_Media_Monitor/logs/social_media_monitor_2026-09-28.log
2026-09-28 22:00:02,423 - __main__ - INFO - === Social Media Monitor v2.0 (Async) Started ===
2026-09-28 22:00:02,451 - social_media_monitor.config.settings - INFO - Configuration loaded from /home/openclaw/Social_Media_Monitor/config.yaml
2026-09-28 22:00:02,452 - __main__ - INFO - Configuration loaded and validated successfully
2026-09-28 22:00:02,454 - social_media_monitor.database.connection - INFO - Database connection pool initialized with 5 connections
2026-09-28 22:00:02,454 - social_media_monitor.database.connection - INFO - add_speaker_filters column already exists
2026-09-28 22:00:02,469 - social_media_monitor.database.connection - INFO - Database setup completed successfully
2026-09-28 22:00:02,469 - __main__ - INFO - Database initialized at: /mnt/data/openclaw-shared/home/shared/Social_Media_Monitor/social_media_monitor.db
2026-09-28 22:00:02,471 - social_media_monitor.scrapers.reddit_scraper - INFO - Reddit API initialized successfully (UA='SocialMediaMonitor/1.0')
2026-09-28 22:00:02,472 - RSSFeedScraper - INFO - RSS scraper initialized: feeds=1, html_urls=0 (0 with custom selectors), include_terms=0, exclude_terms=0
2026-09-28 22:00:02,472 - __main__ - INFO - RSS scraper initialized and enabled
2026-09-28 22:00:02,476 - Yiddish24Scraper - INFO - File downloader initialized with 55 terms, prefix 'Y2', and file type organization enabled
2026-09-28 22:00:02,476 - Yiddish24Scraper - INFO - Yiddish24 scraper initialized: data_id='latest', page_limit=100, total_pages=10, debug_html=True, include_terms=55, exclude_terms=0
2026-09-28 22:00:02,476 - __main__ - INFO - Yiddish24 scraper initialized and enabled
2026-09-28 22:00:02,481 - AhBlickLiveScraper - INFO - AhBlickLive scraper initialized: urls=1, max_items=100, include_terms=49, exclude_terms=0
2026-09-28 22:00:02,485 - TorahAnytimeScraper - INFO - File downloader initialized with 53 terms, prefix 'TA', and file type organization enabled
2026-09-28 22:00:02,485 - TorahAnytimeScraper - INFO - TorahAnytime scraper initialized: limit=400, offset=0, project_id=1, include_terms=53, exclude_terms=13
2026-09-28 22:00:02,488 - BatorahScraper - INFO - File downloader initialized with 19 terms, prefix 'BT', and file type organization enabled
2026-09-28 22:00:02,488 - BatorahScraper - INFO - Batorah scraper initialized: api_urls=2, page_size=100, include_terms=19, exclude_terms=2
2026-09-28 22:00:02,488 - KolHalashonScraper - INFO - Kol Halashon scraper initialized: base_url=https://www2.kolhalashon.com, max_shiurim=50, fetch_all_speakers=True, headless=True, exclude_filters=1, include_filters=0
2026-09-28 22:00:02,490 - AllDafScraper - INFO - All Daf scraper initialized: enabled=True sections=['Daf Yomi Shiurim', 'Supplemental Clips', 'Reference Materials'] skip_terms=0 include_terms=0
2026-09-28 22:00:02,490 - __main__ - INFO - Ivelt scraper initialized and enabled
2026-09-28 22:00:02,490 - __main__ - INFO - Running 3 scrapers concurrently...
2026-09-28 22:00:02,491 - Yiddish24Scraper - INFO - Starting Yiddish24 scraping (max 10 pages, limit 100 items)
2026-09-28 22:00:02,578 - Yiddish24Scraper - INFO - Fetching Yiddish24 page 1 with data_id='latest', page_limit=100
2026-09-28 22:00:02,583 - IveltScraper - INFO - Loaded 182 include, 4 exclude filters from DB topic_filters
2026-09-28 22:00:02,583 - IveltScraper - INFO - Fetching ivelt active topics from https://www.ivelt.com/forum/search.php?search_id=active_topics...
2026-09-28 22:00:02,584 - RSSFeedScraper - INFO - Fetching RSS feed: https://www.theyeshivaworld.com/feed
2026-09-28 22:00:02,880 - RSSFeedScraper - INFO - Total RSS/HTML results: 0
2026-09-28 22:00:03,294 - httpx - INFO - HTTP Request: POST https://www.yiddish24.com/ajax/cat_pagination.php "HTTP/1.1 200 OK"
2026-09-28 22:00:03,524 - Yiddish24Scraper - INFO - Received response with 506057 characters
2026-09-28 22:00:03,527 - Yiddish24Scraper - INFO - Extracted HTML from JSON result field (313498 chars)
2026-09-28 22:00:03,530 - Yiddish24Scraper - INFO - Saved HTML debug file to: /mnt/data/openclaw-shared/home/shared/Social_Media_Monitor/debug/yiddish24_page1_response.html
2026-09-28 22:00:03,621 - Yiddish24Scraper - INFO - Found 100 potential article elements on page 1
2026-09-28 22:00:03,667 - Yiddish24Scraper - INFO - Page 1 extraction: 99 successful, 1 failed out of 100 elements
2026-09-28 22:00:03,667 - Yiddish24Scraper - INFO - Parsed 99 articles from HTML on page 1
2026-09-28 22:00:04,642 - Yiddish24Scraper - INFO - Page 1 summary: 10 new out of 99 total articles
2026-09-28 22:00:04,643 - Yiddish24Scraper - INFO - Page 1: 10/99 new items
2026-09-28 22:00:04,643 - Yiddish24Scraper - INFO - Low new item ratio (10.1%), stopping pagination
2026-09-28 22:00:04,643 - Yiddish24Scraper - INFO - Total Yiddish24 results found: 10
2026-09-28 22:00:19,227 - IveltScraper - INFO - Found 50 active topics on ivelt
2026-09-28 22:00:53,283 - IveltScraper - INFO - Filtering post containing exclude term: '*ื‘ื™ื˜ืข ืžืื›ื˜ ื–ื™ื›ืขืจ ื‘ืœื•ื™ื– ืฆื• ื‘ืืจื™ื›ื˜ืŸ ืงืจืื ื˜ืข ื ื™ื™ืขืก*'
2026-09-28 22:01:34,629 - IveltScraper - INFO - ivelt scrape complete. Checked 7 topics.
2026-09-28 22:01:34,629 - IveltScraper - INFO - All forums scrape complete. Found 12 new posts.
2026-09-28 22:01:34,629 - __main__ - INFO - Scraper rss completed: 0 results
2026-09-28 22:01:34,629 - __main__ - INFO - Scraper yiddish24 completed: 10 results
2026-09-28 22:01:34,629 - __main__ - INFO - Scraper ivelt completed: 12 results
2026-09-28 22:01:36,071 - social_media_monitor.email.sender - INFO - Email sent successfully at 2026-09-28 22:01:36.071437
2026-09-28 22:01:36,071 - __main__ - INFO - Email sent successfully
2026-09-28 22:01:36,072 - __main__ - INFO - Cache stats: 0 items in memory, 0 files on disk
2026-09-28 22:01:36,072 - __main__ - INFO - Monitor complete. Found 22 total results: 10 new yiddish24 results, 12 new ivelt results.
2026-09-28 22:01:36,072 - __main__ - INFO - === Social Media Monitor v2.0 Complete ===
2026-09-28 22:01:36,072 - social_media_monitor.database.connection - INFO - Closing all database connections
2026-09-28 22:01:36,250 - social_media_monitor.database.connection - INFO - Closed 5 database connections
2026-09-28 22:01:36,250 - __main__ - INFO - Closed all database connections
2026-09-28 22:01:36,253 - social_media_monitor.database.connection - INFO - Closing all database connections
2026-09-28 22:01:36,253 - social_media_monitor.database.connection - INFO - Closed 0 database connections
2026-09-28 22:30:02,001 - social_media_monitor.utils.logging_config - INFO - Posts log file: /mnt/data/openclaw-shared/home/shared/Social_Media_Monitor/logs/social_media_monitor_posts_2026-09-28.log
2026-09-28 22:30:02,001 - social_media_monitor.utils.logging_config - INFO - Downloads log file: /mnt/data/openclaw-shared/home/shared/Social_Media_Monitor/logs/social_media_monitor_downloads_2026-09-28.log
2026-09-28 22:30:02,001 - social_media_monitor.utils.logging_config - INFO - Logging configured with level INFO, log file: /mnt/data/openclaw-shared/home/shared/Social_Media_Monitor/logs/social_media_monitor_2026-09-28.log
2026-09-28 22:30:02,001 - __main__ - INFO - === Social Media Monitor v2.0 (Async) Started ===
2026-09-28 22:30:02,030 - social_media_monitor.config.settings - INFO - Configuration loaded from /home/openclaw/Social_Media_Monitor/config.yaml
2026-09-28 22:30:02,030 - __main__ - INFO - Configuration loaded and validated successfully
2026-09-28 22:30:02,032 - social_media_monitor.database.connection - INFO - Database connection pool initialized with 5 connections
2026-09-28 22:30:02,033 - social_media_monitor.database.connection - INFO - add_speaker_filters column already exists
2026-09-28 22:30:02,052 - social_media_monitor.database.connection - INFO - Database setup completed successfully
2026-09-28 22:30:02,052 - __main__ - INFO - Database initialized at: /mnt/data/openclaw-shared/home/shared/Social_Media_Monitor/social_media_monitor.db
2026-09-28 22:30:02,111 - social_media_monitor.scrapers.reddit_scraper - INFO - Reddit API initialized successfully (UA='SocialMediaMonitor/1.0')
2026-09-28 22:30:02,112 - __main__ - INFO - GitHub scraper initialized and enabled
2026-09-28 22:30:02,113 - __main__ - INFO - GitHub feed scraper initialized and enabled
2026-09-28 22:30:02,113 - RSSFeedScraper - INFO - RSS scraper initialized: feeds=1, html_urls=0 (0 with custom selectors), include_terms=0, exclude_terms=0
2026-09-28 22:30:02,120 - Yiddish24Scraper - INFO - File downloader initialized with 55 terms, prefix 'Y2', and file type organization enabled
2026-09-28 22:30:02,120 - Yiddish24Scraper - INFO - Yiddish24 scraper initialized: data_id='latest', page_limit=100, total_pages=10, debug_html=True, include_terms=55, exclude_terms=0
2026-09-28 22:30:02,125 - AhBlickLiveScraper - INFO - AhBlickLive scraper initialized: urls=1, max_items=100, include_terms=49, exclude_terms=0
2026-09-28 22:30:02,132 - TorahAnytimeScraper - INFO - File downloader initialized with 53 terms, prefix 'TA', and file type organization enabled
2026-09-28 22:30:02,132 - TorahAnytimeScraper - INFO - TorahAnytime scraper initialized: limit=400, offset=0, project_id=1, include_terms=53, exclude_terms=13
2026-09-28 22:30:02,134 - BatorahScraper - INFO - File downloader initialized with 19 terms, prefix 'BT', and file type organization enabled
2026-09-28 22:30:02,134 - BatorahScraper - INFO - Batorah scraper initialized: api_urls=2, page_size=100, include_terms=19, exclude_terms=2
2026-09-28 22:30:02,134 - KolHalashonScraper - INFO - Kol Halashon scraper initialized: base_url=https://www2.kolhalashon.com, max_shiurim=50, fetch_all_speakers=True, headless=True, exclude_filters=1, include_filters=0
2026-09-28 22:30:02,136 - AllDafScraper - INFO - All Daf scraper initialized: enabled=True sections=['Daf Yomi Shiurim', 'Supplemental Clips', 'Reference Materials'] skip_terms=0 include_terms=0
2026-09-28 22:30:02,137 - __main__ - INFO - YouTube Intel scraper initialized and enabled
2026-09-28 22:30:02,137 - __main__ - INFO - Running 3 scrapers concurrently...
2026-09-28 22:30:02,137 - GitHubScraper - INFO - Searching GitHub for terms: ['"Hermes Agent"', '"grok bot OR grokbot"', '"muse NOT music NOT museum"', '"prime agent"', 'paperclip']
2026-09-28 22:30:02,540 - httpx - INFO - HTTP Request: GET https://api.github.com/user "HTTP/1.1 200 OK"
2026-09-28 22:30:02,541 - GitHubFeedScraper - INFO - GitHub auth preflight passed for abeperl; token scopes: not reported (fine-grained token)
2026-09-28 22:30:03,284 - httpx - INFO - HTTP Request: GET https://api.github.com/search/repositories?q=%22Hermes+Agent%22+in%3Aname%2Cdescription%2Creadme+stars%3A%3E150&sort=updated&order=desc&per_page=20 "HTTP/1.1 200 OK"
2026-09-28 22:30:03,311 - GitHubScraper - INFO - Found 649 GitHub repositories for term: "Hermes Agent"
2026-09-28 22:30:03,365 - GitHubScraper - INFO - Added new GitHub repository: hermes-conduit
2026-09-28 22:30:03,729 - httpx - INFO - HTTP Request: GET https://api.github.com/users/abeperl/received_events?per_page=100&page=1 "HTTP/1.1 200 OK"
2026-09-28 22:30:03,948 - httpx - INFO - HTTP Request: GET https://api.github.com/search/repositories?q=%22grok+bot+OR+grokbot%22+in%3Aname%2Cdescription%2Creadme+stars%3A%3E150&sort=updated&order=desc&per_page=20 "HTTP/1.1 200 OK"
2026-09-28 22:30:03,949 - GitHubScraper - INFO - Found 0 GitHub repositories for term: "grok bot OR grokbot"
2026-09-28 22:30:04,914 - httpx - INFO - HTTP Request: GET https://api.github.com/search/repositories?q=%22muse+NOT+music+NOT+museum%22+in%3Aname%2Cdescription%2Creadme+stars%3A%3E150&sort=updated&order=desc&per_page=20 "HTTP/1.1 200 OK"
2026-09-28 22:30:04,915 - GitHubScraper - INFO - Found 0 GitHub repositories for term: "muse NOT music NOT museum"
2026-09-28 22:30:05,216 - httpx - INFO - HTTP Request: GET https://api.github.com/users/abeperl/received_events?per_page=100&page=2 "HTTP/1.1 200 OK"
2026-09-28 22:30:06,524 - httpx - INFO - HTTP Request: GET https://api.github.com/users/abeperl/received_events?per_page=100&page=3 "HTTP/1.1 200 OK"
2026-09-28 22:30:06,628 - httpx - INFO - HTTP Request: GET https://api.github.com/search/repositories?q=%22prime+agent%22+in%3Aname%2Cdescription%2Creadme+stars%3A%3E150&sort=updated&order=desc&per_page=20 "HTTP/1.1 200 OK"
2026-09-28 22:30:06,711 - GitHubScraper - INFO - Found 44 GitHub repositories for term: "prime agent"
2026-09-28 22:30:07,902 - httpx - INFO - HTTP Request: GET https://api.github.com/search/repositories?q=paperclip+in%3Aname%2Cdescription%2Creadme+stars%3A%3E150&sort=updated&order=desc&per_page=20 "HTTP/1.1 200 OK"
2026-09-28 22:30:07,907 - GitHubScraper - INFO - Found 266 GitHub repositories for term: paperclip
2026-09-28 22:30:08,128 - GitHubScraper - INFO - Total GitHub results found: 1
2026-09-28 22:30:10,057 - httpx - INFO - HTTP Request: POST https://api.github.com/graphql "HTTP/1.1 200 OK"
2026-09-28 22:30:10,512 - httpx - INFO - HTTP Request: GET https://api.github.com/repos/akivayevdayev-debug/Shelah_app/events?per_page=10 "HTTP/1.1 304 Not Modified"
2026-09-28 22:30:10,514 - httpx - INFO - HTTP Request: GET https://api.github.com/repos/efiles-star/ShasTracker/events?per_page=10 "HTTP/1.1 304 Not Modified"
2026-09-28 22:30:10,532 - httpx - INFO - HTTP Request: GET https://api.github.com/repos/yehoshuayaakov/mishnayos-signup/events?per_page=10 "HTTP/1.1 304 Not Modified"
2026-09-28 22:30:10,610 - httpx - INFO - HTTP Request: GET https://api.github.com/repos/Biswaspr1022/KodeshMode/events?per_page=10 "HTTP/1.1 304 Not Modified"
2026-09-28 22:30:10,799 - httpx - INFO - HTTP Request: GET https://api.github.com/repos/nnanach/torah-bot/events?per_page=10 "HTTP/1.1 304 Not Modified"
2026-09-28 22:30:10,912 - httpx - INFO - HTTP Request: GET https://api.github.com/repos/Oranburg/havruta/events?per_page=10 "HTTP/1.1 304 Not Modified"
2026-09-28 22:30:11,001 - httpx - INFO - HTTP Request: GET https://api.github.com/repos/azernjj/talmudai/events?per_page=10 "HTTP/1.1 304 Not Modified"
2026-09-28 22:30:11,095 - httpx - INFO - HTTP Request: GET https://api.github.com/repos/hitchin999/YidCal/events?per_page=10 "HTTP/1.1 304 Not Modified"
2026-09-28 22:30:11,131 - httpx - INFO - HTTP Request: GET https://api.github.com/repos/ethanpil/daily-zmanim/events?per_page=10 "HTTP/1.1 304 Not Modified"
2026-09-28 22:30:11,273 - httpx - INFO - HTTP Request: GET https://api.github.com/repos/rabbishimon613-lang/chabad-tracker/events?per_page=10 "HTTP/1.1 304 Not Modified"
2026-09-28 22:30:11,365 - httpx - INFO - HTTP Request: GET https://api.github.com/repos/adunnmobilenotary-gif/halacha-search/events?per_page=10 "HTTP/1.1 304 Not Modified"
2026-09-28 22:30:11,457 - httpx - INFO - HTTP Request: GET https://api.github.com/repos/Chabadrt/chabad-cms-backend/events?per_page=10 "HTTP/1.1 304 Not Modified"
2026-09-28 22:30:11,482 - httpx - INFO - HTTP Request: GET https://api.github.com/repos/IsraelCIL/SiddurAmIsrael/events?per_page=10 "HTTP/1.1 304 Not Modified"
2026-09-28 22:30:11,703 - httpx - INFO - HTTP Request: GET https://api.github.com/repos/ronrefael/kav-siddur-site/events?per_page=10 "HTTP/1.1 304 Not Modified"
2026-09-28 22:30:11,704 - httpx - INFO - HTTP Request: GET https://api.github.com/repos/ElikoGrins/Halacha-bot/events?per_page=10 "HTTP/1.1 304 Not Modified"
2026-09-28 22:30:11,861 - httpx - INFO - HTTP Request: GET https://api.github.com/repos/EzraBrand/talmud-nlp-indexer/events?per_page=10 "HTTP/1.1 304 Not Modified"
2026-09-28 22:30:11,944 - httpx - INFO - HTTP Request: GET https://api.github.com/repos/kdroidFilter/Zayit/events?per_page=10 "HTTP/1.1 304 Not Modified"
2026-09-28 22:30:12,040 - httpx - INFO - HTTP Request: GET https://api.github.com/repos/brainheart/tanakh-contabulate/events?per_page=10 "HTTP/1.1 304 Not Modified"
2026-09-28 22:30:12,144 - YouTubeIntelScraper - INFO - YouTube Intel found 30 candidates and 5 new videos
2026-09-28 22:30:12,172 - httpx - INFO - HTTP Request: GET https://api.github.com/repos/Esoterics611/maayan/events?per_page=10 "HTTP/1.1 304 Not Modified"
2026-09-28 22:30:12,343 - httpx - INFO - HTTP Request: GET https://api.github.com/repos/Hbleiberg/Hebrew_Blender/events?per_page=10 "HTTP/1.1 200 OK"
2026-09-28 22:30:12,354 - httpx - INFO - HTTP Request: GET https://api.github.com/repos/stenagmois-byte/SimHebrew-Concordance/events?per_page=10 "HTTP/1.1 304 Not Modified"
2026-09-28 22:30:12,432 - httpx - INFO - HTTP Request: GET https://api.github.com/repos/naorbrown/daf-yomi-history-bot/events?per_page=10 "HTTP/1.1 304 Not Modified"
2026-09-28 22:30:12,517 - httpx - INFO - HTTP Request: GET https://api.github.com/repos/wuisabel-gif/Aleph-Bet/events?per_page=10 "HTTP/1.1 304 Not Modified"
2026-09-28 22:30:12,702 - httpx - INFO - HTTP Request: GET https://api.github.com/repos/jepick/talmud-map/events?per_page=10 "HTTP/1.1 304 Not Modified"
2026-09-28 22:30:12,837 - httpx - INFO - HTTP Request: GET https://api.github.com/repos/shloimie-beep/bnei-neviim-academy/events?per_page=10 "HTTP/1.1 304 Not Modified"
2026-09-28 22:30:12,935 - httpx - INFO - HTTP Request: GET https://api.github.com/repos/paperlesspaper/integrations/events?per_page=10 "HTTP/1.1 304 Not Modified"
2026-09-28 22:30:13,055 - httpx - INFO - HTTP Request: GET https://api.github.com/repos/Asifah-Analytics-C-A-V-E/asifah-analytics/events?per_page=10 "HTTP/1.1 304 Not Modified"
2026-09-28 22:30:13,123 - httpx - INFO - HTTP Request: GET https://api.github.com/repos/DavidPatlas-AI/tehillim-russian-site/events?per_page=10 "HTTP/1.1 304 Not Modified"
2026-09-28 22:30:13,245 - httpx - INFO - HTTP Request: GET https://api.github.com/repos/complytime/complytime-collector-components/events?per_page=10 "HTTP/1.1 304 Not Modified"
2026-09-28 22:30:13,319 - httpx - INFO - HTTP Request: GET https://api.github.com/repos/dniasoff/learning-tracker/events?per_page=10 "HTTP/1.1 304 Not Modified"
2026-09-28 22:30:13,509 - httpx - INFO - HTTP Request: GET https://api.github.com/repos/yossi-weinberger/ten10/events?per_page=10 "HTTP/1.1 304 Not Modified"
2026-09-28 22:30:13,591 - httpx - INFO - HTTP Request: GET https://api.github.com/repos/hytric/NLP_term_project/events?per_page=10 "HTTP/1.1 304 Not Modified"
2026-09-28 22:30:13,604 - httpx - INFO - HTTP Request: GET https://api.github.com/repos/RonTurrentine/KJV-Strongs-EBook/events?per_page=10 "HTTP/1.1 304 Not Modified"
2026-09-28 22:30:13,643 - httpx - INFO - HTTP Request: GET https://api.github.com/repos/dmo18/MySugya/events?per_page=10 "HTTP/1.1 304 Not Modified"
2026-09-28 22:30:13,880 - httpx - INFO - HTTP Request: GET https://api.github.com/repos/zarbinco/zarbin-seo/events?per_page=10 "HTTP/1.1 304 Not Modified"
2026-09-28 22:30:14,053 - httpx - INFO - HTTP Request: GET https://api.github.com/repos/AUX440/liquidtext-ultimate-patch-kit/events?per_page=10 "HTTP/1.1 304 Not Modified"
2026-09-28 22:30:14,059 - httpx - INFO - HTTP Request: GET https://api.github.com/repos/v97969181-commits/master-pdf-editor-legacy-release/events?per_page=10 "HTTP/1.1 304 Not Modified"
2026-09-28 22:30:14,087 - httpx - INFO - HTTP Request: GET https://api.github.com/repos/duvarakesh06/FAB-Subtitler-12.1-Offline-Release/events?per_page=10 "HTTP/1.1 304 Not Modified"
2026-09-28 22:30:14,185 - httpx - INFO - HTTP Request: GET https://api.github.com/repos/elmadanimohammed78-del/daily-flow-landing/events?per_page=10 "HTTP/1.1 304 Not Modified"
2026-09-28 22:30:14,500 - httpx - INFO - HTTP Request: GET https://api.github.com/repos/Aljademaratas/readiris-product-code-generator/events?per_page=10 "HTTP/1.1 200 OK"
2026-09-28 22:30:14,540 - httpx - INFO - HTTP Request: GET https://api.github.com/repos/apusaha0011/MenuSideKick/events?per_page=10 "HTTP/1.1 304 Not Modified"
2026-09-28 22:30:14,558 - httpx - INFO - HTTP Request: GET https://api.github.com/repos/rafistrauss/commons-announcements/events?per_page=10 "HTTP/1.1 304 Not Modified"
2026-09-28 22:30:14,581 - httpx - INFO - HTTP Request: GET https://api.github.com/repos/Roger0129/bitwarden-2024-8-0-patchless-install/events?per_page=10 "HTTP/1.1 304 Not Modified"
2026-09-28 22:30:14,836 - httpx - INFO - HTTP Request: GET https://api.github.com/repos/chfle/chfle/events?per_page=10 "HTTP/1.1 304 Not Modified"
2026-09-28 22:30:14,862 - httpx - INFO - HTTP Request: GET https://api.github.com/repos/semiagenticRob/100Reps/events?per_page=10 "HTTP/1.1 304 Not Modified"
2026-09-28 22:30:14,972 - httpx - INFO - HTTP Request: GET https://api.github.com/repos/benemanuel/status/events?per_page=10 "HTTP/1.1 304 Not Modified"
2026-09-28 22:30:15,194 - httpx - INFO - HTTP Request: GET https://api.github.com/repos/Jossifresben/translation-alignment/events?per_page=10 "HTTP/1.1 304 Not Modified"
2026-09-28 22:30:15,229 - httpx - INFO - HTTP Request: GET https://api.github.com/repos/egkristi/ToponymiaEuropaea/events?per_page=10 "HTTP/1.1 304 Not Modified"
2026-09-28 22:30:15,310 - httpx - INFO - HTTP Request: GET https://api.github.com/repos/revanite-io/pvtr-gcp-cloud-storage/events?per_page=10 "HTTP/1.1 304 Not Modified"
2026-09-28 22:30:15,477 - httpx - INFO - HTTP Request: GET https://api.github.com/repos/hebcal/hebcal/events?per_page=10 "HTTP/1.1 304 Not Modified"
2026-09-28 22:30:15,518 - httpx - INFO - HTTP Request: GET https://api.github.com/repos/KosherJava/zmanim/events?per_page=10 "HTTP/1.1 304 Not Modified"
2026-09-28 22:30:15,597 - httpx - INFO - HTTP Request: GET https://api.github.com/repos/Israel-Kli/shabbat-timetable/events?per_page=10 "HTTP/1.1 304 Not Modified"
2026-09-28 22:30:15,754 - httpx - INFO - HTTP Request: GET https://api.github.com/repos/woodpeckerbros/watch-reminder/events?per_page=10 "HTTP/1.1 304 Not Modified"
2026-09-28 22:30:15,900 - httpx - INFO - HTTP Request: GET https://api.github.com/repos/svenalot-ui/sidur-native/events?per_page=10 "HTTP/1.1 304 Not Modified"
2026-09-28 22:30:15,941 - httpx - INFO - HTTP Request: GET https://api.github.com/repos/hebcal/learning/events?per_page=10 "HTTP/1.1 304 Not Modified"
2026-09-28 22:30:16,130 - httpx - INFO - HTTP Request: GET https://api.github.com/repos/hebcal/hebcal-go/events?per_page=10 "HTTP/1.1 200 OK"
2026-09-28 22:30:16,307 - httpx - INFO - HTTP Request: GET https://api.github.com/repos/levidao-eth/Torah/events?per_page=10 "HTTP/1.1 304 Not Modified"
2026-09-28 22:30:16,320 - httpx - INFO - HTTP Request: GET https://api.github.com/repos/joelcw/intsayt/events?per_page=10 "HTTP/1.1 304 Not Modified"
2026-09-28 22:30:16,401 - httpx - INFO - HTTP Request: GET https://api.github.com/repos/Sefaria/ai-chatbot/events?per_page=10 "HTTP/1.1 304 Not Modified"
2026-09-28 22:30:16,701 - httpx - INFO - HTTP Request: GET https://api.github.com/repos/complytime/complytime-policies/events?per_page=10 "HTTP/1.1 304 Not Modified"
2026-09-28 22:30:16,704 - httpx - INFO - HTTP Request: GET https://api.github.com/repos/AdamYairLee/torah-korean-translation/events?per_page=10 "HTTP/1.1 304 Not Modified"
2026-09-28 22:30:16,707 - httpx - INFO - HTTP Request: GET https://api.github.com/repos/zakdev26/sefaria-links/events?per_page=10 "HTTP/1.1 304 Not Modified"
2026-09-28 22:30:16,780 - httpx - INFO - HTTP Request: GET https://api.github.com/repos/ainihonuragawa-lang/TalmudFinance/events?per_page=10 "HTTP/1.1 304 Not Modified"
2026-09-28 22:30:17,017 - httpx - INFO - HTTP Request: GET https://api.github.com/repos/Yiddish-Dialect-Dictionary/-brukhe-machine/events?per_page=10 "HTTP/1.1 304 Not Modified"
2026-09-28 22:30:17,047 - httpx - INFO - HTTP Request: GET https://api.github.com/repos/SnirRadomsky/daf-yomi/events?per_page=10 "HTTP/1.1 304 Not Modified"
2026-09-28 22:30:17,126 - httpx - INFO - HTTP Request: GET https://api.github.com/repos/elconde/kosher-races/events?per_page=10 "HTTP/1.1 304 Not Modified"
2026-09-28 22:30:17,131 - httpx - INFO - HTTP Request: GET https://api.github.com/repos/vaisisrael/gemara-benichuta/events?per_page=10 "HTTP/1.1 304 Not Modified"
2026-09-28 22:30:17,396 - httpx - INFO - HTTP Request: GET https://api.github.com/repos/jacopovirga/kindle-zmanim/events?per_page=10 "HTTP/1.1 304 Not Modified"
2026-09-28 22:30:17,424 - httpx - INFO - HTTP Request: GET https://api.github.com/repos/Selecr-Surplus/minyan-finder/events?per_page=10 "HTTP/1.1 304 Not Modified"
2026-09-28 22:30:17,482 - httpx - INFO - HTTP Request: GET https://api.github.com/repos/robmoffat/FOSS-Contribution-Gemara/events?per_page=10 "HTTP/1.1 304 Not Modified"
2026-09-28 22:30:17,483 - httpx - INFO - HTTP Request: GET https://api.github.com/repos/aharonium/opensiddur.org/events?per_page=10 "HTTP/1.1 304 Not Modified"
2026-09-28 22:30:17,755 - httpx - INFO - HTTP Request: GET https://api.github.com/repos/theOnionsAreWatching/Shtigletz/events?per_page=10 "HTTP/1.1 304 Not Modified"
2026-09-28 22:30:17,784 - httpx - INFO - HTTP Request: GET https://api.github.com/repos/vistaTeam/satmar/events?per_page=10 "HTTP/1.1 304 Not Modified"
2026-09-28 22:30:17,810 - httpx - INFO - HTTP Request: GET https://api.github.com/repos/naorbrown/peninei-halacha-yomi-bot/events?per_page=10 "HTTP/1.1 304 Not Modified"
2026-09-28 22:30:17,850 - httpx - INFO - HTTP Request: GET https://api.github.com/repos/mig2/mishnah-style/events?per_page=10 "HTTP/1.1 304 Not Modified"
2026-09-28 22:30:18,057 - httpx - INFO - HTTP Request: GET https://api.github.com/repos/mak-sandboxuser/chabad-react-and-supabase/events?per_page=10 "HTTP/1.1 304 Not Modified"
2026-09-28 22:30:18,159 - httpx - INFO - HTTP Request: GET https://api.github.com/repos/skilead770/stoliskikosher/events?per_page=10 "HTTP/1.1 304 Not Modified"
2026-09-28 22:30:18,195 - httpx - INFO - HTTP Request: GET https://api.github.com/repos/a0556770172-png/koshermat/events?per_page=10 "HTTP/1.1 304 Not Modified"
2026-09-28 22:30:18,206 - httpx - INFO - HTTP Request: GET https://api.github.com/repos/bnq9705-creator/siddur/events?per_page=10 "HTTP/1.1 304 Not Modified"
2026-09-28 22:30:18,439 - httpx - INFO - HTTP Request: GET https://api.github.com/repos/yosefmalul52-lang/costa-rica-kosher-retreat/events?per_page=10 "HTTP/1.1 304 Not Modified"
2026-09-28 22:30:18,480 - httpx - INFO - HTTP Request: GET https://api.github.com/repos/Kosher-T/fannexus/events?per_page=10 "HTTP/1.1 304 Not Modified"
2026-09-28 22:30:18,514 - httpx - INFO - HTTP Request: GET https://api.github.com/repos/ReuvenB-S/kosher-browser-data/events?per_page=10 "HTTP/1.1 304 Not Modified"
2026-09-28 22:30:18,521 - httpx - INFO - HTTP Request: GET https://api.github.com/repos/RavBogard/shirei-tfilah/events?per_page=10 "HTTP/1.1 304 Not Modified"
2026-09-28 22:30:18,715 - httpx - INFO - HTTP Request: GET https://api.github.com/repos/shliachflow/chabad-southern-delaware-camp/events?per_page=10 "HTTP/1.1 304 Not Modified"
2026-09-28 22:30:18,810 - httpx - INFO - HTTP Request: GET https://api.github.com/repos/TanachatGKRB/TanachatGKRB.github.io/events?per_page=10 "HTTP/1.1 304 Not Modified"
2026-09-28 22:30:18,849 - httpx - INFO - HTTP Request: GET https://api.github.com/repos/AviMish10/tanach-map/events?per_page=10 "HTTP/1.1 304 Not Modified"
2026-09-28 22:30:18,880 - httpx - INFO - HTTP Request: GET https://api.github.com/repos/TamamTamar/chabad-yafo/events?per_page=10 "HTTP/1.1 304 Not Modified"
2026-09-28 22:30:19,045 - httpx - INFO - HTTP Request: GET https://api.github.com/repos/DeepSynchronicity/chabadofhurlingham/events?per_page=10 "HTTP/1.1 304 Not Modified"
2026-09-28 22:30:19,155 - httpx - INFO - HTTP Request: GET https://api.github.com/repos/TG-CONSULTANT/tanachok-stock-FG1/events?per_page=10 "HTTP/1.1 304 Not Modified"
2026-09-28 22:30:19,162 - httpx - INFO - HTTP Request: GET https://api.github.com/repos/semiagenticRob/digital-siddur/events?per_page=10 "HTTP/1.1 304 Not Modified"
2026-09-28 22:30:19,221 - httpx - INFO - HTTP Request: GET https://api.github.com/repos/orelzion/daily-tanach/events?per_page=10 "HTTP/1.1 304 Not Modified"
2026-09-28 22:30:19,381 - httpx - INFO - HTTP Request: GET https://api.github.com/repos/TG-CONSULTANT/tanachok-stock-mat/events?per_page=10 "HTTP/1.1 304 Not Modified"
2026-09-28 22:30:19,528 - httpx - INFO - HTTP Request: GET https://api.github.com/repos/satmarthubby-blip/Satmarthubby/events?per_page=10 "HTTP/1.1 304 Not Modified"
2026-09-28 22:30:19,593 - httpx - INFO - HTTP Request: GET https://api.github.com/repos/akshatasatmar/WeatherServiceApi/events?per_page=10 "HTTP/1.1 304 Not Modified"
2026-09-28 22:30:19,603 - httpx - INFO - HTTP Request: GET https://api.github.com/repos/Frish-Gebakn/kol-satmar-website/events?per_page=10 "HTTP/1.1 304 Not Modified"
2026-09-28 22:30:19,677 - httpx - INFO - HTTP Request: GET https://api.github.com/repos/elchunenfreund/satmar-matzos-app/events?per_page=10 "HTTP/1.1 304 Not Modified"
2026-09-28 22:30:19,875 - httpx - INFO - HTTP Request: GET https://api.github.com/repos/satmaribianca05-eng/AFSD-Satmari-Bianca/events?per_page=10 "HTTP/1.1 304 Not Modified"
2026-09-28 22:30:20,011 - httpx - INFO - HTTP Request: GET https://api.github.com/repos/mritunjaysharma2004/SATMARG-An-Online-Pilgrimage-planner-/events?per_page=10 "HTTP/1.1 304 Not Modified"
2026-09-28 22:30:20,091 - httpx - INFO - HTTP Request: GET https://api.github.com/repos/Sefaria/Sefaria-Export/events?per_page=10 "HTTP/1.1 304 Not Modified"
2026-09-28 22:30:20,154 - httpx - INFO - HTTP Request: GET https://api.github.com/repos/hebcal/hebcal-es6/events?per_page=10 "HTTP/1.1 304 Not Modified"
2026-09-28 22:30:20,188 - GitHubFeedScraper - INFO - GitHub feed healthy: login=abeperl fetched=300 account_new=0 list_repos=100 list_new=0 private_new=0 total_new=0
2026-09-28 22:30:20,189 - __main__ - INFO - Scraper github completed: 1 results
2026-09-28 22:30:20,189 - __main__ - INFO - Scraper github_feed completed: 0 results
2026-09-28 22:30:20,189 - __main__ - INFO - Scraper youtube_intel completed: 5 results
2026-09-28 22:30:20,785 - social_media_monitor.email.sender - INFO - Email sent successfully at 2026-09-28 22:30:20.785902
2026-09-28 22:30:20,786 - __main__ - INFO - Email sent successfully
2026-09-28 22:30:20,786 - __main__ - INFO - Cache stats: 0 items in memory, 0 files on disk
2026-09-28 22:30:20,786 - __main__ - INFO - Monitor complete. Found 6 total results: 1 new github results, 5 new youtube intel results.
2026-09-28 22:30:20,786 - __main__ - INFO - === Social Media Monitor v2.0 Complete ===
2026-09-28 22:30:20,786 - social_media_monitor.database.connection - INFO - Closing all database connections
2026-09-28 22:30:20,807 - social_media_monitor.database.connection - INFO - Closed 5 database connections
2026-09-28 22:30:20,807 - __main__ - INFO - Closed all database connections
2026-09-28 22:30:20,809 - social_media_monitor.database.connection - INFO - Closing all database connections
2026-09-28 22:30:20,809 - social_media_monitor.database.connection - INFO - Closed 0 database connections
2026-09-28 23:00:01,706 - social_media_monitor.utils.logging_config - INFO - Posts log file: /mnt/data/openclaw-shared/home/shared/Social_Media_Monitor/logs/social_media_monitor_posts_2026-09-28.log
2026-09-28 23:00:01,706 - social_media_monitor.utils.logging_config - INFO - Downloads log file: /mnt/data/openclaw-shared/home/shared/Social_Media_Monitor/logs/social_media_monitor_downloads_2026-09-28.log
2026-09-28 23:00:01,706 - social_media_monitor.utils.logging_config - INFO - Logging configured with level INFO, log file: /mnt/data/openclaw-shared/home/shared/Social_Media_Monitor/logs/social_media_monitor_2026-09-28.log
2026-09-28 23:00:01,706 - __main__ - INFO - === Social Media Monitor v2.0 (Async) Started ===
2026-09-28 23:00:01,735 - social_media_monitor.config.settings - INFO - Configuration loaded from /home/openclaw/Social_Media_Monitor/config.yaml
2026-09-28 23:00:01,735 - __main__ - INFO - Configuration loaded and validated successfully
2026-09-28 23:00:01,738 - social_media_monitor.database.connection - INFO - Database connection pool initialized with 5 connections
2026-09-28 23:00:01,738 - social_media_monitor.database.connection - INFO - add_speaker_filters column already exists
2026-09-28 23:00:01,768 - social_media_monitor.database.connection - INFO - Database setup completed successfully
2026-09-28 23:00:01,768 - __main__ - INFO - Database initialized at: /mnt/data/openclaw-shared/home/shared/Social_Media_Monitor/social_media_monitor.db
2026-09-28 23:00:01,772 - social_media_monitor.scrapers.reddit_scraper - INFO - Reddit API initialized successfully (UA='SocialMediaMonitor/1.0')
2026-09-28 23:00:01,774 - RSSFeedScraper - INFO - RSS scraper initialized: feeds=1, html_urls=0 (0 with custom selectors), include_terms=0, exclude_terms=0
2026-09-28 23:00:01,774 - __main__ - INFO - RSS scraper initialized and enabled
2026-09-28 23:00:01,780 - Yiddish24Scraper - INFO - File downloader initialized with 55 terms, prefix 'Y2', and file type organization enabled
2026-09-28 23:00:01,780 - Yiddish24Scraper - INFO - Yiddish24 scraper initialized: data_id='latest', page_limit=100, total_pages=10, debug_html=True, include_terms=55, exclude_terms=0
2026-09-28 23:00:01,780 - __main__ - INFO - Yiddish24 scraper initialized and enabled
2026-09-28 23:00:01,786 - AhBlickLiveScraper - INFO - AhBlickLive scraper initialized: urls=1, max_items=100, include_terms=49, exclude_terms=0
2026-09-28 23:00:01,793 - TorahAnytimeScraper - INFO - File downloader initialized with 53 terms, prefix 'TA', and file type organization enabled
2026-09-28 23:00:01,794 - TorahAnytimeScraper - INFO - TorahAnytime scraper initialized: limit=400, offset=0, project_id=1, include_terms=53, exclude_terms=13
2026-09-28 23:00:01,796 - BatorahScraper - INFO - File downloader initialized with 19 terms, prefix 'BT', and file type organization enabled
2026-09-28 23:00:01,796 - BatorahScraper - INFO - Batorah scraper initialized: api_urls=2, page_size=100, include_terms=19, exclude_terms=2
2026-09-28 23:00:01,796 - KolHalashonScraper - INFO - Kol Halashon scraper initialized: base_url=https://www2.kolhalashon.com, max_shiurim=50, fetch_all_speakers=True, headless=True, exclude_filters=1, include_filters=0
2026-09-28 23:00:01,798 - AllDafScraper - INFO - All Daf scraper initialized: enabled=True sections=['Daf Yomi Shiurim', 'Supplemental Clips', 'Reference Materials'] skip_terms=0 include_terms=0
2026-09-28 23:00:01,798 - __main__ - INFO - Ivelt scraper initialized and enabled
2026-09-28 23:00:01,798 - __main__ - INFO - Running 3 scrapers concurrently...
2026-09-28 23:00:01,799 - Yiddish24Scraper - INFO - Starting Yiddish24 scraping (max 10 pages, limit 100 items)
2026-09-28 23:00:01,888 - Yiddish24Scraper - INFO - Fetching Yiddish24 page 1 with data_id='latest', page_limit=100
2026-09-28 23:00:01,892 - IveltScraper - INFO - Loaded 182 include, 4 exclude filters from DB topic_filters
2026-09-28 23:00:01,892 - IveltScraper - INFO - Fetching ivelt active topics from https://www.ivelt.com/forum/search.php?search_id=active_topics...
2026-09-28 23:00:01,893 - RSSFeedScraper - INFO - Fetching RSS feed: https://www.theyeshivaworld.com/feed
2026-09-28 23:00:02,440 - RSSFeedScraper - INFO - Total RSS/HTML results: 0
2026-09-28 23:00:03,765 - httpx - INFO - HTTP Request: POST https://www.yiddish24.com/ajax/cat_pagination.php "HTTP/1.1 200 OK"
2026-09-28 23:00:03,979 - Yiddish24Scraper - INFO - Received response with 504669 characters
2026-09-28 23:00:03,980 - Yiddish24Scraper - INFO - Extracted HTML from JSON result field (312278 chars)
2026-09-28 23:00:03,982 - Yiddish24Scraper - INFO - Saved HTML debug file to: /mnt/data/openclaw-shared/home/shared/Social_Media_Monitor/debug/yiddish24_page1_response.html
2026-09-28 23:00:04,051 - Yiddish24Scraper - INFO - Found 100 potential article elements on page 1
2026-09-28 23:00:04,096 - Yiddish24Scraper - INFO - Page 1 extraction: 99 successful, 1 failed out of 100 elements
2026-09-28 23:00:04,096 - Yiddish24Scraper - INFO - Parsed 99 articles from HTML on page 1
2026-09-28 23:00:04,921 - Yiddish24Scraper - INFO - Page 1 summary: 3 new out of 99 total articles
2026-09-28 23:00:04,921 - Yiddish24Scraper - INFO - Page 1: 3/99 new items
2026-09-28 23:00:04,921 - Yiddish24Scraper - INFO - Low new item ratio (3.0%), stopping pagination
2026-09-28 23:00:04,922 - Yiddish24Scraper - INFO - Total Yiddish24 results found: 3
2026-09-28 23:00:16,130 - IveltScraper - INFO - Found 50 active topics on ivelt
2026-09-28 23:00:24,951 - IveltScraper - INFO - Filtering post containing exclude term: '*ื‘ื™ื˜ืข ืžืื›ื˜ ื–ื™ื›ืขืจ ื‘ืœื•ื™ื– ืฆื• ื‘ืืจื™ื›ื˜ืŸ ืงืจืื ื˜ืข ื ื™ื™ืขืก*'
2026-09-28 23:00:29,106 - IveltScraper - INFO - Filtering post containing exclude term: '*ื‘ื™ื˜ืข ืžืื›ื˜ ื–ื™ื›ืขืจ ื‘ืœื•ื™ื– ืฆื• ื‘ืืจื™ื›ื˜ืŸ ืงืจืื ื˜ืข ื ื™ื™ืขืก*'
2026-09-28 23:00:48,507 - IveltScraper - INFO - ivelt scrape complete. Checked 6 topics.
2026-09-28 23:00:48,508 - IveltScraper - INFO - All forums scrape complete. Found 23 new posts.
2026-09-28 23:00:48,508 - __main__ - INFO - Scraper rss completed: 0 results
2026-09-28 23:00:48,509 - __main__ - INFO - Scraper yiddish24 completed: 3 results
2026-09-28 23:00:48,509 - __main__ - INFO - Scraper ivelt completed: 23 results
2026-09-28 23:00:49,271 - social_media_monitor.email.sender - INFO - Email sent successfully at 2026-09-28 23:00:49.271006
2026-09-28 23:00:49,271 - __main__ - INFO - Email sent successfully
2026-09-28 23:00:49,271 - __main__ - INFO - Cache stats: 0 items in memory, 0 files on disk
2026-09-28 23:00:49,271 - __main__ - INFO - Monitor complete. Found 26 total results: 3 new yiddish24 results, 23 new ivelt results.
2026-09-28 23:00:49,271 - __main__ - INFO - === Social Media Monitor v2.0 Complete ===
2026-09-28 23:00:49,271 - social_media_monitor.database.connection - INFO - Closing all database connections
2026-09-28 23:00:49,387 - social_media_monitor.database.connection - INFO - Closed 5 database connections
2026-09-28 23:00:49,387 - __main__ - INFO - Closed all database connections
2026-09-28 23:00:49,390 - social_media_monitor.database.connection - INFO - Closing all database connections
2026-09-28 23:00:49,390 - social_media_monitor.database.connection - INFO - Closed 0 database connections