← Back to scrapers

Configuration

config.yaml — editable key-value pairs

Quick Settings

reddit.enabled
reddit.search_enabled
reddit.monitor_multis
reddit.monitor_feeds
reddit.monitor_top
reddit.top_post_limit
25
reddit.run_id
5
reddit.save_to_sheets
reddit.feed_post_limit
25
github.enabled
github.run_id
5
github.save_to_sheets
github_feed.enabled
github_feed.run_id
5
github_feed.per_page
100
github_feed.max_pages
3
github_feed.lookback_hours
24
github_feed.list_max_repos
100
github_feed.list_events_per_repo
10
github_feed.list_concurrency
4
github_feed.timeout_seconds
300
twitter.enabled
twitter.run_id
5
twitter_run_id
5
medium
medium_run_id
3
exa.enabled
exa.run_id
5
exa.num_results
20
exa.use_autoprompt
youtube_sheet.enabled
youtube_sheet.run_id
5
youtube_sheet.max_downloads
10
youtube_sheet.sleep_between
5
youtube_intel.enabled
youtube_intel.run_id
5
youtube_intel.feed_limit
20
youtube_intel.results_per_search
5
serpapi.enabled
serpapi.run_id
3
serpapi.num_results
20
serpapi.use_autoprompt
linkedin.enabled
linkedin.run_id
5
linkedin.results_per_page
20
linkedin_feed.enabled
linkedin_feed.run_id
5
linkedin_feed.feed_count
25
linkedin_notifications.enabled
linkedin_notifications.run_id
9
linkedin_notifications.count
15
ahblicklive.enabled
ahblicklive.run_id
7
ahblicklive.max_items
100
yiddish24.enabled
yiddish24.debug_html
yiddish24.run_id
3
yiddish24.page_limit
100
yiddish24.total_pages
10
gmail_downloads.enabled
gmail_downloads.run_id
6
gmail_downloads.send_notification
gmail_downloads.imap_port
993
gmail_downloads.use_ssl
gmail_downloads.organize_by_type
gmail_downloads.timeout_seconds
300
gmail_downloads.file_type_detection
gmail_downloads.parse_html_body
gmail_downloads.parse_text_body
gmail_downloads.duplicate_prevention
batorah.enabled
batorah.run_id
6
batorah.limit
100
torahanytime.enabled
torahanytime.run_id
6
torahanytime.limit
400
torahanytime.offset
0
torahanytime.project_id
1
yiddishvideos.enabled
yiddishvideos.run_id
10
yiddishvideos.max_items
50
yiddishvideos.timeout_seconds
300
ivelt.enabled
ivelt.run_id
3
ivelt.max_topics
30
kolhalashon.enabled
kolhalashon.run_id
8
kolhalashon.timeout_seconds
900
kolhalashon.headless
kolhalashon.max_shiurim
50
kolhalashon.language_filter
None
kolhalashon.page_load_wait
230
kolhalashon.flaresolverr_enabled
kolhalashon.download_budget_seconds
600
kolhalashon.search_urls
None
kolhalashon.max_search_pages
10
alldaf.enabled
alldaf.run_id
8
alldaf.max_downloads_per_run
20
alldaf.timeout_seconds
900

Scraper Status Overview

reddit
github
github_feed
twitter
medium
exa
youtube_sheet
youtube_intel
serpapi
linkedin
linkedin_feed
linkedin_notifications
ahblicklive
yiddish24
gmail_downloads
batorah
torahanytime
yiddishvideos
ivelt
kolhalashon
alldaf