Settings

Rate Limit (requests/second)

Requests per second to avoid being blocked

Concurrent Requests

Maximum simultaneous requests

Max Retries

Number of retry attempts for failed requests

Retry Delay (seconds)

Delay between retry attempts

Max Pages (Discovery)

Maximum pages to crawl during discovery

Max Pages (Sync)

Maximum pages to crawl during sync

User Agent

User agent string for HTTP requests

Respect robots.txt

Enable page caching

Skip duplicate URLs

Select Categories to Scrape (0 selected)

Hair Salons

Spa

Massage

Beauty

Wellness

Clinic

Select specific categories to focus discovery on. Leave all unchecked to discover all categories.

Discovery Schedule

How often to run discovery jobs

Sync Schedule

How often to run sync jobs

Enable scheduler

Prevent overlapping jobs

Keep Changes (days)

Days to keep change records

Keep Scrape Runs (days)

Days to keep scrape run records

Enable automatic cleanup