We are looking for an experienced Python web scraping expert who can extract structured data from websites and deliver it in JSON format or through a custom API.
The data should be clean, well-organized, and ready for use in applications or databases.
Key Responsibilities:
Scrape data from static and dynamic websites
Handle pagination, AJAX-loaded content, and login-protected pages (if needed)
Format extracted data into clean and structured JSON
Optionally expose the data via REST API (Flask/FastAPI)
Ensure the scraper is efficient, robust, and handles exceptions/errors
Maintain and update scrapers if website structures change
Required Skills:
Strong experience with Python and libraries like BeautifulSoup, Scrapy, or Selenium
Experience using requests, or aio to deal with JavaScript-heavy sites using Selenium or Playwright
Data formatting skills (JSON, CSV, etc.)
Optional but preferred: experience with Flask or FastAPI
Knowledge of proxies, rate-limiting, and CAPTCHA handling
Clear and well-documented code practices
Attention to detail and ability to work independently
Deliverables:
Python code for scraping
JSON files or working API endpoint
Brief documentation or instructions to run the script/API
Bonus Points:
Ability to schedule scraping jobs (e.g., using cron or Celery)
Experience with storing data in a database (MongoDB, PostgreSQL, etc.)
Ability to containerize with Docker (optional)
Job Specification :