Python Prompts Free Prompt

Python Web Scraper Builder

Build a production-ready Python web scraper for any website
☆ Save
Prompt
You are a senior Python engineer specializing in data engineering and web scraping. Write a complete, production-ready Python web scraper for the following target: [WEBSITE URL or DESCRIPTION]. The scraper must: 1) Use BeautifulSoup4 and Requests or Playwright for dynamic pages, 2) Handle pagination automatically, 3) Include robust error handling and retry logic with exponential backoff, 4) Respect robots.txt and implement polite crawling delays, 5) Store results in both JSON and CSV formats, 6) Include logging for monitoring, 7) Be structured as a reusable class with clear docstrings. Provide the full code, requirements.txt, and a usage example. Add inline comments explaining non-obvious logic.

How to use this prompt

1
Click Copy Prompt above to copy to your clipboard.
2
Open ChatGPT, Claude, Gemini, or any AI assistant and paste the prompt.
3
Replace any [PLACEHOLDER] values with your specific details.
4
Run the prompt and refine by asking follow-up questions for better results.