Dr-Scraper is a Python-based tool for scraping and checking HTTP/HTTPS proxies. It collects proxy IP addresses from different sources and checks them against a target website to find working proxies. Dr-Scraper uses multiple threads, random User-Agents, and browser-like headers while checking proxies.
Installation Commands
Use the following commands to install Dr-Scraper in Termux. Run them one by one to download the tool and prepare it for use.
Update Termux packages.
pkg update && pkg upgrade -y
Install Python and Git.
pkg install python git -y
Clone the Dr-Scraper repository.
git clone https://github.com/drhakoo-beep/Dr-Scraper
Go to the Dr-Scraper directory.
cd Dr-Scraper
Install the required Python packages.
pip install requests beautifulsoup4
Dr-Scraper is now installed and ready to use.
Usage Guide
Run Dr-Scraper.
python drscraper.py
After starting the tool, it will ask you to enter a target domain.
Enter the target domain when prompted, for example:
example.comYou do not need to add https:// or www. because the tool formats the target automatically.
The tool will then ask for an output file name. You can enter your own file name or press Enter to use the default:
proxy.txtDr-Scraper will scrape proxy IP addresses and test them against the target website. Working proxies are shown as [ SUCCESS ] and are automatically saved to the output file.
The tool also checks the existing output file to avoid saving duplicate proxy IP addresses.
How Dr-Scraper Works
Dr-Scraper uses several features while checking proxies:
- Target Testing: Tests proxies directly against the target website.
- Anti-Bot Detection: Checks for CAPTCHA, Cloudflare blocks, and other blocking pages.
- Browser Simulation: Uses random User-Agents and browser-like headers.
- Duplicate Prevention: Checks the output file before saving new proxies.
- Multi-Threading: Tests multiple proxies at the same time.
Overview
Dr-Scraper is a simple Python tool for collecting and checking HTTP/HTTPS proxies from the terminal. It provides useful features such as target testing, and multi-threaded proxy verification for learning about web scraping and proxy testing.
