GoSpider is a simple web crawling tool that helps find URLs, JavaScript files, subdomains, and other resources from websites. You can install GoSpider in Termux to perform website reconnaissance and information gathering tasks using simple commands. It is useful for learning how websites are structured and discovering publicly accessible content.
GoSpider Install Commands
You can install GoSpider in Termux with a few simple commands. Run each command one by one to install and setup GoSpider on your Termux.
Update Termux packages.
pkg update && pkg upgrade -y
Install Git and Golang.
pkg install git golang -y
Install GoSpider.
go install github.com/jaeles-project/gospider@latest
Add the Go binary directory to PATH.
export PATH=$PATH:$HOME/go/bin
Check the installation.
gospider -h
GoSpider is now installed and ready to use.
GoSpider Usage Commands
Display the help menu and available options.
gospider -h
Crawl a website.
gospider -s https://example.com
Crawl multiple websites from a list.
gospider -S sites.txt
Set crawl depth.
gospider -s https://example.com -d 3
Include subdomains during crawling.
gospider -s https://example.com --subs
Find URLs from third-party sources such as Archive.org and Common Crawl.
gospider -s https://example.com -a
Include URLs discovered from third-party sources in the crawl.
gospider -s https://example.com -a -r
Extract links from JavaScript files.
gospider -s https://example.com --js
Try to crawl the sitemap file.
gospider -s https://example.com --sitemap
Try to crawl the robots.txt file.
gospider -s https://example.com --robots
Save results into an output folder.
gospider -s https://example.com -o output
Set the number of concurrent requests.
gospider -s https://example.com -c 10
Set the number of threads.
gospider -s https://example.com -t 5
Use a mobile user agent.
gospider -s https://example.com -u mobi
Display only discovered URLs.
gospider -s https://example.com -q
Enable JSON output.
gospider -s https://example.com --json
Show verbose output.
gospider -s https://example.com -v
Choose the command you want and GoSpider will automatically crawl the target website and display the discovered URLs and resources in Termux.
