jhao104/proxy_pool

• Gold III

Incredibly fast crawler designed for OSINT.

• 1,573−2 Elo, change today
• 13.3k stars
• 1.6k forks
• 29d ago

git clone https://github.com/s0md3v/Photon.git

• crawler
• information-gathering
• osint
• python

• Gold IV

A Powerful Spider(Web Crawler) System in Python.

• 1,538−8 Elo, change today
• 16.7k stars
• 3.6k forks
• 2y ago
• archived

git clone https://github.com/binux/pyspider.git

• crawler
• python

• Platinum IV

Elegant Scraper and Crawler Framework for Golang

• 1,658+9 Elo, change today
• 25.5k stars
• 1.8k forks
• 1mo ago

git clone https://github.com/gocolly/colly.git

• crawler
• crawling
• framework
• go

• Bronze II

Go client for Redis

• 1,413−3 Elo, change today
• 9.8k stars
• 1.2k forks
• 1y ago

git clone https://github.com/gomodule/redigo.git

• go
• redis

• Copper II

Web Crawler/Spider for NodeJS + server-side jQuery ;-)

• 1,339−2 Elo, change today
• 6.7k stars
• 863 forks
• 5mo ago

git clone https://github.com/bda-research/node-crawler.git

• cheerio
• crawler
• extract-data
• javascript

• Bronze IV

A collection of awesome web crawler,spider in different languages

• 1,384−2 Elo, change today
• 7.3k stars
• 751 forks
• 2y ago

git clone https://github.com/BruceDone/awesome-crawler.git

• awesome
• crawler
• node-crawler
• scraper