bestofgithub.dev

← Back to the directory

code4craft/webmagic

View on GitHub

A scalable web crawler framework for Java.

code4craft/webmagic is #5,789 on the bestofgithub Elo ladder, in Silver II, with a rating of 1,483 (−47 this week). It has 11,663 stars.

Silver II

1,483 Elo

  • #5,789 on the ladder
  • −6 today · −47 this week
  • Peak 1,613
18 Elo to Silver I

Stars

11,663

  • +0 today
  • +0 this week
  • – this month

Achievements

bestofgithub badgeMaintainer? Show your league in your README

Card: your league's emblem, from Copper to Legendary

[![bestofgithub](https://bestofgithub.dev/badge/code4craft/webmagic.svg?style=card)](https://bestofgithub.dev/repo/code4craft/webmagic)

Flat: fits beside your other badges

[![bestofgithub](https://bestofgithub.dev/badge/code4craft/webmagic.svg)](https://bestofgithub.dev/repo/code4craft/webmagic)

Install

git
git clone https://github.com/code4craft/webmagic.git

Similar repositories

gocolly/colly

Open on GitHubGo

Elegant Scraper and Crawler Framework for Golang

  • 1,658+9 Elo, change today
  • 25.5k stars
  • 1.8k forks
git
git clone https://github.com/gocolly/colly.git
  • crawler
  • crawling
  • framework
  • go

scrapy/scrapy

Open on GitHubPython

Scrapy, a fast high-level web crawling & scraping framework for Python.

  • 2,020+7 Elo, change today
  • 65.4k stars
  • 12.1k forks
  • 1.7k watchers
brew install scrapy
2.20.09d ago
  • crawler
  • crawling
  • framework
  • hacktoberfest

binux/pyspider

Open on GitHubPython

A Powerful Spider(Web Crawler) System in Python.

  • 1,538−8 Elo, change today
  • 16.7k stars
  • 3.6k forks
  • archived
git
git clone https://github.com/binux/pyspider.git
  • crawler
  • python

alirezamika/autoscraper

Open on GitHubPython

A Smart, Automatic, Fast and Lightweight Web Scraper for Python

  • 1,466+12 Elo, change today
  • 8k stars
  • 834 forks
git
git clone https://github.com/alirezamika/autoscraper.git
  • ai
  • artificial-intelligence
  • automation
  • crawler

injetlee/Python

Open on GitHubPython

Python脚本。模拟登录知乎， 爬虫，操作excel，微信公众号，远程开机

  • 1,550−8 Elo, change today
  • 10.9k stars
  • 4.2k forks
git
git clone https://github.com/injetlee/Python.git
  • crawler
  • excel
  • python
  • wechat

apify/crawlee

Open on GitHubTypeScript

Crawlee—A web scraping and browser automation library for Node.js to build reliable crawlers. In JavaScript and TypeScript. Extract data for AI, LLMs, RAG, or GPTs. Download HTML, PDF, JPG, PNG, and other files from websites. Works with Puppeteer, Playwright, Cheerio, JSDOM, and raw HTTP. Both headful and headless mode. With proxy rotation.

  • 1,821−2 Elo, change today
  • 26.4k stars
  • 1.7k forks
  • 135 watchers
git
git clone https://github.com/apify/crawlee.git
v3.18.21mo ago
  • apify
  • automation
  • crawler
  • crawling

README

Loading README…