bestofgithub.dev

← Back to the directory

lorien/awesome-web-scraping

View on GitHub

List of libraries, tools and APIs for web scraping and data processing.

lorien/awesome-web-scraping is #6,029 on the bestofgithub Elo ladder, in Silver III, with a rating of 1,472 (+1 this week). It has 8,230 stars, 9 gained in the last week.

Silver III

1,472 Elo

  • #6,029 on the ladder
  • −8 today · +1 this week
  • Peak 1,481
10 Elo to Silver II

Stars

8,230

  • +0 today
  • +9 this week
  • – this month

Repository

Forks
977
Language
JavaScript
Last push

Achievements

bestofgithub badgeMaintainer? Show your league in your README

Card: your league's emblem, from Copper to Legendary

[![bestofgithub](https://bestofgithub.dev/badge/lorien/awesome-web-scraping.svg?style=card)](https://bestofgithub.dev/repo/lorien/awesome-web-scraping)

Flat: fits beside your other badges

[![bestofgithub](https://bestofgithub.dev/badge/lorien/awesome-web-scraping.svg)](https://bestofgithub.dev/repo/lorien/awesome-web-scraping)

Install

git
git clone https://github.com/lorien/awesome-web-scraping.git

Similar repositories

gocolly/colly

Open on GitHubGo

Elegant Scraper and Crawler Framework for Golang

  • 1,658+9 Elo, change today
  • 25.5k stars
  • 1.8k forks
git
git clone https://github.com/gocolly/colly.git
  • crawler
  • crawling
  • framework
  • go

scrapy/scrapy

Open on GitHubPython

Scrapy, a fast high-level web crawling & scraping framework for Python.

  • 2,020+7 Elo, change today
  • 65.4k stars
  • 12.1k forks
  • 1.7k watchers
brew install scrapy
2.20.09d ago
  • crawler
  • crawling
  • framework
  • hacktoberfest

D4Vinci/Scrapling

Open on GitHubPython

🕷️ The BEST Web Scraping framework that handles everything from a single request to a full-scale crawl! Don't be shy, join here: https://discord.gg/EMgGbDceNQ and follow here for daily tips and tricks: https://x.com/Scrapling_dev

  • 2,448+2 Elo, change today
  • +392 stars gained
  • 103k stars
  • 10.7k forks
  • 350 watchers
pip
pipx install scrapling
v0.5.111d ago
  • ai
  • ai-scraping
  • automation
  • crawler

code4craft/webmagic

Open on GitHubJava

A scalable web crawler framework for Java.

  • 1,483−6 Elo, change today
  • 11.6k stars
  • 4k forks
git
git clone https://github.com/code4craft/webmagic.git
  • crawler
  • framework
  • java
  • scraping

alirezamika/autoscraper

Open on GitHubPython

A Smart, Automatic, Fast and Lightweight Web Scraper for Python

  • 1,466+12 Elo, change today
  • 8k stars
  • 834 forks
git
git clone https://github.com/alirezamika/autoscraper.git
  • ai
  • artificial-intelligence
  • automation
  • crawler

README

Loading README…