Top Spark repositories

Repositories with the GitHub topic “spark”: 20 repositories with 350k stars between them, mostly written in Java, Python and Jupyter Notebook. The highest-rated is DataTalksClub/data-engineering-zoomcamp, #475 on the bestofgithub Elo ladder.

Biggest movers this week

tobymao/sqlglot

Open on GitHubPython

Python SQL Parser and Transpiler

  • 1,582−8 Elo, change today
  • 9.7k stars
  • 1.4k forks
git
git clone https://github.com/tobymao/sqlglot.git
  • bigquery
  • clickhouse
  • databricks
  • duckdb

delta-io/delta

Open on GitHubScala

An open-source storage framework that enables building a Lakehouse architecture with compute engines including Spark, PrestoDB, Flink, Trino, and Hive and APIs

  • 1,522+13 Elo, change today
  • 9k stars
  • 2.2k forks
git
git clone https://github.com/delta-io/delta.git
  • acid
  • analytics
  • big-data
  • delta-lake

h2oai/h2o-3

Open on GitHubJupyter Notebook

H2O is an Open Source, Distributed, Fast & Scalable Machine Learning Platform: Deep Learning, Gradient Boosting (GBM) & XGBoost, Random Forest, Generalized Linear Modeling (GLM with Elastic Net), K-Means, PCA, Generalized Additive Models (GAM), RuleFit, Support Vector Machine (SVM), Stacked Ensembles, Automatic Machine Learning (AutoML), etc.

  • 1,432+9 Elo, change today
  • 7.5k stars
  • 2k forks
git
git clone https://github.com/h2oai/h2o-3.git
  • automl
  • big-data
  • data-science
  • deep-learning

wangzhiwubigdata/God-Of-BigData

Open on GitHub

专注大数据学习面试,大数据成神之路开启。Flink/Spark/Hadoop/Hbase/Hive...

  • 1,500−4 Elo, change today
  • 10.5k stars
  • 3.2k forks
git
git clone https://github.com/wangzhiwubigdata/God-Of-BigData.git
  • azkaban
  • bigdata
  • flink
  • flume

Top 20 on the Elo ladder

  1. #475
    DataTalksClub/data-engineering-zoomcamp

    Data Engineering Zoomcamp is a free 9-week course on building production-ready data pipelines. Join the course here 👇🏼

    1,984 Elo −546.9kJupyter Notebook
  2. #662
    apache/spark

    Apache Spark - A unified analytics engine for large-scale data processing

    1,925 Elo 44.5kScala
  3. #1,980
    getredash/redash

    Make Your Company Data Driven. Connect to any data source, easily visualize, dashboard and share your data.

    1,730 Elo −6228.8kTypeScript
  4. #2,268
    yeasy/docker_practice

    最新Docker容器技术,从真实案例中学习最佳实践!| Learn and understand Docker&Container technologies, with real DevOps practice!

    1,699 Elo −7226.3kGo
  5. #2,329
    donnemartin/data-science-ipython-notebooks

    Data science Python notebooks: Deep learning (TensorFlow, Theano, Caffe, Keras), scikit-learn, Kaggle, big data (Spark…

    1,693 Elo −5529.3kPython
  6. #3,455
    heibaiying/BigData-Notes

    大数据入门指南 :star:

    1,608 Elo −1717kJava
  7. #3,851
    tobymao/sqlglot

    Python SQL Parser and Transpiler

    1,582 Elo +399.7kPython
  8. #4,134
    FavioVazquez/ds-cheatsheets

    List of Data Science Cheatsheets to rule the world

    1,566 Elo −2616.3k
  9. #4,476
    zhisheng17/flink-learning

    flink learning blog. http://www.54tianzhisheng.cn/ 含 Flink 入门、概念、原理、实战、性能调优、源码解析等内容。涉及 Flink…

    1,546 Elo −3015.1kJava
  10. #4,518
    aalansehaiyang/technology-talk

    【大厂面试专栏】一份Java程序员需要的技术指南,这里有面试题、系统架构、职场锦囊、主流中间件等,让你成为更牛的自己!

    1,544 Elo −2714.7k
  11. #4,658
    deeplearning4j/deeplearning4j

    Suite of tools for deploying and training deep learning models using the JVM. Highlights include model import for…

    1,536 Elo −2314.2kJava
  12. #4,951
    delta-io/delta

    An open-source storage framework that enables building a Lakehouse architecture with compute engines including Spark…

    1,522 Elo +299kScala
  13. #5,424
    wangzhiwubigdata/God-Of-BigData

    专注大数据学习面试,大数据成神之路开启。Flink/Spark/Hadoop/Hbase/Hive...

    1,500 Elo +110.5k
  14. #5,445
    horovod/horovod

    Distributed training framework for TensorFlow, Keras, PyTorch, and Apache MXNet.

    1,499 Elo −4114.6kPython
  15. #5,617
    GaiZhenbiao/ChuanhuChatGPT

    GUI for ChatGPT API and many LLMs. Supports agents, file-based QA, GPT finetuning and query with web search. All with a…

    1,492 Elo −3915.2kPython
  16. #7,032
    h2oai/h2o-3

    H2O is an Open Source, Distributed, Fast & Scalable Machine Learning Platform: Deep Learning, Gradient Boosting (GBM) &…

    1,432 Elo +197.5kJupyter Notebook
  17. #7,264
    Alluxio/alluxio

    Alluxio, data orchestration for analytics and machine learning in the cloud

    1,423 Elo −177.2kJava
  18. #7,268
    mage-ai/mage-ai

    🧙 Build, run, and manage data pipelines for integrating and transforming data.

    1,423 Elo −118.8kPython
  19. #7,822
    apache/zeppelin

    Web-based notebook that enables data-driven, interactive data analytics and collaborative documents with SQL, Scala and…

    1,402 Elo −66.6kJava
  20. #8,797
    Angel-ML/angel

    A Flexible and Powerful Parameter Server for large-scale machine learning

    1,361 Elo −106.7kJava