Top Data Engineering repositories

Biggest movers this week

pipx install semantica

git clone https://github.com/cocoindex-io/cocoindex.git

git clone https://github.com/dagster-io/dagster.git

git clone https://github.com/growthbook/growthbook.git

git clone https://github.com/xonsh/xonsh.git

git clone https://github.com/evidence-dev/evidence.git

Top 25 on the Elo ladder

1. #475
   DataTalksClub/data-engineering-zoomcamp

   Data Engineering Zoomcamp is a free 9-week course on building production-ready data pipelines. Join the course here 👇🏼

   • Diamond IV

   1,984 Elo −546.9kJupyter Notebook
2. #557
   apache/superset

   Apache Superset is a Data Visualization and Data Exploration Platform

   • Diamond IV

   1,955 Elo −2975.5kPython
3. #577
   apache/airflow

   Apache Airflow - A platform to programmatically author, schedule, and monitor workflows

   • Diamond IV

   1,948 Elo −3347.4kPython
4. #932
   GokuMohandas/Made-With-ML

   Learn how to develop, deploy and iterate on production-grade ML applications.

   • Emerald II

   1,869 Elo −5250kJupyter Notebook
5. #1,084
   eugeneyan/applied-ml

   📚 Papers & tech blogs by companies sharing their work on data science & machine learning in production.

   • Emerald II

   1,843 Elo −1330.9k
6. #1,169
   kestra-io/kestra

   Orchestrate everything, everywhere. The open-source control plane for workflows across data, infrastructure, business…

   • Emerald III

   1,831 Elo −730.2kJava
7. #1,292
   semantica-agi/semantica

   Graph-Native Infrastructure for Context and Accountable AI Systems

   • Emerald III

   1,814 Elo +6814.6kPython
8. #1,478
   airbytehq/airbyte

   Open-source data movement for ELT pipelines and AI agents — from APIs, databases & files to warehouses, lakes, and AI…

   • Emerald IV

   1,787 Elo +222.3kPython
9. #1,810
   PrefectHQ/prefect

   Prefect is a workflow orchestration framework for building resilient data pipelines in Python.

   • Platinum I

   1,749 Elo −2324.1kPython
10. #2,289
    dagster-io/dagster

    An orchestration platform for the development, production, and observation of data assets.

    • Platinum II

    1,697 Elo +3716.4kPython
11. #3,107
    argoproj/argo-workflows

    Workflow Engine for Kubernetes

    • Platinum V

    1,629 Elo −517kGo
12. #3,290
    cocoindex-io/cocoindex

    Incremental engine for long horizon agents 🌟 Star if you like it!

    • Gold I

    1,618 Elo +5111.8kRust
13. #3,518
    andkret/Cookbook

    The Data Engineering Cookbook

    • Gold I

    1,604 Elo +515.5kPython
14. #4,152
    Avaiga/taipy

    Turns Data and AI algorithms into production-ready web applications in no time.

    • Gold III

    1,565 Elo −3819.4kPython
15. #4,376
    fivetran/great_expectations

    Always know what to expect from your data.

    • Gold IV

    1,552 Elo +2011.9kPython
16. #4,646
    growthbook/growthbook

    Open Source Feature Flags, Experimentation, and Product Analytics

    • Gold V

    1,537 Elo +378.5kTypeScript
17. #4,690
    Zipstack/unstract

    LLM-Driven Extraction of Unstructured Data — Built for API Deployments & ETL Pipeline Workflows

    • Gold V

    1,535 Elo +237.4kPython
18. #6,055
    xonsh/xonsh

    🐚 Python-powered shell. Full-featured, cross-platform and AI-friendly.

    • Silver III

    1,471 Elo +369.7kPython
19. #6,098
    datastacktv/data-engineer-roadmap

    Roadmap to becoming a data engineer in 2021

    • Silver III

    1,469 Elo −3212.7k
20. #6,306
    risingwavelabs/risingwave

    Event streaming platform for agentic AI. Continuously ingest, transform, and serve event streams in real time, at scale.

    • Silver IV

    1,460 Elo +139.4kRust
21. #6,531
    evidence-dev/evidence

    Business intelligence as code: build fast, interactive data visualizations in SQL and markdown

    • Silver IV

    1,451 Elo +357.1kTypeScript
22. #6,550
    feast-dev/feast

    The Open Source Feature Store for AI/ML

    • Silver IV

    1,450 Elo +297.3kPython
23. #7,268
    mage-ai/mage-ai

    🧙 Build, run, and manage data pipelines for integrating and transforming data.

    • Bronze I

    1,423 Elo −118.8kPython
24. #7,474
    redpanda-data/connect

    Fancy stream processing made operationally mundane

    • Bronze II

    1,415 Elo +98.7kGo
25. #9,340
    cloudquery/cloudquery

    Data pipelines for cloud config and security data. Build cloud asset inventory, CSPM, FinOps, and vulnerability…

    • Copper II

    1,330 Elo −146.5kGo