Analytical database for data-driven Web applications 🪶

Spark-free Python utilities for Microsoft Fabric focused on Data Engineering using Polars and delta-rs

A platform and cloud-based service for data sharing based on the Delta Sharing protocol.

OpenAutoLoader: A lightweight, open-source alternative to Databricks Auto Loader. Built with Polars and SQLite for efficient, incremental file ingestion.

This project demos how to integrate DuckDB and Delta Lake with dbt, orchestrated by Dagster. All data is stored and processed locally.

RFCs for changes to Delta

Resources for BlackPortal such as logo, banner, fonts etc.

Thin Python wrapper for reading Delta tables from Azure Blob Storage with low and stable latency.

ELT for Polygon.io financial data using Prefect

A Python ETL library for creating declarative data pipelines.

A Flight SQL proxy for Delta Lake. Query Delta tables via Apache Arrow Flight with efficient streaming and predicate pushdown.

A pipeline to pull data from S3 and process using Polars, Delta-RS and DuckDB

A demo repository for integrating a 3rd party data source (e.g. a data platform exposing its data via APIs) to Apache Superset via Deltalake

AWS Lambda function for optimizing Delta tables

Implementation of event-driven lakehouse.

Adaptive Delta Lake CDC streaming consumer with a real-time, tunable web dashboard, in Rust

A transformation pipeline for Delta Lake using AWS SDK for Pandas

Delta Lake SDK for .NET backed by a native Rust delta-rs runtime.