Home / Projects
TRANS · 04Live

Big Data Engines Benchmark

The One Billion Row Challenge in Python: DuckDB, PyArrow and Polars run head to head on a real 1,000,000,000-row aggregation, including a reproduced out-of-memory failure with the kernel's own log as evidence.

Data sourceSynthetic weather-station measurements, generated locally: 1,000,000,000 rows, 14.8 GiB, 41,343 distinct stations.
ArchitectureShared input contract → 8 independent implementations (stdlib, chunked stdlib, PyArrow, pandas, chunked pandas, Polars eager/lazy, DuckDB) → CSV/Parquet output → Streamlit dashboard
This runFull 1B rows, on 20 CPUs / 15 GiB RAM / 4 GiB swap. DuckDB and PyArrow re-verified at scale; Polars lazy reproduced an OOM kill
StackPython DuckDB PyArrow Polars pandas Streamlit
StatusLive: public repo, reproduced and verified on this portfolio's hardware

Context

This project is a from-scratch build of the One Billion Row Challenge in Python: generate a billion station;temperature records, then compute min/mean/max per station as fast as possible. It predates this portfolio (first built and published independently) and was brought in here, renamed to the portfolio's ID convention, and re-run end to end on new hardware to produce fresh, artifact-sourced evidence instead of just carrying over the original numbers.

Business Problem

Given a workload too large to comfortably fit in memory, which processing engine actually handles it, and what does "handles it" cost in time versus memory? The answer isn't universal: it depends on whether an engine's execution model is genuinely out-of-core or only claims to be.

Architecture

One generator (create_measurements.py) produces a single input file. Eight independent implementations read the same file and write the same output schema (station;min;mean;max), so approaches are comparable without changing the contract on either end. A Streamlit dashboard reads whichever run's output landed in data/station_metrics_mart.csv.

Reproduced in this portfolio session (2026-07-18)

Re-run on 20 CPUs / 15 GiB RAM / 4 GiB swap, deliberately different hardware from the original benchmark, to test whether the conclusion holds rather than just repeat it. Full 1,000,000,000-row input (14.8 GiB, generated in 324 s). Scope was narrowed to the three fast implementations; the slow stdlib/pandas paths already have historical numbers (12-24 min each) that re-running wouldn't change the conclusion of.

ImplementationResultSource
DuckDB22.73 s, 41,343 stationslogs/log_duckdb.csv
PyArrow1,185.02 s (~19.75 min), 41,343 stationslogs/log_pyarrow.csv
Polars (lazy)OOM-killed at 12.51 GiB residentjournalctl -k (kernel OOM-killer log)

Full machine-readable results: docs/evidence/run_results.json in the repo.

Results

The Polars OOM is not a bug: it independently reproduces the project's own historical finding ("did not complete in a 16 GiB environment"), now on a 15 GiB + 4 GiB swap machine, with the kernel's own out-of-memory log as evidence instead of an assumption. DuckDB's out-of-core aggregation finished in under 23 seconds on the same box that killed Polars's lazy engine at 1 billion rows.
Streamlit dashboard showing the station metrics mart built from the 1-billion-row DuckDB run, with min/avg/max temperature across 41,343 stations and a full sortable table

Challenges

1. A killed process still has to reach a real explanation

The driver script used set -e, so the Polars OOM kill (exit 137) stopped the batch before the dashboard-mart step ran. Rather than treat that as a script bug to route around silently, the kernel's own OOM log (journalctl -k) was pulled for the exact resident-memory figure at time of kill, and the mart was rebuilt manually from DuckDB's output, the implementation that actually finished.

2. Historical numbers vs. reproduced numbers

The original README's benchmark table was collected on different hardware (Intel i5-14500T, 16 GiB RAM). Rather than overwrite it with this session's numbers, which would quietly erase the original evidence, both live in the README: the new run first, clearly dated and scoped, the historical table preserved below it and labeled as such.

Tech Stack

CategoryTool
Engines comparedstdlib Python, pandas, PyArrow, Polars, DuckDB
DashboardStreamlit + Plotly
Testingpytest (tiny fixtures, never the 1B-row file)
ToolingRuff, Black, pre-commit, pip-audit
← Previous
AWS Cloud Foundations