Create Parquet files
Persisting tabular data for later Polars or analytics-tool reads.
Polars · Module
Writes a Polars DataFrame to Apache Parquet. It accepts a path or writable binary stream and supports compression, statistics, layout tuning, PyArrow options, metadata, and partitioned datasets.
Call this method on a DataFrame with a destination path or writable binary stream. The writer produces an Apache Parquet file, using zstd compression by default; optional settings control compression, statistics, row groups, data pages, the PyArrow backend, metadata, and directory-based partitioning.
DataFrame.write_parquet(file: str | Path | IO[bytes], *, compression: ParquetCompression = 'zstd', compression_level: int | None = None, statistics: bool | str | dict[str, bool] = True, row_group_size: int | None = None, data_page_size: int | None = None, use_pyarrow: bool = False, pyarrow_options: dict[str, Any] | None = None, partition_by: str | Sequence[str] | None = None, metadata: ParquetMetadata | None = None, ...) -> None| Name | Type | Description |
|---|---|---|
file | str | Path | IO[bytes] | Destination path or writable binary file-like object; use a directory when writing a partitioned dataset. |
compression | ParquetCompression | Compression codec; defaults to zstd. |
compression_level | int | None | Compression level, when supported by the selected codec. |
statistics | bool | str | dict[str, bool] | Whether and how to write Parquet statistics; defaults to true. |
row_group_size | int | None | Row-group size measured in rows. |
data_page_size | int | None | Data-page size measured in bytes. |
use_pyarrow | bool | Whether to use PyArrow for writing. |
pyarrow_options | dict[str, Any] | None | Options passed to PyArrow. |
partition_by | str | Sequence[str] | None | Column name or sequence of names used to partition the dataset. |
metadata | ParquetMetadata | None | Optional Parquet metadata. |
None
import tempfile
from pathlib import Path
import polars as pl
with tempfile.TemporaryDirectory() as tmp:
path = Path(tmp) / "data.parquet"
df = pl.DataFrame({"city": ["Oslo", "Lima"], "value": [3, 7]})
df.write_parquet(path)
print(pl.read_parquet(path))Creates a DataFrame, writes it to a temporary Apache Parquet file using the default compression, and reads it back.
Persisting tabular data for later Polars or analytics-tool reads.
Organizing large output for selective dataset access.
Reducing storage size or changing write-time performance.
Sending Parquet bytes directly to another service.
It writes the DataFrame to Apache Parquet and returns None. The destination can be a path or a writable file-like object, and the default compression is zstd.
Follow-up: How would you write a partitioned dataset? Pass partition_by with one or more column names and use a directory destination.
compression selects the Parquet compression codec; the documented choices include lz4, uncompressed, snappy, gzip, brotli, and zstd. zstd is the default and is recommended for good compression performance, while lz4 favors speed.
Follow-up: What happens if compression_level is provided? It controls the compression level, with supported bounds depending on the selected codec.
statistics controls whether statistics are written to Parquet headers and defaults to true. row_group_size controls row-group size in rows, while data_page_size controls data-page size in bytes.
Follow-up: Why might statistics be disabled? Writing them requires extra computation, so disabling them can reduce write-time work when those statistics are not needed.
DataFrame.write_ipcDataFrame.write_jsonDataFrame.write_csvread_parquet