Eager dataset loading
A local or remote Parquet dataset must be loaded for immediate DataFrame operations.
Polars · Function
Eagerly reads one or more Parquet sources into a Polars DataFrame. It supports column projection, row limits, parallel reading, Hive partitioning, cloud-storage options, and native or PyArrow backends.
read_parquet eagerly loads one or more Parquet sources into a DataFrame. It accepts local paths, file-like inputs, bytes, and supported cloud sources; optional arguments control column selection, row limits, parallelism, partition handling, schema behavior, credentials, and the reader backend.
polars.read_parquet(source: FileSource, *, columns: list[int] | list[str] | None = None, n_rows: int | None = None, row_index_name: str | None = None, row_index_offset: int = 0, parallel: ParallelStrategy = 'auto', use_statistics: bool = True, hive_partitioning: bool | None = None, glob: bool = True, schema: SchemaDict | None = None, hive_schema: SchemaDict | None = None, try_parse_hive_dates: bool = True, low_memory: bool = False, storage_options: StorageOptionsDict | None = None, credential_provider: CredentialProviderFunction | Literal['auto'] | None = 'auto', use_pyarrow: bool = False, pyarrow_options: dict[str, Any] | None = None, memory_map: bool = True, include_file_paths: str | None = None, missing_columns: Literal['insert', 'raise'] = 'raise', _expand_paths: bool = True) -> DataFrame| Name | Type | Description |
|---|---|---|
source | FileSource | Path, paths, file-like object, bytes, or another supported Parquet file source. |
columns | list[int] | list[str] | None | Column names or zero-based column indices to read. |
n_rows | int | None | Maximum number of rows to read; only valid with the native reader. |
row_index_name | str | None | Name for an inserted first row-index column. |
row_index_offset | int | Starting value for the inserted row index. |
parallel | ParallelStrategy | Strategy used to parallelize reading. |
use_statistics | bool | Whether to use Parquet statistics. |
hive_partitioning | bool | None | Whether Hive-style partitioning is enabled or disabled. |
glob | bool | Whether to expand glob patterns in paths. |
schema | SchemaDict | None | Schema override for the input. |
hive_schema | SchemaDict | None | Schema information for Hive partitions. |
try_parse_hive_dates | bool | Whether to parse dates in Hive partition paths. |
DataFrame
DataFrame.columns to project by zero-based indices or by column names.n_rows provides an optional upper bound on rows read by the native reader.n_rows is unavailable when use_pyarrow=True.row_index_offset must not be negative.scan_parquet when a lazy query plan is needed.import polars as pl
# Replace with an existing Parquet file.
df = pl.read_parquet("data.parquet", columns=["id", "amount"])
print(df)Reads the selected columns from a Parquet file into a Polars DataFrame.
A local or remote Parquet dataset must be loaded for immediate DataFrame operations.
Only a subset of fields is needed from a wide Parquet dataset.
Files are organized in Hive-style directory partitions.
Input is stored in cloud object storage.
polars.read_parquet return, and what kinds of input can it read?It eagerly reads a Parquet source and returns a Polars DataFrame. The source can be a file path, a list of paths, a file-like object, bytes, or another supported FileSource.
Follow-up: When should a lazy alternative be preferred? Use scan_parquet when you want lazy execution and query optimization before collecting results.
Pass columns as either column names or zero-based column indices. n_rows limits how many rows are read, but the page specifies that this option is valid only with the native reader, not PyArrow.
Follow-up: How would you reduce the input further without eagerly materializing it? Start with scan_parquet, apply lazy projections and filters, then collect.
parallel='auto' lets Polars choose a parallel reading strategy, while use_statistics=True enables use of Parquet statistics. hive_partitioning controls interpretation of partitioned paths, and glob controls path expansion.
Follow-up: What should be considered when reading cloud data? Configure storage_options and, where needed, a credential provider for the cloud source.
polars.scan_parquetpolars.read_parquet_schemapolars.read_parquet_metadataDataFrame.write_parquet