Polars · Function

pl.scan_parquet

Lazily scans one or more local or cloud-hosted Parquet files and returns a LazyFrame. Deferred execution lets Polars optimize filters and selected columns before reading data.

How it works

scan_parquet creates a LazyFrame backed by one or more local or cloud-hosted Parquet files. The query is deferred, allowing filters and selected columns to be pushed toward the scan before collect() executes it.

Syntax

polars.scan_parquet(source: FileSource, *, n_rows: int | None = None, row_index_name: str | None = None, row_index_offset: int = 0, parallel: ParallelStrategy = 'auto', use_statistics: bool = True, hive_partitioning: bool | None = None, glob: bool = True, hidden_file_prefix: str | Sequence[str] | None = None, schema: SchemaDict | None = None, cache: bool = True, rechunk: bool = False, low_memory: bool = False, storage_options: dict[str, Any] | None = None, credential_provider: CredentialProviderFunction | Literal['auto'] | None = 'auto', retries: int = 2, include_file_paths: str | None = None, allow_missing_columns: bool = False, extra_columns: dict[str, Any] | None = None, cast_options: ScanCastOptions | None = None, _column_mapping: str | None = None, _default_values: dict[str, Any] | None = None, _deletion_files: Sequence[str] | None = None, _table_statistics: dict[str, Any] | None = None, _page_statistics: bool = True) -> LazyFrame

Parameters

Name	Type	Description
`source`	`FileSource`	Path or other supported file source identifying local or cloud-hosted Parquet input; glob patterns can represent multiple files.
`n_rows`	`int \| None`	Stop reading after this many rows; defaults to no limit.
`row_index_name`	`str \| None`	Name of an optional row-index column.
`row_index_offset`	`int`	Starting offset for the row-index column.
`parallel`	`ParallelStrategy`	Strategy controlling scan parallelism.
`cache`	`bool`	Whether to cache the scan result.
`use_statistics`	`bool`	Whether to use Parquet statistics to skip pages.
`low_memory`	`bool`	Whether to reduce memory pressure at the possible expense of performance.
`storage_options`	`dict[str, Any] \| None`	Options for the storage connection, such as cloud authentication settings.

Returns

LazyFrame

Key points

Use it when the input should become a lazy query plan instead of being read eagerly.
Filters and column selection can be moved toward file scanning, potentially avoiding unnecessary data.
n_rows optionally limits how many rows are read from the Parquet input.
row_index_name can request an additional row-index column in the resulting lazy data.
The parallel option chooses the scan parallelism strategy, with auto as the default.

Common mistakes

The function does not immediately load the dataset; call collect() or another terminal operation to execute the lazy query.
Cloud paths require suitable storage configuration when authentication or provider-specific connection options are needed.

Example: Filter and project a Parquet scan

import polars as pl

lf = pl.scan_parquet("data/events/*.parquet")
result = (
    lf.filter(pl.col("value") > 0)
      .select(["event_id", "value"])
      .collect()
)

Creates a lazy Parquet scan, lets the query remain lazy through filtering and column selection, then executes it with collect().

When to use it

Deferred analytics queries

Build a lazy transformation pipeline over Parquet data.

Predicate and projection pushdown

Read only matching rows and required columns from large files.

Cloud Parquet ingestion

Scan datasets stored in object storage.

Hive-partitioned data

Read directory-organized partitioned datasets.

Interview questions

QWhat does pl.scan_parquet return, and why use it?

scan_parquet returns a LazyFrame rather than an eagerly materialized DataFrame. This delays execution and allows the optimizer to work on the query before the data is collected.

Follow-up: Why can this reduce resource use? Predicate and projection operations can be pushed toward the scan, so irrelevant rows and columns may be avoided earlier.

QHow can scan_parquet read remote data?

The source can be a local or cloud-hosted Parquet file or collection of files, including a glob pattern such as an S3 path. The function's storage_options argument carries connection-specific settings for remote access.

Follow-up: How would you scan an authenticated S3 glob? Pass the S3 glob as source and provide credentials and region information through storage_options.

QWhich options affect scan performance?

A scan can use Parquet statistics to skip pages that cannot satisfy the query, controlled by use_statistics. Parallel controls the strategy used while reading, and low_memory trades performance for lower memory pressure.

Follow-up: When might low_memory be useful? It is useful when memory is constrained and reduced pressure matters more than maximum throughput.

polars.read_parquet
polars.read_parquet_schema
polars.LazyFrame.collect

Source: polars.scan_parquet — Polars documentation. Checked 2026-10-15.