Local analytics
Load a local .parquet file into a DataFrame for analysis.
Pandas · Function
Reads a Parquet object into a pandas DataFrame. It supports column selection, engine and storage configuration, dtype backends, and optional row filtering.
pandas.read_parquet loads a Parquet file, directory, URL, or binary file-like object through a supported Parquet engine and returns the data as a DataFrame. You can select columns, pass filesystem or storage settings, filter partitioned data, and control dtype conversion.
pandas.read_parquet(path, engine='auto', columns=None, storage_options=None, dtype_backend=<no_default>, filesystem=None, filters=None, to_pandas_kwargs=None, **kwargs)| Name | Type | Description |
|---|---|---|
path | str, path object or file-like object | Parquet path, URL, path-like object, or binary file-like object. |
engine | {‘auto’, ‘pyarrow’, ‘fastparquet’}, default ‘auto’ | Parquet implementation to use; auto, pyarrow, or fastparquet. |
columns | list, default None | Names of columns to load; None reads all columns. |
storage_options | dict, optional | Options for the relevant storage connection. |
dtype_backend | {‘numpy_nullable’, ‘pyarrow’}, default ‘numpy_nullable’ | Nullable dtype backend selection for the resulting DataFrame. |
filesystem | fsspec or pyarrow filesystem, optional | Filesystem object used to access the path. |
filters | List[Tuple] or List[List[Tuple]], optional | Row-group or partition predicates passed to the engine. |
to_pandas_kwargs | dict, optional | Additional keyword arguments forwarded to the underlying reader. |
**kwargs | dict | Additional engine-specific keyword arguments. |
path | str, path object or file-like object | Object to load. |
DataFrame
import pandas as pd
# Requires the optional pyarrow dependency.
df = pd.read_parquet("data.parquet", columns=["user_id", "score"])
print(df)Reads the selected columns from a Parquet file into a pandas DataFrame.
Load a local .parquet file into a DataFrame for analysis.
Read only selected fields from a large Parquet dataset.
Access Parquet data stored behind a supported URL or filesystem.
It accepts a path, URL, or binary file-like object and returns a DataFrame. The engine can be selected explicitly or determined automatically.
Follow-up: How would you reduce I/O? Pass columns to read only the required fields, and use filters when supported by the Parquet engine.
The columns argument is a list of column names. When provided, pandas reads only those columns instead of the complete Parquet dataset.
Follow-up: What additional optimization is available for partitioned data? Use filters to restrict rows during the Parquet read when the engine supports them.
The function supports auto, pyarrow, and fastparquet engine choices. With auto, pandas uses the io.parquet.engine option to determine the engine.
Follow-up: What is the practical consequence of passing filesystem? The documented behavior uses pyarrow for that access path, so engine compatibility must be checked.
DataFrame.to_parquetpandas.read_csvpandas.read_featherpandas.read_orcpandas.read_pickle