Arrow table ingestion
Convert a PyArrow table produced by an external data pipeline into a Polars DataFrame.
Polars · Function
pl.from_arrow
Converts an Arrow table or array into a Polars DataFrame or Series, usually without copying data. Optional schema, type overrides, and rechunking control the result.
How it works
Pass a PyArrow table, array, record batch, iterable of Arrow batches or tables, or a supported Arrow-exportable object to obtain a Polars DataFrame or Series. Polars performs the conversion mostly without copying data; optional schema controls naming and types, schema_overrides adjusts selected types, and rechunk requests contiguous memory.
Syntax
polars.from_arrow(data, schema=None, *, schema_overrides=None, rechunk=False) → DataFrame | SeriesParameters
| Name | Type | Description |
|---|---|---|
data | pa.Table | pa.Array | pa.ChunkedArray | pa.RecordBatch | Iterable[pa.RecordBatch | pa.Table] | ArrowArrayExportable | ArrowStreamExportable | Arrow table, array, chunked array, record batch, iterable of Arrow record batches or tables, or an Arrow exportable object. |
schema | SchemaDefinition | None | Optional schema definition for the conversion. |
schema_overrides | SchemaDict | None | Optional mapping of schema names to type overrides. |
rechunk | bool | Whether to make all converted data contiguous in memory. |
Returns
DataFrame or Series
Key points
- Use pl.from_arrow to convert Arrow data into either a Polars DataFrame or Series.
- The conversion is designed to avoid copying data in most cases.
- If the desired output type is already known, the corresponding Polars constructor is an alternative.
- The data parameter accepts common PyArrow table, array, chunked-array, and record-batch objects.
- The optional schema argument can provide schema information during conversion.
- Use schema_overrides to supply conversion-specific type overrides.
- The rechunk option is Boolean and defaults to False in the documented signature.
- Enabling rechunk requests contiguous memory for the converted data.
Common mistakes
- Arrow types without a direct Polars equivalent may be converted to the nearest supported Polars type.
- Do not assume every conversion is fully zero-copy; the documentation qualifies zero-copy behavior as applying for the most part.
Example: Create a DataFrame from an Arrow table
import polars as pl
import pyarrow as pa
table = pa.table({"a": [1, 2, 3], "b": [4, 5, 6]})
df = pl.from_arrow(table)
print(df)Converts a PyArrow table into a Polars DataFrame.
When to use it
Typed Series creation
Turn a PyArrow array into a typed Polars Series by supplying a schema.
Low-copy interoperability
Import Arrow-compatible objects while preserving memory efficiency in most cases.
Interview questions
QWhat inputs and outputs does pl.from_arrow support?
The function accepts Arrow tables, arrays, chunked arrays, record batches, iterables of Arrow record batches or tables, and supported Arrow exportable objects. It returns a DataFrame or Series depending on the input.
Follow-up: How can an Arrow array be made into a named, typed Series? Pass a schema such as {"s": pl.Int32}; the documentation demonstrates this form.
QWhat are schema and schema_overrides used for?
schema provides schema information for the conversion, while schema_overrides supplies type overrides. Both are optional and default to None.
Follow-up: When the input is an Arrow array, what does the example use to control the resulting Series? It passes schema={"s": pl.Int32}.
QWhat does the rechunk parameter control?
rechunk controls whether the resulting data is made contiguous in memory. The documented signature defaults rechunk to False.
Follow-up: Why might rechunk be enabled? It can ensure contiguous memory, though that may require reorganizing the input chunks.
Related
polars.DataFramepolars.Seriespolars.DataFrame.to_arrow