Multi-column custom transformation
A calculation combines several columns in a way that native expressions cannot express.
Polars · Module
Maps a custom function over multiple columns or expressions and returns one expression backed by a Series result. Specify the output type when possible, and use the scalar and elementwise flags to control result handling and optimization.
Use polars.map_batches when a custom function must receive multiple complete input Series and return one Series. Polars can infer the output type, but explicitly supplying return_dtype is preferred; mark elementwise functions for possible optimization and preserve scalar outputs with returns_scalar=True.
polars.map_batches(exprs: Sequence[str | Expr], function: Callable[[Sequence[Series]], Series | Any], return_dtype: PolarsDataType | DataTypeExpr | None = None, *, is_elementwise: bool = False, returns_scalar: bool = False) -> Expr| Name | Type | Description |
|---|---|---|
exprs | Sequence[str | Expr] | Expressions representing the input Series to the function. |
function | Callable[[Sequence[Series]], Series | Any] | Function applied to the input Series; it returns a Series or another supported result. |
return_dtype | PolarsDataType | DataTypeExpr | None | Datatype of the output Series; recommended whenever possible. |
is_elementwise | bool = False | Whether the callback is elementwise, enabling possible performance and optimization improvements. |
returns_scalar | bool = False | Whether a scalar callback result should remain scalar instead of being wrapped in a list. |
Expr with the data type given by return_dtype.
import polars as pl
df = pl.DataFrame({"a": [1, 2, 3], "b": [10, 20, 30]})
out = df.select(
pl.map_batches(
[pl.col("a"), pl.col("b")],
lambda series: series[0] + series[1],
return_dtype=pl.Int64,
).alias("total")
)
print(out)Adds two input columns by passing their full Series to a custom batch function.
A calculation combines several columns in a way that native expressions cannot express.
A callback naturally processes complete Series values rather than individual rows.
A callback returns one aggregate value and that value must remain scalar.
A slice-safe, equal-length operation can benefit from batch optimization.
polars.map_batches accepts multiple expressions and passes the corresponding full Series objects to one callback. The callback produces a single Series result, represented as an Expr.
Follow-up: How would you improve planning for a row-wise operation? Set is_elementwise=True only when equal-length inputs can be processed independently in slices without changing results.
return_dtype declares the output Series datatype. The documentation recommends setting it whenever possible; otherwise Polars attempts inference by calling the function with dummy data.
Follow-up: What risk does inference introduce? The callback must tolerate arbitrary or dummy input and should not depend on external state.
By default, a scalar callback result is wrapped in a list because the API assumes the function returns something Series-like. Set returns_scalar=True when the scalar should remain scalar.
Follow-up: When is is_elementwise useful? It can improve performance and optimization when operations have equal-length inputs and are safe to run sequentially on slices.
polars.Expr.map_batchespolars.map_elementspolars.replace