Heterogeneous tabular preprocessing
A DataFrame mixes numeric and categorical fields that need different preprocessing.
Scikit-Learn · Class
ColumnTransformer applies different transformers to selected columns of an array or pandas DataFrame and concatenates their outputs into one feature space. It is useful for heterogeneous data requiring separate preprocessing paths.
ColumnTransformer routes selected columns to separate transformers, fits each transformer on its assigned subset, and concatenates their outputs into one feature space. Unselected columns are dropped by default or can be passed through or processed by a remainder estimator.
class sklearn.compose.ColumnTransformer(*transformers*, *, remainder='drop', sparse_threshold=0.3, n_jobs=None, transformer_weights=None, verbose=False, verbose_feature_names_out=True)| Name | Type | Description |
|---|---|---|
transformers | list of tuples | List of (name, transformer, columns) tuples specifying transformers and the input subsets they process. |
remainder | {'drop', 'passthrough'} or estimator, default='drop' | Treatment for columns not selected by transformers: 'drop', 'passthrough', or an estimator. |
sparse_threshold | float, default=0.3 | Output density threshold used to decide whether a sparse result remains sparse. |
n_jobs | int, default=None | Number of jobs used to fit and transform transformers in parallel; None uses one job. |
transformer_weights | dict, default=None | Multiplicative weights applied to the outputs of individual transformers. |
verbose | bool, default=False | Whether to print progress messages during fitting and transforming. |
verbose_feature_names_out | bool or callable, default=True | Controls whether generated output feature names include transformer prefixes. |
A fitted ColumnTransformer instance from fit; transform produces the concatenated transformed feature space.
from sklearn.compose import ColumnTransformer
from sklearn.preprocessing import OneHotEncoder, StandardScaler
preprocessor = ColumnTransformer(
transformers=[
("numeric", StandardScaler(), ["age", "income"]),
("categorical", OneHotEncoder(handle_unknown="ignore"), ["city"]),
],
remainder="drop",
)
X_transformed = preprocessor.fit_transform(X)
This applies scaling and one-hot encoding to separate DataFrame columns, then concatenates the resulting feature blocks.
A DataFrame mixes numeric and categorical fields that need different preprocessing.
Several feature extraction steps must feed one estimator as a unified input.
Only selected columns should be transformed while untouched columns remain available.
The constructor receives named (name, transformer, columns) tuples. Each transformer handles its selected subset, and the outputs are concatenated into a single feature space.
Follow-up: How would you preserve columns not listed in the tuples? Set remainder='passthrough', or provide an estimator as remainder if those columns need additional processing.
The default remainder is 'drop', so unspecified columns are omitted. Setting remainder='passthrough' appends the remaining columns unchanged; an estimator can instead transform them.
Follow-up: What constraint applies to a remainder estimator? It must support both fit and transform, just like the explicitly configured transformers.
The columns selector may be a name, integer, array-like selector, boolean mask, slice, or callable. A callable receives X and returns a valid selector.
Follow-up: How can selectors differ between arrays and DataFrames? Integer selectors are positional, while strings can refer to DataFrame column names.
sklearn.compose.make_column_transformersklearn.compose.make_column_selectorsklearn.pipeline.Pipelinesklearn.preprocessing.StandardScalersklearn.preprocessing.OneHotEncoder