Preprocessing plus prediction
Apply repeatable preprocessing before a classifier or regressor.
Scikit-Learn · Class
Pipeline
Pipeline chains transformers in order and optionally finishes with a predictor. It lets the complete sequence be fitted, cross-validated, and parameter-tuned as one estimator.
How it works
Pipeline chains named estimators so each transformer fits and transforms the data before the next step receives it; the final estimator then fits the transformed data. It exposes the combined estimator for fitting, prediction, parameter tuning, and optional transformer caching.
Syntax
class sklearn.pipeline.Pipeline(*steps, transform_input=None, memory=None, verbose=False)Parameters
| Name | Type | Description |
|---|---|---|
steps | list of tuples | Ordered list of (name, estimator) tuples to chain. Every step must define fit; every non-final step must also define transform. |
transform_input | list or None | Metadata-routing configuration for data passed to methods; available in the current signature. |
memory | str, joblib.Memory, or None | Location or joblib.Memory-like object used to cache fitted transformers. |
verbose | bool | Whether to print timing information while fitting each step. |
Key points
- Pipeline represents an ordered preprocessing chain that may end in a predictive estimator.
- Construct it with named estimator tuples; their list order defines execution order.
- During fitting, each transformer is fitted and its transformed output becomes the next step's input.
- After preprocessing completes, the terminal estimator is fitted on the resulting representation.
- It packages multiple estimators so the whole sequence can be evaluated and tuned as one model.
Common mistakes
- Every step except the final one must provide
transform; otherwise it cannot participate as an intermediate pipeline stage. - The last step does not need to be a transformer, so a predictor can terminate the pipeline.
- Caching is optional and applies to transformers through the
memoryargument.
Example: Scale features and classify
from sklearn.datasets import load_iris
from sklearn.linear_model import LogisticRegression
from sklearn.pipeline import Pipeline
from sklearn.preprocessing import StandardScaler
X, y = load_iris(return_X_y=True)
model = Pipeline([
("scale", StandardScaler()),
("classify", LogisticRegression(max_iter=200)),
])
model.fit(X, y)
print(model.predict(X[:3]))This pipeline scales the features before fitting and using the logistic-regression predictor.
When to use it
Joint model selection
Evaluate preprocessing choices and model settings in one cross-validation workflow.
Transformer caching
Avoid recomputing costly transformer fits across repeated evaluations.
Interview questions
QHow does Pipeline fit its steps?
Pipeline applies the steps sequentially: it fits each transformer, transforms the data, passes that output onward, and finally fits the last estimator. Intermediate steps need both fit and transform, while the final estimator only needs fit.
Follow-up: How can pipeline parameters be tuned together? Use the step names with parameter names in the estimator's parameter interface, allowing cross-validation over the assembled sequence.
QWhat does the steps argument contain?
steps is an ordered list of (name, estimator) tuples. Each name identifies the corresponding estimator, and the order determines the data-processing sequence.
Follow-up: What constraint applies to names? They must identify distinct pipeline steps so the assembled estimator can address each step unambiguously.
QWhat is the purpose of memory?
Pipeline can cache its transformers with memory. This can avoid repeatedly recomputing transformer work during repeated fitting, such as cross-validation.
Follow-up: When is caching most useful? It is most useful when transformer fitting is expensive and the same preprocessing is evaluated repeatedly.
Related
make_pipelineFeatureUnionColumnTransformerTransformedTargetRegressor