Scikit-Learn · Function

permutation_importance

Computes feature importance for a fitted estimator by repeatedly shuffling each feature and measuring the resulting score change. It returns per-feature averages, variability, and raw scores across permutations.

How it works

The function evaluates a fitted estimator on X to obtain a baseline score, then repeatedly shuffles one feature column and scores the altered data. Each feature's importance is the baseline score minus the score after permutation; the returned object also summarizes variation across repetitions.

Syntax

sklearn.inspection.permutation_importance(estimator, X, y, *, scoring=None, n_repeats=5, n_jobs=None, random_state=None, sample_weight=None, max_samples=1.0)

Parameters

Name	Type	Description
`estimator`	`object`	Already-fitted estimator compatible with the scorer.
`X`	`ndarray or DataFrame`	Data used to compute permutation importance, with shape (n_samples, n_features).
`y`	`array-like or None`	Targets for supervised evaluation, or None for unsupervised evaluation.
`scoring`	`str, callable, list, tuple, or dict`	Scorer or collection of scorers; default is None.
`n_repeats`	`int`	Number of permutations per feature; default is 5.
`n_jobs`	`int or None`	Parallelism setting; default is None.
`random_state`	`int, RandomState instance, or None`	Pseudo-random generator control; default is None.
`sample_weight`	`array-like or None`	Sample weights used for scoring; default is None.
`max_samples`	`int or float`	Number or fraction of samples used; default is 1.0.

Returns

A Bunch containing importances_mean, importances_std, and importances. The mean and standard deviation have shape (n_features,), while raw scores have shape (n_features, n_repeats).

Key points

Use the function to evaluate how much individual features contribute to a fitted estimator's chosen metric.
The selected scorer determines the reference performance against which every feature permutation is compared.
The method disrupts one column at a time while leaving the other feature columns unchanged.
Larger positive values indicate a larger score drop after shuffling that feature.
Increase n_repeats when a more stable estimate across random shuffles is needed.
The returned importances array preserves each feature's score for every repetition.

Common mistakes

Call the function only after fitting the estimator.
Importance is score decrease after shuffling, so interpret it relative to the selected scoring function.
A training set measures reliance during fitting, whereas a hold-out set can assess feature contribution on unseen data.

Example: Measure feature importance

from sklearn.datasets import load_iris
from sklearn.ensemble import RandomForestClassifier
from sklearn.inspection import permutation_importance

X, y = load_iris(return_X_y=True)
model = RandomForestClassifier(random_state=0).fit(X, y)
result = permutation_importance(
    model, X, y, n_repeats=10, random_state=0
)
print(result.importances_mean)

Fit a classifier, permute each feature repeatedly, and inspect the average resulting score decrease.

When to use it

Model inspection

Explaining nonlinear or opaque estimators.

Generalization analysis

Assessing features on validation or test data.

Feature evaluation

Comparing relative reliance across tabular features.

Interview questions

QHow does permutation importance work?

It measures how much a fitted model's score changes when one feature column is randomly permuted. The value is the baseline score minus the score after permuting that column, averaged over repetitions.

Follow-up: Why can the value be negative? A permutation can occasionally improve the measured score, producing a negative difference.

QWhat inputs are required?

The estimator must already be fitted, and X supplies the data on which the baseline and permuted scores are evaluated. y supplies targets for supervised use or can be None for unsupervised use.

Follow-up: What dataset should be preferred? A held-out dataset is useful when the goal is to understand contribution to generalization rather than training-set reliance.

QHow do repetitions affect the output?

n_repeats controls how many permutations are performed for each feature. The result reports the mean, standard deviation, and all raw importance scores across those repetitions.

Follow-up: How can reproducibility be improved? Set random_state to control the pseudo-random permutations.

sklearn.inspection
partial_dependence

Source: permutation_importance — scikit-learn 1.9.1 documentation. Checked 2026-10-15.