Classify documents from word-count vectors
Text categorization
Scikit-Learn · Class
MultinomialNB
MultinomialNB is a Naive Bayes classifier for multinomial data, particularly discrete features such as word counts. It can also work with fractional values such as tf-idf and uses additive smoothing by default.
How it works
MultinomialNB estimates a class-specific probability for each feature from smoothed relative frequencies, then combines those probabilities with class priors to classify samples. It is designed for discrete features such as word counts, though fractional values such as tf-idf can also work.
Syntax
class sklearn.naive_bayes.MultinomialNB(*, alpha=1.0, force_alpha=True, fit_prior=True, class_prior=None)Parameters
| Name | Type | Description |
|---|---|---|
alpha | float or array-like of shape (n_features,), default=1.0 | Additive Laplace/Lidstone smoothing value; a float or an array-like value with one entry per feature. Default: 1.0. |
force_alpha | bool, default=True | Whether to keep alpha at its supplied value when it is zero. Default: True. |
fit_prior | bool, default=True | Whether to learn class prior probabilities. If false, uniform priors are used. Default: True. |
class_prior | array-like of shape (n_classes,), default=None | Prior probabilities for the classes. If omitted, priors are learned when fit_prior is true. |
Returns
An instance of sklearn.naive_bayes.MultinomialNB; prediction and probability methods return estimates for supplied samples.
Key points
- MultinomialNB applies the Naive Bayes approach to multinomially distributed feature data.
- It is a natural choice for discrete features, especially word-count vectors in text classification.
- Although count data is the usual representation, fractional feature values such as tf-idf can work empirically.
- alpha controls additive smoothing and can be a scalar or a separate value for each feature.
- fit_prior selects whether class priors are estimated from the training data.
- When fit_prior is false, every class receives a uniform prior rather than a learned prior.
Common mistakes
- The model is intended for count-like discrete features; although fractional tf-idf values may work in practice, integer counts are the natural input.
- Setting alpha to zero disables additive smoothing only when force_alpha remains true; use this cautiously because unseen feature-class combinations can cause zero probabilities.
Example: Text classification with word counts
from sklearn.feature_extraction.text import CountVectorizer
from sklearn.naive_bayes import MultinomialNB
texts = ["red apple", "red berry", "blue berry", "blue sky"]
y = [0, 0, 1, 1]
vectorizer = CountVectorizer()
X = vectorizer.fit_transform(texts)
model = MultinomialNB(alpha=1.0)
model.fit(X, y)
print(model.predict(vectorizer.transform(["red fruit"])))Converts text into word-count vectors, trains the classifier, and predicts a class for new text.
When to use it
Train efficiently on high-dimensional sparse representations
Sparse feature classification
Use tf-idf-like fractional feature values when empirically appropriate
Weighted text features
Interview questions
QWhat kind of data is MultinomialNB designed for?
MultinomialNB is a Naive Bayes classifier for multinomially distributed data. It is especially suitable for discrete feature representations such as word-count vectors, while fractional tf-idf values can also work in practice.
Follow-up: Why is it useful for text classification? Its features naturally represent counts of vocabulary terms, and the class-conditional probabilities can be estimated from relative feature frequencies.
QHow does MultinomialNB estimate feature probabilities?
For class y and feature i, the smoothed estimate is hat{theta}_{yi} = (N_{yi} + alpha) / (N_y + alpha n), where N_{yi} is the feature count for the class, N_y is the total count for that class, and n is the feature count.
Follow-up: What does alpha control? It is the additive Laplace/Lidstone smoothing parameter; larger values add more smoothing to the relative-frequency estimates.
QWhich constructor parameters most affect probability estimation?
alpha controls additive smoothing and accepts either one float or an array with one value per feature. fit_prior determines whether class priors are learned; if false, the classifier uses uniform priors, while class_prior can provide explicit priors.
Follow-up: What happens when alpha is zero? With force_alpha true, smoothing is not applied, which can leave zero probabilities for unseen feature-class combinations.
Related
CategoricalNBComplementNBGaussianNBBernoulliNBfitpartial_fitpredictpredict_proba