Create categorical codes
Convert repeated labels such as categories, names, or tokens into compact integer representations for downstream array operations.
Pandas · Function
pd.factorize
pandas.factorize encodes a one-dimensional sequence as integer codes and returns the corresponding unique values. By default, missing values receive -1 and are omitted from uniques.
How it works
Pass a one-dimensional sequence to receive two aligned results: integer codes and the distinct valid values they index. By default, distinct values retain discovery order, while missing values are represented by -1 and omitted from the uniques result; sorting or explicit missing-value encoding changes those behaviors.
Syntax
pandas.factorize(values, sort=False, use_na_sentinel=True, size_hint=None)Parameters
| Name | Type | Description |
|---|---|---|
values | sequence | A one-dimensional sequence. Non-pandas sequences are coerced to ndarrays. |
sort | bool, default False | Whether to sort unique values and remap codes to preserve their relationship; default is False. |
use_na_sentinel | bool, default True | Whether missing values use sentinel -1; default is True. If False, missing values receive non-negative codes and remain in uniques. |
size_hint | int, optional | Optional hint for the hashtable sizer. |
Returns
A tuple of codes and uniques. codes is an integer ndarray indexing uniques. uniques contains the unique valid values and is a Categorical for Categorical input, an Index for other pandas objects, or a one-dimensional ndarray otherwise.
Key points
- factorize converts one-dimensional values into integer labels plus a collection of distinct values.
- The values argument must represent a one-dimensional sequence.
- The codes result is an integer NumPy array whose entries point into uniques.
- The relationship between codes and uniques can reconstruct the original values through indexed selection.
- Categorical input returns categorical uniques.
- For other pandas objects, the unique values are returned as an Index.
- size_hint can provide an expected-size hint to the internal hashtable.
Common mistakes
- With the default missing-value setting, missing entries receive code -1.
- Set use_na_sentinel=False when missing values should receive an ordinary non-negative code.
- sort=True changes the order of uniques and correspondingly remaps the codes.
Example: Encode repeated values
import pandas as pd
values = ["cat", "dog", "cat", None]
codes, uniques = pd.factorize(values)
print(codes) # [ 0 1 0 -1]
print(uniques) # ['cat' 'dog']The returned integer codes identify each value's position in the unique-values array; the missing value receives the default sentinel code -1.
When to use it
Build a value dictionary
Obtain both an encoded array and the distinct values that define its codebook.
Control unique-value ordering
Use stable encounter-order labels or sorted labels depending on whether ordering or deterministic ranking is required.
Interview questions
QWhat does pandas.factorize return?
It returns a pair: an integer ndarray of codes and the unique valid values. Each code indexes into uniques, so uniques.take(codes) reconstructs the input values where codes are valid.
Follow-up: What happens with missing values? By default they map to -1 and are excluded from uniques; use_na_sentinel=False instead assigns them a non-negative code and retains them.
QHow does sort affect factorize?
The default preserves the order in which distinct values are encountered. With sort=True, pandas sorts the unique values and adjusts the codes so they still refer to the corresponding values.
Follow-up: Why must codes be shuffled after sorting? Because each code is an index into the reordered uniques array, so unchanged codes would identify the wrong values.
QHow does factorize handle different input types?
The input is a one-dimensional sequence, and non-pandas sequences are converted to ndarrays before factorization. The type of uniques depends on the input: categoricals produce a Categorical, other pandas objects produce an Index, and other inputs produce a one-dimensional ndarray.
Follow-up: How would you preserve an input-specific unique-value type? Pass the relevant pandas object or Categorical rather than first converting it to a plain array.
Related
pandas.Series.factorizepandas.Index.factorize