Filter missing records
Data cleaning
Pandas · Function
pd.notna
Detect whether a scalar or array-like object contains non-missing values. The result is a scalar Boolean for scalar input and a corresponding Boolean structure for array-like input.
How it works
pd.notna accepts either one value or an array-like object and checks each value for missingness. It returns one Boolean for scalar input, or a Boolean-shaped result matching the input container for array-like input.
Syntax
pandas.notna(obj)Parameters
| Name | Type | Description |
|---|---|---|
obj | array-like or object value | Object to check for not null or non-missing values. |
Returns
bool or array-like of bool. Scalar input returns a scalar Boolean; array-like input returns corresponding Boolean indicators.
Key points
- Use pd.notna to test whether values are present rather than missing.
- It accepts either a single object or an array-like input.
- A scalar input produces one Boolean result.
- An array-like input produces one Boolean per corresponding element.
- Series and DataFrame inputs preserve their container type while replacing values with Boolean indicators.
- Strings passed directly are evaluated as scalar values and return a scalar Boolean.
Common mistakes
- Empty strings and positive or negative infinity are treated as present values, not missing values.
Example: Detect valid values
import numpy as np
import pandas as pd
values = pd.Series([1, np.nan, None, "ok"])
mask = pd.notna(values)
print(mask.tolist()) # [True, False, False, True]Creates a boolean mask that identifies which Series entries contain non-missing values.
When to use it
Check required fields
Validation
Count available observations
Exploration
Interview questions
QWhat does pd.notna return for scalar versus array-like input?
pd.notna returns True for a valid value and False for a missing one. With a scalar it returns a scalar bool; with array-like input it returns a corresponding Boolean result.
Follow-up: How would you retain only valid rows in a DataFrame column? Apply the returned mask, for example, df[pd.notna(df["column"])].
QWhich kinds of values does pd.notna identify as missing?
The function recognizes missing values according to pandas' missing-data rules, including NaN for numeric data, None or NaN in object data, and NaT for datetimelike data.
Follow-up: Would an empty string be removed by this test? No: the documentation says empty strings are not considered NA.
QHow can pd.notna be used to build a missing-value filter?
For a Series or DataFrame, the result contains Boolean values aligned with the input's shape and labels. Each output position indicates whether the corresponding input value is valid.
Follow-up: How can you count valid entries in a Series? Use pd.notna(series).sum(), because True values contribute one.
Related
pandas.isnapandas.isnullpandas.notnullSeries.notnaDataFrame.notna