Compare with prior or subsequent rows
Create lag or lead features for time-series analysis.
Pandas · Module
df.shift
DataFrame.shift moves values by a chosen number of periods, inserting missing values where needed. With freq, it advances date-like index labels without realigning the data.
How it works
DataFrame.shift moves values by a specified number of row periods, leaving newly uncovered positions missing by default. With a date-like index and freq, it instead changes index labels by the offset without realigning the data; iterable periods can produce multiple shifted columns.
Syntax
DataFrame.shift(periods=1, freq=None, axis=0, fill_value=<no_default>, suffix=None)Parameters
| Name | Type | Description |
|---|---|---|
periods | int or Sequence | Number of periods to shift; may be positive or negative, or a sequence of integers for multiple shifts. |
freq | DateOffset, tseries.offsets, timedelta, or str, optional | Offset from the tseries module or a time rule; optional. |
axis | {0 or 'index', 1 or 'columns'}, optional | Axis to shift; optional. |
fill_value | object, optional | Scalar used for newly introduced missing values; optional and dtype-dependent when omitted. |
suffix | str, optional | String inserted into generated column names when periods is iterable; optional. |
Returns
DataFrame
Key points
- The method shifts a DataFrame by a chosen number of periods and optionally accepts a time offset.
- Use positive periods for a forward row displacement or negative periods for a backward displacement.
- A sequence of period values requests multiple shifts in one call.
- The periods argument controls the displacement count and accepts an integer or sequence.
- fill_value lets you choose the scalar placed into positions exposed by the shift.
- For numeric columns, the default replacement for newly exposed positions is NumPy NaN.
Common mistakes
- Supplying freq requires a date- or datetime-like index; otherwise the operation raises NotImplementedError.
- Using freq changes labels only, so the underlying data stays in its existing row positions.
- The special infer value works only when the index exposes a frequency or inferred frequency.
Example: Create a lagged column
import pandas as pd
sales = pd.DataFrame({"units": [10, 12, 15]})
sales["previous_units"] = sales["units"].shift(1)
print(sales)Creates a one-row lag column; the first row has no preceding value and therefore contains the default missing value.
When to use it
Build multiple shifted features
Generate several lagged versions of columns in one operation.
Apply a time offset to the index
Move calendar labels while preserving data positions.
Use a custom fill value
Control the values introduced at the edges of a shifted frame.
Interview questions
QWhat does DataFrame.shift do when freq is omitted?
It shifts values by row periods and inserts missing values where no source row exists. A positive period moves values toward later rows, while a negative period moves them toward earlier rows.
Follow-up: How would you shift calendar labels without moving values? Use a date-like index and pass freq; pandas advances the index by the offset without realigning the data.
QHow does periods behave when it is a sequence?
periods accepts one integer or a sequence of integers. With a sequence, pandas performs one shift for each integer and returns the resulting columns together.
Follow-up: How can generated columns be distinguished? Pass suffix when periods is iterable; pandas places it after the original column name and before each shift value.
QHow are new missing values filled?
fill_value supplies the scalar used for newly introduced missing positions. If omitted, pandas chooses a dtype-dependent missing value, such as np.nan for numeric data.
Follow-up: What constraint applies to freq? The index must be date or datetime-like, or pandas raises NotImplementedError.
Related
DataFrame.asfreqDataFrame.diffDataFrame.pct_changeSeries.shift