Find which columns consume the most memory
Diagnosing a large DataFrame
Pandas · Module
df.memory_usage
Returns per-column memory usage in bytes, optionally including the index and deeper accounting for object-dtype values. DataFrame.info uses this measurement by default.
How it works
Call this method to obtain per-column memory estimates in bytes. You can include the index and request deeper accounting for object-dtype values, which is useful when assessing a frame’s actual footprint.
Syntax
DataFrame.memory_usage(index=True, deep=False)Parameters
| Name | Type | Description |
|---|---|---|
index | bool, default True | Whether to include memory used by the DataFrame index. When true, its value is first in the returned Series. |
deep | bool, default False | Whether to inspect object-dtype data deeply for system-level memory consumption. |
Returns
Series containing the memory usage of each column in bytes; the index contribution is included first when index=True.
Key points
- The result is a Series of byte counts, one measurement for each DataFrame column.
- Set index=True to add the index’s memory contribution; set it to False for column-only accounting.
- DataFrame.info uses this measurement when displaying a frame’s memory usage.
- The info display can be turned off globally through pandas.options.display.memory_usage.
- deep=True expands accounting by examining object-dtype data instead of relying only on shallow storage estimates.
Common mistakes
- The default shallow calculation does not fully inspect object-dtype contents; use deep=True when their system-level memory matters.
- The index is included by default and appears as the first result, so exclude it explicitly when comparing only data columns.
Example: Inspect DataFrame memory
import pandas as pd
df = pd.DataFrame({"name": ["Ada", "Grace"], "score": [10, 20]})
print(df.memory_usage())
print(df.memory_usage(index=False, deep=True))The first call reports each column plus the index using shallow accounting; the second excludes the index and inspects object values more deeply.
When to use it
Measure memory before and after dtype changes
Comparing storage strategies
Use deep accounting for object values
Auditing text-heavy columns
Interview questions
QWhat does DataFrame.memory_usage return?
It returns a Series containing memory usage for each column, measured in bytes. The index can optionally contribute a separate first item.
Follow-up: How would you estimate object-column overhead more accurately? Use deep=True so object dtypes are interrogated for system-level memory consumption.
QWhat is the purpose of index=False?
index controls whether the index contributes to the returned Series and defaults to True. With index=False, the result contains column measurements only.
Follow-up: What ordering should callers expect when the index is included? The index measurement is the first item, followed by column measurements.
QWhen should deep=True be used?
deep defaults to False, providing shallow accounting. Setting it to True performs deeper inspection of object-dtype values and includes that consumption in the returned values.
Follow-up: Why might deep=True be especially relevant for text-heavy data? Object values can consume memory beyond the array storage, so shallow accounting can understate their footprint.
Related
DataFrame.infopandas.options.display.memory_usage