Model structured scraped data
A spider extracts consistent attributes such as a product name and price.
Scrapy · Class
Item
scrapy.Item is a dictionary-like container for structured data extracted by spiders. Define its permitted fields with scrapy.Field() and populate instances using normal mapping operations.
How it works
Subclass scrapy.Item to define the fields a spider may produce. Each field is declared with scrapy.Field(), while the resulting object behaves like a dictionary for storing and accessing values; its fields mapping retains declarations and metadata.
Syntax
class scrapy.Item(**args: Any, **kwargs: Any)Parameters
| Name | Type | Description |
|---|---|---|
args | Any | Optional positional mapping or iterable accepted by the dictionary-like initializer. |
kwargs | Any | Field values supplied by name during initialization. |
Returns
A new Item instance initialized from the supplied arguments.
Key points
- Subclass
scrapy.Itemwhen modeling structured data emitted by a spider. - Populate and read item values with normal dictionary operations, including constructor arguments.
- The declared-field boundary catches accidental references to unknown names.
- Attach metadata to fields when downstream processing needs field-specific behavior.
- Item compatibility is defined through support by the itemadapter library.
Common mistakes
- Only declared field names are accepted; using an undeclared name raises
KeyError.
Example: Declare and populate an Item
import scrapy
class Product(scrapy.Item):
name = scrapy.Field()
price = scrapy.Field()
product = Product(name="Book", price="12.99")
print(product["name"])
print(product.fields.keys())Declares fields on a custom item, initializes values with keyword arguments, reads a value using dictionary syntax, and inspects all declared fields.
When to use it
Describe output schema
A project needs exporters or pipelines to know the available output fields.
Catch extraction errors
A team wants misspelled field names to fail instead of silently adding keys.
Store field metadata
A field needs processing-related metadata for later Scrapy components.
Interview questions
QWhat is scrapy.Item, and how do you define one?
Item is Scrapy's feature-complete item type with a dictionary-like API. A custom item subclasses scrapy.Item and declares its allowed fields with Field attributes.
Follow-up: How does Scrapy recognize an item for pipelines? The documentation says Item is supported by itemadapter, so spider callback results of this type can be passed to item pipelines.
QHow are values and field declarations accessed?
The class declaration creates the permitted field names, and instances store values using the standard dictionary interface. The declaration metadata is available through the instance's fields attribute.
Follow-up: What does fields contain? It contains the declared field definitions, including fields that have not yet received values.
QWhy use Item instead of a plain dictionary?
An Item replicates the standard dictionary API, including its initializer, but limits keys to declared fields. Therefore, assigning or referring to an undefined field raises KeyError, helping expose spelling mistakes.
Follow-up: What alternative item types does Scrapy support? The documentation identifies dictionaries, dataclasses, and attrs objects through itemadapter as well as Item objects.
Related
scrapy.FieldItem.fieldsitemadapteritem pipelinesItem.copy()Item.deepcopy()