In-memory text processing
Parse or generate text using file-like operations without creating a temporary file.
Python · Module
io
Python’s io module provides the core stream interfaces for text, binary, and raw input/output. It also supplies in-memory file-like streams such as StringIO and BytesIO.
How it works
The io module defines the stream interfaces used for text, binary, and raw I/O. Text streams work with strings and can apply encoding and newline handling, binary streams work with bytes without those text transformations, and raw streams provide a lower-level unbuffered layer.
Syntax
import ioKey points
- Use io as the standard foundation for stream-oriented input and output.
- Choose among text, binary, and raw streams according to the representation and control level your code needs.
- Text streams expose character data as Python strings.
- Binary streams are appropriate for non-text data and byte-precise processing.
- For filesystem text, open the file in text mode and provide an explicit encoding when appropriate.
Common mistakes
- Binary streams do not encode, decode, or translate newlines, so callers must handle text conversion themselves.
- Raw streams are low-level building blocks and are usually not the right interface for ordinary application code.
- Reading text from a non-blocking stream can raise BlockingIOError when the operation cannot complete immediately.
Example: Read text from an in-memory stream
import io
with io.StringIO("alpha\nbeta\n") as stream:
lines = stream.readlines()
print([line.rstrip("\n") for line in lines])Uses an in-memory text stream as a file-like object and reads its lines.
When to use it
In-memory binary processing
Transform, buffer, or test binary data through read and write methods.
Stream-compatible components
Implement or adapt stream-oriented APIs using Python’s standard I/O interfaces.
Interview questions
QWhat are the main I/O categories in io?
The module covers three broad stream categories: text, binary, and raw I/O. Text streams consume and produce str, binary streams consume bytes-like values and produce bytes, and raw streams are the unbuffered lower-level layer.
Follow-up: Why should a network protocol usually use binary I/O rather than text I/O? Binary I/O avoids implicit encoding, decoding, and newline translation, allowing the protocol to control its byte representation.
QHow do StringIO and BytesIO differ?
StringIO is an in-memory text stream, while BytesIO is an in-memory binary stream. They provide file-like interfaces without requiring a filesystem object.
Follow-up: When is BytesIO preferable to StringIO? Use BytesIO when the data is inherently binary or when exact byte-level handling is required.
QWhat distinguishes text I/O from binary I/O?
Text I/O handles str values and may perform encoding, decoding, and newline processing at the boundary to a byte-oriented backing store. Binary I/O works directly with bytes and performs none of those text transformations.
Follow-up: What design choice avoids platform-dependent text behavior? Specify the intended encoding when creating a text stream, rather than relying on an implicit default.
Related
open()IOBaseRawIOBaseBufferedIOBaseTextIOBaseFileIOBytesIOStringIO