Incremental data processing
Process a file, request, or generated source progressively without assembling the entire content first.
Node.js · Module
stream
Node.js streams are abstractions for processing data incrementally. The node:stream module defines readable, writable, duplex, and transform stream types plus callback- and Promise-oriented utilities.
How it works
The node:stream module supplies abstractions for data that is read, written, or both. Readable, Writable, Duplex, and Transform streams coordinate incremental data flow through events and piping, while the promises API offers asynchronous utility functions that resolve with Promise objects.
Syntax
const stream = require('node:stream');Key points
- Streams provide a common abstraction for processing data incrementally rather than requiring one particular source or destination implementation.
- A stream's directionality depends on whether it exposes reading, writing, or both capabilities.
- Stream lifecycle and availability notifications use the EventEmitter model, including events emitted by readable streams.
- Node.js categorizes streams as Writable, Readable, Duplex, and Transform.
- Transform streams combine readable and writable behavior while changing data between the two sides.
- Stream utilities are also available in a Promise-based form for asynchronous application code.
Common mistakes
- Use node:stream to implement or create stream instances; consuming an existing stream often does not require importing the module.
- The promises variant changes utility APIs from callback-oriented functions to Promise-returning functions; choose the import path deliberately.
Example: Transform streamed data
import { Readable, Transform, Writable } from 'node:stream';
const upper = new Transform({
transform(chunk, _encoding, callback) {
callback(null, chunk.toString().toUpperCase());
}
});
Readable.from(['hello\n', 'world\n'])
.pipe(upper)
.pipe(process.stdout);Creates a readable stream from iterable data, transforms each chunk, and pipes the result to standard output.
When to use it
Data pipelines
Connect a producer, transformation stage, and destination with backpressure-aware stream composition.
Streaming transformation
Use Transform or Duplex implementations when data must be changed while it flows.
Interview questions
QWhat are the four fundamental stream types?
A Readable supplies data, a Writable accepts data, a Duplex supports both directions, and a Transform is a Duplex that modifies data as it passes through. These abstractions let applications process data incrementally.
Follow-up: How would you connect stages while handling completion and errors? Use pipeline-style stream composition rather than manually coordinating every write, and select the callback or promises utility API according to the surrounding control flow.
QHow does a Readable expose available data?
Streams are EventEmitter instances, so readable streams can notify application code when data is available. Data can also be consumed through other documented readable-stream mechanisms, including piping and async iteration.
Follow-up: Why prefer incremental consumption for large inputs? It avoids requiring the complete input to be held and processed as one value, allowing work to proceed as chunks arrive.
QHow do you access the stream module and its promises API?
The module can be loaded with require('node:stream'), and its promises API is available through require('node:stream/promises') or require('node:stream').promises. The promises API provides asynchronous stream utilities that return Promise objects instead of using callbacks.
Follow-up: When is the promises API especially convenient? It fits naturally inside async functions, where completion and failure can be handled with await and ordinary Promise control flow.
Related
ReadableWritableDuplexTransformstream/promisespipelinefinishedPassThrough