Decode streamed text safely
A network or file stream divides a UTF-8 character between chunks.
Node.js · Module
node:string_decoder
The node:string_decoder module decodes Buffer chunks into text without breaking multi-byte UTF-8 or UTF-16 characters across chunk boundaries. StringDecoder buffers incomplete sequences until later input completes them.
How it works
Create a StringDecoder with the intended character encoding, then pass each incoming binary chunk to write(). It preserves incomplete multi-byte sequences internally, allowing characters split across chunks to be reconstructed; call end() to flush any remaining buffered input.
Syntax
import { StringDecoder } from 'node:string_decoder';
const decoder = new StringDecoder(encoding);Parameters
| Name | Type | Description |
|---|---|---|
encoding | string | The character encoding used by the decoder; defaults to utf8. |
Returns
A StringDecoder instance; write() and end() return decoded strings.
Key points
- The module turns binary Buffer chunks into strings while preserving character boundaries.
- Incomplete multi-byte sequences are retained internally instead of being emitted as broken text.
- The constructor accepts the character encoding used for decoding.
- If no encoding is supplied, UTF-8 is selected.
- write() returns decoded text and postpones incomplete characters until later input can complete them.
Common mistakes
- Calling end() with an incomplete final character does not preserve it; Node replaces those bytes with encoding-appropriate substitution characters.
Example: Decode UTF-8 across chunks
import { StringDecoder } from 'node:string_decoder';
const decoder = new StringDecoder('utf8');
const first = Buffer.from([0xF0, 0x9F]);
const second = Buffer.from([0x98, 0x80]);
console.log(decoder.write(first));
console.log(decoder.write(second));
console.log(decoder.end());The decoder retains the incomplete emoji bytes from the first chunk and emits the complete character when the remaining bytes arrive.
When to use it
Process chunked encoded data
Binary input arrives incrementally and must become text without replacement characters at chunk boundaries.
Decode with an explicit character encoding
A stream uses an encoding other than the default UTF-8.
Interview questions
QWhy use StringDecoder instead of converting every Buffer with toString() independently?
StringDecoder converts Buffer data into text while avoiding corruption when a multi-byte UTF-8 or UTF-16 character is split between chunks. Its internal buffer holds incomplete character bytes until a later write() completes them.
Follow-up: What should happen at end-of-stream if the final character is incomplete? Call end(); the remaining bytes are converted using the encoding’s substitution behavior.
QHow is StringDecoder used with chunked input?
Instantiate StringDecoder with an optional encoding, which defaults to UTF-8. Feed chunks through write(), and use end() when the input stream finishes.
Follow-up: How would you process an arbitrary stream? Create one decoder for the stream and call write() for every chunk, rather than creating a decoder per chunk.
QWhat happens when a chunk ends halfway through a character?
write() returns decoded text and retains incomplete multi-byte characters internally. This means a character divided across two Buffers is emitted only after enough bytes have arrived.
Follow-up: What input types can write() handle? The documentation describes Buffer, TypedArray, and DataView input for its incomplete-character handling.
Related
BufferTextDecoderstream.Transform