Node.js Buffers and Text Encoding Boundaries

Buffers contain bytes, while strings contain decoded text. Byte counts and string lengths are different, especially for multibyte characters.

Before you start

You should know JavaScript promises, asynchronous errors and the distinction between a process and a request. When following a server example, identify the resource owner and the point where work completes. Try experiments locally with bounded input instead of assuming production traffic behaves like a single request.

The practical goal is to reason through this situation: A UTF-8 character split across network chunks needs incremental decoding. Read the walkthrough first, then try the interview exercise before opening its answer. The important part is explaining the decision and its consequences, rather than remembering a definition alone.

Step-by-step walkthrough

Step 1: Distinguish characters from bytes

A string’s length and its UTF-8 byte count can differ. A character may span several chunks.

Step 2: Keep decoder state

Use StringDecoder or another incremental decoder that retains incomplete trailing byte sequences.

Step 3: Flush at completion

Call the decoder’s final operation when the stream ends so trailing state is handled deliberately.

Worked scenario

A UTF-8 character split across network chunks needs incremental decoding.

import { StringDecoder } from 'node:string_decoder';
const decoder = new StringDecoder('utf8');
const bytes = Buffer.from('€');
console.log(decoder.write(bytes.subarray(0, 1))); // empty
console.log(decoder.write(bytes.subarray(1))); // €
console.log(decoder.end());

JavaScript

Independent chunk.toString calls lose the relationship between the split bytes; the decoder preserves it.

Common mistake

Decoding each chunk independently can corrupt a character divided between chunks.

Verify the behavior

Split multibyte text at every possible byte boundary and compare the reconstructed string with the original.

Interview exercise

Read streamed text correctly.

Answer and reasoning

Use a decoder that preserves partial byte sequences and verify multibyte input across chunk boundaries.

Continue learning

Compare the scenario with the Node.js interview questions and test your understanding with the Node.js MCQs. For terminology and implementation details, consult the reference material.