Buffers & Binary Data
JavaScript strings are Unicode text, not raw bytes. When you work with TCP sockets, file I/O, cryptography, image processing, or network protocols, you need binary data containers. Node.js provides several abstractions for binary data, each optimized for different use cases.
The Buffer class is the oldest and most powerful binary data type in Node.js. It represents a fixed-length sequence of bytes backed by raw memory outside the V8 heap. More recent APIs like Blob, ArrayBuffer, and typed arrays align Node.js with browser standards.
This guide covers creating, converting, slicing, and encoding binary data; choosing between Buffer, Blob, and ArrayBuffer; and writing safe code that avoids memory leaks and encoding bugs.
Buffers can be created from strings, arrays, other Buffers, or allocated with a specific size. When you allocate a Buffer, Node.js may return uninitialized memory, so always fill or write to it before reading.
| 1 | const buf1 = Buffer.alloc(16); // zero-filled, safe |
| 2 | const buf2 = Buffer.allocUnsafe(16); // faster, may contain old data |
| 3 | const buf3 = Buffer.from('hello'); // from UTF-8 string |
| 4 | const buf4 = Buffer.from([0x48, 0x69]); // from byte array |
| 5 | |
| 6 | buf2.fill(0); // zero out unsafe allocation before use |
| 7 | |
| 8 | console.log(buf3.toString()); // hello |
| 9 | console.log(buf3.toString('hex')); // 68656c6c6f |
| 10 | console.log(buf3.toString('base64')); // aGVsbG8= |
Buffer.allocUnsafe is faster because it skips zeroing memory, but it can leak data from previous allocations. Use it only for performance-critical code and immediately overwrite the contents.
warning
Node.js supports several string encodings for converting between Buffers and strings. Choosing the wrong encoding silently corrupts data, especially with multi-byte characters or binary payloads.
| Encoding | Use Case | Caveat |
|---|---|---|
| utf8 | Text, default encoding | Variable byte length per character |
| ascii | 7-bit English text | Strips high bits; do not use for Unicode |
| base64 | Binary in text formats, URLs | 33% size overhead |
| hex | Debugging, hashes | 100% size overhead |
| latin1 | One-byte char sets | Maps bytes 0-255 directly |
| 1 | const text = 'Node.js 🚀'; |
| 2 | const buf = Buffer.from(text, 'utf8'); |
| 3 | |
| 4 | console.log(buf.length); // 12 bytes |
| 5 | console.log(Buffer.byteLength(text)); // 12 bytes |
| 6 | |
| 7 | // Decoding binary data as utf8 can produce replacement chars |
| 8 | const corrupted = Buffer.from([0xff, 0xfe]); |
| 9 | console.log(corrupted.toString('utf8')); // �� |
info
Buffer provides methods for reading and writing integers, floats, and arbitrary bytes at offsets. Always specify the endianness explicitly to avoid platform-dependent bugs.
| 1 | const buf = Buffer.alloc(8); |
| 2 | |
| 3 | buf.writeUInt16BE(0x1234, 0); // big-endian |
| 4 | buf.writeUInt16LE(0x5678, 2); // little-endian |
| 5 | buf.writeUInt32BE(0xdeadbeef, 4); |
| 6 | |
| 7 | console.log(buf); |
| 8 | // <Buffer 12 34 78 56 de ad be ef> |
| 9 | |
| 10 | console.log(buf.readUInt16BE(0)); // 4660 |
| 11 | console.log(buf.readUInt16LE(2)); // 22136 |
| 12 | console.log(buf.readUInt32BE(4)); // 3735928559 |
Network protocols almost always use big-endian byte order. File formats and hardware registers may use little-endian. Document your assumptions in comments and validate data length before reading.
best practice
Modern Node.js supports the same binary APIs as browsers. ArrayBuffer is a fixed-length raw binary buffer. Typed arrays like Uint8Array provide views into an ArrayBuffer. Blob represents immutable binary data and is useful for file uploads and fetch responses.
| 1 | // ArrayBuffer and typed arrays |
| 2 | const ab = new ArrayBuffer(16); |
| 3 | const view = new Uint8Array(ab); |
| 4 | view.set([1, 2, 3, 4]); |
| 5 | |
| 6 | console.log(view.buffer === ab); // true |
| 7 | |
| 8 | // Convert Buffer to ArrayBuffer without copying |
| 9 | const buf = Buffer.from('hello'); |
| 10 | const ab2 = buf.buffer.slice(buf.byteOffset, buf.byteOffset + buf.byteLength); |
| 11 | |
| 12 | // Blob from file-like data |
| 13 | const blob = new Blob(['hello world'], { type: 'text/plain' }); |
| 14 | console.log(blob.size); // 11 |
| 15 | |
| 16 | const text = await blob.text(); |
| 17 | console.log(text); // hello world |
When converting between Buffer and ArrayBuffer, be careful about shared memory. A Buffer may be a view into a larger underlying ArrayBuffer, so slicing without proper offset handling can expose unintended bytes.
note
Many protocols use fixed-length headers followed by variable-length payloads. Parsing them with Buffer requires careful offset tracking. Streams are usually the right choice for protocols where data arrives in chunks.
| 1 | function parseHeader(buf) { |
| 2 | if (buf.length < 8) { |
| 3 | throw new Error('Incomplete header'); |
| 4 | } |
| 5 | |
| 6 | return { |
| 7 | magic: buf.readUInt16BE(0), |
| 8 | version: buf.readUInt8(2), |
| 9 | flags: buf.readUInt8(3), |
| 10 | payloadLength: buf.readUInt32BE(4) |
| 11 | }; |
| 12 | } |
| 13 | |
| 14 | const header = Buffer.from([ |
| 15 | 0x4e, 0x44, // magic 'ND' |
| 16 | 0x01, // version |
| 17 | 0x00, // flags |
| 18 | 0x00, 0x00, 0x00, 0x10 // payload length: 16 |
| 19 | ]); |
| 20 | |
| 21 | console.log(parseHeader(header)); |
| 22 | // { magic: 20036, version: 1, flags: 0, payloadLength: 16 } |
best practice
Readable streams emit data as Buffer chunks by default. Writable streams consume Buffers. Transform streams often inspect or reshape chunks. Concatenating all chunks into a single Buffer is convenient but dangerous for large inputs.
| 1 | const { Transform } = require('stream'); |
| 2 | |
| 3 | const upperCase = new Transform({ |
| 4 | transform(chunk, encoding, callback) { |
| 5 | // chunk is a Buffer in object mode off |
| 6 | this.push(chunk.toString().toUpperCase()); |
| 7 | callback(); |
| 8 | } |
| 9 | }); |
| 10 | |
| 11 | process.stdin.pipe(upperCase).pipe(process.stdout); |
For accumulating small payloads, Buffer.concat works well. For unbounded streams, process data incrementally to avoid loading the entire input into memory.
| 1 | const chunks = []; |
| 2 | for await (const chunk of readable) { |
| 3 | chunks.push(chunk); |
| 4 | } |
| 5 | const full = Buffer.concat(chunks); |
| 6 | console.log(full.length); |
warning
Binary data handling is a common source of vulnerabilities. Integer overflow, unchecked lengths, and uninitialized memory can lead to crashes, information disclosure, or remote code execution.
Integer overflow
Reading a 32-bit length field and allocating that many bytes without bounds checks can exhaust memory. Cap sizes to reasonable limits.
Uninitialized memory
Buffer.allocUnsafe can leak secrets. Prefer Buffer.alloc unless you have benchmarked a need for the unsafe variant.
Encoding confusion
Decoding arbitrary bytes as UTF-8 produces replacement characters. Decoding text as binary corrupts it. Keep binary and text data separate.
best practice