Stream Buffer Read: A Defensive Design Pattern for Content Size Validation

Tl;dr: Apps rely on untrusted parameter to perform size check. This can result into DoS attack. Stream Buffer Read is a defensive design pattern that prevents this.

(This is another post in my series of articles on defensive design patterns: Avoid validation with privilege return, Not normalising before validation bypasses security checks, Do not use String to store secret. It gets disclosed)

Verifying the size of input data is a critical factor in ensuring data safety. Without enforcing minimum and maximum limits on untrusted input, our application becomes vulnerable to a variety of exploitation scenarios, including Denial of Service (DoS) attacks.

During my secure code reviews, I often encounter applications that handle various types of files (binary, text, etc.) or parse messaging protocols (e.g., HTTP, Protobuf) without properly validating the input size. While some applications implement partial size checks, they often rely on untrusted parameters to make judgement calls. In both cases, the application remains susceptible to DoS attacks and other forms of exploitation.

Let’s delve into a specific example.

const acceptableSize = 1024 * 1024; // 1 MB'/check-size', (req, res) => {
  // Get the content length from the request headers
  const contentLength = req.headers['content-length'];

  if (!contentLength) {
    return res.status(400).json({ error: 'Content length header missing.' });

  if (Number(contentLength) > acceptableSize) {
    return res.status(400).json({ error: 'Content length exceeds acceptable size.' });

  // Input is within  acceptable range, let's process it

  // SNIP

The above POST request anticipates a HTTP body size of less than 1MB. It derives this size from the Content-Length HTTP header. However, this check is weak and susceptible to bypass. The Content-Length header is untrusted, allowing adversaries to arbitrarily set values within an HTTP request’s Content Header (as shown in the example below).

curl -X POST \
  -H "Content-Length: 10" \
  -H "Content-Type: application/json" \
  -d '{"key": "x"*2048}' \

The above request bypasses the size check, causing the application to process an unexpected 2MB of data.

Let’s consider another example.

The following Go code examines the size of uncompressed files by validating a parameter from the zip file.

func checkUncompressedFiles(zipFile io.Reader) (int, error) {
	uncompressedFiles := 0

	zipReader, err := zip.NewReader(zipFile, int64(1024*1024))
	if err != nil {
		return 0, err

	for _, file := range zipReader.File {
		if !file.FileInfo().IsDir() {
			uncompressedSize := int64(file.UncompressedSize64)
			if uncompressedSize > acceptableUncompressedFileSize {
				return 0, fmt.Errorf("uncompressed file size exceeds acceptable range")

	return uncompressedFiles, nil

This file size verification also illustrates reliance on an untrusted field. The UncompressedSize64 parameter resides in the Zip file header and can be manipulated by adversaries to sidestep the size check. This vulnerability can be exploited through a Compression Bomb or Zip Bomb attack, which fills up disk space.

We should avoid depending on parameters inherent to the file (e.g., uncompressed file size) or the protocol (e.g., HTTP Content Length) for making judgments. What constitutes a secure method of validating file or messaging protocol sizes?

Introducing the Stream Buffer Read Defensive Design Pattern

The Stream Buffer Read Defensive Design Pattern offers an approach to validate the sizes of untrusted files or messaging protocols (e.g., HTTP requests). When our application receives a file or a request, we only read the amount of data that we can handle. If the data surpasses our acceptable size, it is disregarded.

Implementing this design pattern involves creating temporary memory (i.e., a buffer) that a stream utilises to store data. As the buffer reaches capacity within our acceptable range, further data reading stops.

import { createReadStream, ReadStream } from 'fs';

const bufferSize = 1024; // Example buffer size of 1 KB

var readStream: ReadStream = createReadStream('./untrusted.bin');
var accumulatedBuffer: Buffer = Buffer.alloc(0);

readStream.on('data', chunk => { 
  // Accumulate the chunk into the buffer
  accumulatedBuffer = Buffer.concat([accumulatedBuffer, chunk]);
  // Check if the accumulated buffer size exceeds the buffer size limit
  if (accumulatedBuffer.length >= bufferSize) {
    console.log('Buffer is full. Stopping further reading.');
    readStream.pause(); // Pause the stream to stop further reading

readStream.on('end', () => {
  console.log('Stream ended.');

The bufferSize variable indicates the buffer’s maximum size before we halt stream reading. The accumulatedBuffer stores stream chunks. The code verifies if the accumulated buffer size exceeds the buffer size limit. In such cases, the stream pauses to prevent additional reading.

In the case of a compression bomb, we can omit the erroneous file size check and instead extract only the expected amount of data.

func readOnlyEnoughFromZip(zipFile io.Reader) error {
	r, err := zip.NewReader(zipFile, int64(1024*1024))
	if err != nil {
		return err
	defer zipFile.Close() // Close the zip reader when done

	buffer := make([]byte, 1024)
	for _, file := range r.File {
		fileReader, err := file.Open()
		if err != nil {
			return err
		defer fileReader.Close() // Close each file reader when done

		_, err = io.ReadFull(fileReader, buffer)
		if err != nil {
			return err

		// SNIP. Save buffer to a file.
		fmt.Println("Buffer:", buffer)

	return nil

The io.ReadFull function is used to copy “1024” bytes from the zip (source) to the buffer (destination). Files exceeding our acceptable size result in an error.

Wrap up

Reliance on parameters embedded within files or sent by protocols can lead to vulnerabilities that adversaries can exploit. By using the Stream Buffer Read Defensive Design Pattern, applications can safely validate the sizes of untrusted files and messaging protocols. This pattern directs applications to read data within predefined limits without relying on untrusted parameters.

:point_right: Patch this vulnerability in practice


Try BadZip.go to explore a real application that has a weak file size check. Can you write a security patch for it?