File Upload Validation

Overview

File uploads are one of the most common attack vectors in web applications. Unvalidated uploads can lead to remote code execution, cross-site scripting, and data breaches. Here is how to how to validate file uploads by checking size limits, MIME types, magic bytes, and content structure before accepting any file from a user.

When to Use

Use this resource when:

• Building a web app that accepts user-generated images, documents, or media. See Image Optimization for post-upload processing.
• Implementing a CMS, forum, or SaaS with attachment support. See Export CSV Excel for data export capabilities.
• You need to comply with security standards (PCI-DSS, SOC 2). See Secret Management for secure credential storage.
• Processing files from untrusted sources (public forms, APIs). See Input Validation for untrusted input handling.

Solution

Python

import os
import magic
from werkzeug.utils import secure_filename

ALLOWED_EXTENSIONS = {"png", "jpg", "jpeg", "gif", "pdf"}
MAX_FILE_SIZE = 5 * 1024 * 1024  # 5 MB

def validate_upload(file_storage):
    # 1. Check filename extension
    filename = secure_filename(file_storage.filename)
    ext = filename.rsplit(".", 1)[1].lower() if "." in filename else ""
    if ext not in ALLOWED_EXTENSIONS:
        raise ValueError(f"Extension not allowed: {ext}")

    # 2. Check file size
    file_storage.seek(0, os.SEEK_END)
    size = file_storage.tell()
    file_storage.seek(0)
    if size > MAX_FILE_SIZE:
        raise ValueError(f"File too large: {size} bytes")

    # 3. Check magic bytes (libmagic)
    mime = magic.from_buffer(file_storage.read(2048), mime=True)
    file_storage.seek(0)
    expected_mimes = {
        "png": "image/png", "jpg": "image/jpeg",
        "jpeg": "image/jpeg", "gif": "image/gif", "pdf": "application/pdf"
    }
    if mime != expected_mimes.get(ext):
        raise ValueError(f"MIME mismatch: got {mime}, expected {expected_mimes.get(ext)}")

    return filename

JavaScript (Node.js)

const path = require("path");
const multer = require("multer");
const fileType = require("file-type");
const fs = require("fs");

const ALLOWED = { png: "image/png", jpg: "image/jpeg", pdf: "application/pdf" };
const MAX_SIZE = 5 * 1024 * 1024;

const upload = multer({
  limits: { fileSize: MAX_SIZE },
  fileFilter: (req, file, cb) => {
    const ext = path.extname(file.originalname).toLowerCase().replace(".", "");
    if (!ALLOWED[ext]) return cb(new Error("Extension not allowed"));
    cb(null, true);
  },
});

async function validateBuffer(buffer, ext) {
  const type = await fileType.fromBuffer(buffer);
  if (!type || type.mime !== ALLOWED[ext]) {
    throw new Error(`MIME mismatch: ${type?.mime}`);
  }
  return true;
}

Java (Spring Boot)

import org.springframework.web.multipart.MultipartFile;
import java.util.Set;

public class UploadValidator {
    private static final Set<String> ALLOWED = Set.of("image/png", "image/jpeg", "application/pdf");
    private static final long MAX_SIZE = 5L * 1024 * 1024;

    public static void validate(MultipartFile file) {
        if (file.getSize() > MAX_SIZE) {
            throw new IllegalArgumentException("File exceeds 5 MB");
        }
        String contentType = file.getContentType();
        if (!ALLOWED.contains(contentType)) {
            throw new IllegalArgumentException("MIME type not allowed: " + contentType);
        }
        // Additional: check magic bytes with Apache Tika or similar
    }
}

Explanation

Validation should happen in layers:

1. Client-side — improves UX but is trivial to bypass.
2. Server-side extension check — fast but easily spoofed.
3. Server-side MIME type check — better, but still relies on HTTP headers.
4. Magic bytes (file signature) — reads the actual file content to determine type. The most reliable single check.
5. Content scanning / AV — essential for any environment handling untrusted files.

Each layer catches different threats. Never rely on a single check.

Variants

Technology	Validation Library	Notes
Python	python-magic	Reads libmagic database; very accurate
Node.js	file-type	Pure JS, fast, no native deps
Java	Apache Tika	Heavyweight but handles 1000+ formats
Go	mimetype	Fast, pure Go, zero-allocation reads
Ruby	Marcel	Rails default, uses both extension and magic

What Works

• Validate before saving to disk: Check everything in memory or a temp buffer first.
• Use random filenames: Never store files with original user-provided names. Map to UUIDs internally.
• Store outside web root: Serve files via controller/API, not direct filesystem access.
• Scan with AV: Integrate ClamAV or a cloud scanner for untrusted uploads.
• Rate limit uploads: Prevent abuse and disk exhaustion.

Common Mistakes

• Trusting the Content-Type header: Attackers can set this to anything.
• Relying only on extension: A .jpg can contain PHP code.
• No size limit: A single upload can fill your disk.
• Saving to public directories: If the file is executable, it may be served and run.
• No virus scanning: Malicious files may pass type checks but still harm users.

Additional Best Practices

1. Set per-user upload quotas. Track total uploaded bytes per user to prevent disk exhaustion from a single account. Store quotas in Redis or your database:

import redis

r = redis.Redis()

def check_quota(user_id: str, file_size: int, max_quota: int = 100 * 1024 * 1024) -> bool:
    """Check if user has enough quota for this upload."""
    key = f"upload_quota:{user_id}"
    used = int(r.get(key) or 0)
    if used + file_size > max_quota:
        return False
    r.incrby(key, file_size)
    return True

# if not check_quota(user_id, file_size):
#     raise ValueError("Upload quota exceeded")

2. Strip EXIF data from uploaded images. EXIF can contain GPS coordinates, camera serial numbers, and other PII. Re-encoding with sharp or PIL strips most metadata:

const sharp = require('sharp');

async function stripExif(inputBuffer) {
    return sharp(inputBuffer)
        .rotate()
        .removeExif()
        .png()
        .toBuffer();
}

// const cleanBuffer = await stripExif(req.file.buffer);

3. Use Content-Disposition: attachment for serving user uploads. Prevent browsers from rendering uploaded files inline, which could execute scripts in the context of your domain:

location /uploads/ {
    add_header Content-Disposition "attachment";
    add_header X-Content-Type-Options "nosniff";
    add_header Content-Security-Policy "default-src 'none'";
}

Additional Common Mistakes

1. Not checking for decompression bombs. A small uploaded ZIP can expand to gigabytes on extraction. Limit decompressed size during extraction:

import zipfile

MAX_DECOMPRESSED_SIZE = 100 * 1024 * 1024  # 100 MB

def safe_extract_zip(zip_path: str, dest_dir: str) -> int:
    """Extract ZIP with decompression bomb protection."""
    total_size = 0
    count = 0
    with zipfile.ZipFile(zip_path, 'r') as zf:
        for info in zf.infolist():
            total_size += info.file_size
            if total_size > MAX_DECOMPRESSED_SIZE:
                raise ValueError(f"Decompression bomb: {total_size} bytes")
            zf.extract(info, dest_dir)
            count += 1
    return count

# safe_extract_zip('upload.zip', '/app/extracted/')

2. Allowing SVG uploads without sanitization. SVG files can contain <script> tags and onload handlers. Sanitize SVGs or disallow them entirely:

// SVG can contain XSS: <svg onload="alert(document.cookie)">
// Option 1: Disallow SVG entirely
const ALLOWED = { png: 'image/png', jpg: 'image/jpeg', pdf: 'application/pdf' };

// Option 2: Sanitize SVG with DOMPurify (server-side)
// const DOMPurify = require('isomorphic-dompurify');
// const clean = DOMPurify.sanitize(svgString, { USE_PROFILES: { svg: true, svgFilters: true } });

3. Not logging upload failures. Upload validation failures are security events. Log them with context for audit and incident response:

import logging

logger = logging.getLogger('upload_security')

def validate_upload_with_logging(file_storage, user_id: str):
    try:
        result = validate_upload_secure(file_storage)
        logger.info(f"Upload accepted: user={user_id} file={result['original_name']} size={result['size']}")
        return result
    except ValueError as e:
        logger.warning(f"Upload rejected: user={user_id} reason={e} filename={file_storage.filename}")
        raise
    except Exception as e:
        logger.error(f"Upload error: user={user_id} error={e} filename={file_storage.filename}", exc_info=True)
        raise