Compress and Decompress Files with ZIP, GZIP, and TAR

Overview

A nightly job has to bundle three weeks of application logs into one file for an auditor, and the export balloons to 2 GB uncompressed. Every runtime already ships an archive library, but the defaults differ in ways that matter: ZIP compresses each entry separately so you can pull one file without unpacking the rest, TAR only concatenates and needs an external compressor, and GZIP handles exactly one stream. This recipe covers all three formats in Python, Node.js, Java, and Bash, including the streaming and extraction-safety code that a basic ZipFile snippet leaves out.

When to Use

Use this resource when:

• Packaging log bundles, report exports, or build artifacts for download
• Compressing HTTP responses or payloads to cut bandwidth (see Gzip and Brotli compression for HTTP-specific tuning)
• Extracting uploaded archives in web applications
• Rotating and archiving logs on disk (see bash log rotation)

Skip programmatic compression when:

• The payload is already compressed (JPEG, MP4, .zip, .parquet): you spend CPU to gain nothing
• Files are tiny and the endpoint is latency-sensitive; gzip below ~1 KB can produce output larger than the input
• You need random access into archives frequently; that calls for a real database or object store, not a bigger zip

Prerequisites

• Python 3.9+ (zipfile, gzip, tarfile, shutil in the standard library; is_relative_to needs 3.9, compresslevel on tarfile needs 3.12)
• Node.js 18+ with npm install archiver extract-zip for the ZIP examples (zlib is built in)
• JDK 11+ (java.util.zip is standard library)
• pigz optional, for parallel gzip in the Bash section
• pyminizip / Zip4j only if you need encrypted ZIPs (see the FAQ)

Companion code: the compress-decompress-files companion repo has these examples as runnable files, including the safe-extraction helpers.

Solution

Python

import zipfile
import gzip
import tarfile

# ZIP archive
with zipfile.ZipFile('archive.zip', 'w', zipfile.ZIP_DEFLATED) as z:
    z.write('file.txt')

# GZIP single file
with open('file.txt', 'rb') as f_in:
    with gzip.open('file.txt.gz', 'wb') as f_out:
        f_out.writelines(f_in)

# TAR archive
with tarfile.open('archive.tar.gz', 'w:gz') as tar:
    tar.add('data/')

JavaScript

const fs = require('fs');
const zlib = require('zlib');
const archiver = require('archiver');

// GZIP compress
const input = fs.createReadStream('file.txt');
const output = fs.createWriteStream('file.txt.gz');
input.pipe(zlib.createGzip()).pipe(output);

// ZIP archive
const archive = archiver('zip', { zlib: { level: 9 } });
archive.pipe(fs.createWriteStream('archive.zip'));
archive.file('file.txt', { name: 'file.txt' });
archive.finalize();

Java

import java.io.*;
import java.util.zip.*;

public class Compressor {
    // GZIP compress
    public void gzip(String src, String dest) throws IOException {
        try (FileInputStream fis = new FileInputStream(src);
             FileOutputStream fos = new FileOutputStream(dest);
             GZIPOutputStream gzos = new GZIPOutputStream(fos)) {
            fis.transferTo(gzos);
        }
    }

    // ZIP archive
    public void zip(String src, String dest) throws IOException {
        try (FileOutputStream fos = new FileOutputStream(dest);
             ZipOutputStream zos = new ZipOutputStream(fos);
             FileInputStream fis = new FileInputStream(src)) {
            zos.putNextEntry(new ZipEntry(new File(src).getName()));
            fis.transferTo(zos);
            zos.closeEntry();
        }
    }
}

Explanation

The three formats solve different problems. ZIP compresses each entry independently and keeps a central directory at the end of the file, which is why you can read one member without decompressing the whole archive — and why truncated ZIPs can lose the index even when the data is intact. GZIP wraps a single stream in DEFLATE compression plus a CRC32 checksum; it doesn’t know about filenames or directories. TAR concatenates files with metadata and applies no compression at all, which is why .tar is always paired with .gz, .bz2, or .xz on Unix systems.

All three default to DEFLATE (LZ77 + Huffman coding), which is fast and compresses text well (typically 60–80% on logs and source code). It does almost nothing on already-compressed data, and the algorithm predates multicore: for bulk work, pigz parallelizes gzip while staying format-compatible, and Zstandard (zstd) gives better ratios at comparable speed but produces a different format entirely.

Picking a format is mostly about the consumer, not the compressor:

The extraction is the dangerous part

Reading archives is riskier than writing them. A malicious ZIP can embed paths like ../../etc/cron.d/evil that escape the destination directory on extract — this is the zip-slip attack. A naive dest / entry check using string prefixes is itself exploitable: a member resolving to /tmp/out-evil/x passes a startswith("/tmp/out") test. Always compare resolved paths component-wise (Path.is_relative_to in Python, Path.startsWith on Path objects in Java, path.relative in Node), and cap the total uncompressed size to stop zip bombs. For a deeper walkthrough, see safe ZIP extraction in Python.

Advanced Usage

Python: streaming compression and safe extraction

import zipfile
import gzip
import tarfile
from pathlib import Path

def compress_directory_streaming(src_dir: str, dest_zip: str,
                                 level: int = 6) -> int:
    """Compress a directory to ZIP entry by entry. Returns file count."""
    src_path = Path(src_dir)
    file_count = 0
    with zipfile.ZipFile(dest_zip, 'w', zipfile.ZIP_DEFLATED,
                         compresslevel=level) as zf:
        for file_path in sorted(src_path.rglob('*')):
            if file_path.is_file():
                zf.write(file_path, file_path.relative_to(src_path))
                file_count += 1
    return file_count

def decompress_zip_safe(zip_path: str, dest_dir: str,
                        max_size: int = 1 << 30) -> int:
    """Extract ZIP with zip-slip and zip-bomb protection."""
    dest_path = Path(dest_dir).resolve()
    dest_path.mkdir(parents=True, exist_ok=True)
    total = 0
    count = 0
    with zipfile.ZipFile(zip_path, 'r') as zf:
        for info in zf.infolist():
            member = info.filename
            member_path = (dest_path / member).resolve()
            # is_relative_to compares path components, not string prefixes,
            # so "/tmp/out-evil/x" cannot pass a check for "/tmp/out"
            if not member_path.is_relative_to(dest_path):
                raise ValueError(f"Unsafe path detected: {member}")
            total += info.file_size
            if total > max_size:
                raise ValueError(f"Archive exceeds {max_size} bytes")
            zf.extract(member, dest_path)
            count += 1
    return count

def gzip_file_streaming(src: str, dest: str, level: int = 6) -> None:
    """GZIP a single file in 64 KB chunks."""
    with open(src, 'rb') as f_in, gzip.open(dest, 'wb',
                                           compresslevel=level) as f_out:
        while chunk := f_in.read(65536):
            f_out.write(chunk)

def tar_directory(src_dir: str, dest: str, mode: str = 'w:gz') -> None:
    """Create a TAR.GZ archive of a directory."""
    with tarfile.open(dest, mode) as tar:
        tar.add(src_dir, arcname=Path(src_dir).name, recursive=True)

def list_archive_contents(archive_path: str) -> list[str]:
    """List contents of a ZIP or TAR archive."""
    if archive_path.endswith('.zip'):
        with zipfile.ZipFile(archive_path, 'r') as zf:
            return zf.namelist()
    elif archive_path.endswith(('.tar.gz', '.tgz', '.tar')):
        with tarfile.open(archive_path, 'r:*') as tar:
            return tar.getnames()
    raise ValueError(f"Unsupported archive format: {archive_path}")

# Usage
# count = compress_directory_streaming('logs/', 'logs.zip', level=6)
# print(f"Compressed {count} files")
# extracted = decompress_zip_safe('upload.zip', 'extracted/')
# print(f"Extracted {extracted} files safely")

Node.js: streaming pipelines with zlib

const fs = require('fs');
const zlib = require('zlib');
const { pipeline } = require('stream');
const { promisify } = require('util');
const pipe = promisify(pipeline);

async function gzipFile(srcPath, destPath, level = 6) {
    const src = fs.createReadStream(srcPath);
    const gzip = zlib.createGzip({ level });
    const dest = fs.createWriteStream(destPath);
    await pipe(src, gzip, dest);   // rejects if any stage fails
}

async function gunzipFile(srcPath, destPath) {
    const src = fs.createReadStream(srcPath);
    const gunzip = zlib.createGunzip();
    const dest = fs.createWriteStream(destPath);
    await pipe(src, gunzip, dest);
}

async function gzipDirectory(srcDir, destZip) {
    const archiver = require('archiver');
    const output = fs.createWriteStream(destZip);
    const archive = archiver('zip', { zlib: { level: 6 } });

    const done = new Promise((resolve, reject) => {
        output.on('close', () => resolve(archive.pointer()));
        output.on('error', reject);
        archive.on('error', reject);
    });

    archive.pipe(output);
    archive.directory(srcDir, false);
    archive.finalize();

    return await done;  // resolves with total bytes written
}

async function extractZipSafe(zipPath, destDir) {
    const extract = require('extract-zip');
    const path = require('path');
    const destRoot = path.resolve(destDir);

    await extract(zipPath, {
        dir: destRoot,
        onEntry: (entry) => {
            // path.relative returns "../..." for escapes; a string
            // startsWith check would let sibling prefixes slip through
            const rel = path.relative(destRoot, path.resolve(destRoot, entry.fileName));
            if (rel.startsWith('..') || path.isAbsolute(rel)) {
                throw new Error(`Unsafe path in archive: ${entry.fileName}`);
            }
        },
    });
}

// Usage
// gzipFile('large.log', 'large.log.gz', 9);
// const bytes = await gzipDirectory('logs/', 'logs.zip');
// console.log(`Archive size: ${bytes} bytes`);
// await extractZipSafe('upload.zip', 'extracted/');

Java: batch compression with try-with-resources

import java.io.*;
import java.nio.file.*;
import java.util.zip.*;
import java.util.List;
import java.util.ArrayList;
import java.util.stream.Stream;

public class BatchCompressor {

    // GZIP a single file
    public static void gzipFile(Path src, Path dest, int bufferSize) throws IOException {
        try (InputStream fis = Files.newInputStream(src);
             OutputStream fos = Files.newOutputStream(dest);
             GZIPOutputStream gzos = new GZIPOutputStream(fos, bufferSize)) {
            fis.transferTo(gzos);
        }
    }

    // ZIP multiple files with streaming
    public static int zipFiles(List<Path> sources, Path destZip) throws IOException {
        int count = 0;
        try (OutputStream fos = Files.newOutputStream(destZip);
             ZipOutputStream zos = new ZipOutputStream(fos)) {
            for (Path src : sources) {
                zos.putNextEntry(new ZipEntry(src.getFileName().toString()));
                try (InputStream fis = Files.newInputStream(src)) {
                    fis.transferTo(zos);
                }
                zos.closeEntry();
                count++;
            }
        }
        return count;
    }

    // Extract ZIP with zip-slip protection
    public static int extractZipSafe(Path zipPath, Path destDir) throws IOException {
        Files.createDirectories(destDir);
        int count = 0;
        try (InputStream fis = Files.newInputStream(zipPath);
             ZipInputStream zis = new ZipInputStream(fis)) {
            ZipEntry entry;
            while ((entry = zis.getNextEntry()) != null) {
                // Path.startsWith compares name components, so a sibling
                // prefix like "out-evil" cannot pass a check for "out"
                Path destFile = destDir.resolve(entry.getName()).normalize();
                if (!destFile.startsWith(destDir)) {
                    throw new IOException("Unsafe zip entry: " + entry.getName());
                }
                if (entry.isDirectory()) {
                    Files.createDirectories(destFile);
                } else {
                    Files.createDirectories(destFile.getParent());
                    Files.copy(zis, destFile, StandardCopyOption.REPLACE_EXISTING);
                }
                count++;
            }
        }
        return count;
    }

    // Compress all files in a directory
    public static int compressDirectory(Path srcDir, Path destZip) throws IOException {
        List<Path> files = new ArrayList<>();
        try (Stream<Path> stream = Files.walk(srcDir)) {
            stream.filter(Files::isRegularFile).forEach(files::add);
        }
        return zipFiles(files, destZip);
    }
}

// Usage
// BatchCompressor.gzipFile(Path.of("large.log"), Path.of("large.log.gz"), 8192);
// int count = BatchCompressor.compressDirectory(Path.of("logs/"), Path.of("logs.zip"));
// int extracted = BatchCompressor.extractZipSafe(Path.of("upload.zip"), Path.of("extracted/"));

Bash: tar/gzip with safety checks and parallel compression

#!/usr/bin/env bash
set -euo pipefail

# Compress directory to tar.gz
compress_tarball() {
    local src="$1"
    local dest="$2"
    local level="${3:-6}"
    tar -c -C "$(dirname "$src")" "$(basename "$src")" \
        | gzip -"$level" -c > "$dest"
    echo "Created $dest ($(du -h "$dest" | cut -f1))"
}

# Extract tarball safely: reject absolute paths and ../ escapes
extract_tarball_safe() {
    local archive="$1"
    local dest="$2"
    mkdir -p "$dest"
    if tar -tf "$archive" | grep -E '^/|\.\.'; then
        echo "Error: unsafe paths detected in $archive" >&2
        return 1
    fi
    tar -xzf "$archive" -C "$dest"
    echo "Extracted to $dest"
}

# Parallel gzip via pigz — faster on multicore, same format
compress_parallel() {
    local src="$1"
    local dest="$2"
    if command -v pigz &>/dev/null; then
        tar -c -C "$(dirname "$src")" "$(basename "$src")" | pigz -p 4 > "$dest"
    else
        tar -czf "$dest" -C "$(dirname "$src")" "$(basename "$src")"
    fi
    echo "Created $dest"
}

# Compress each file in a directory individually
batch_gzip() {
    local dir="$1"
    local count=0
    for file in "$dir"/*; do
        [[ -f "$file" ]] || continue
        gzip -c "$file" > "${file}.gz"
        ((count++))
    done
    echo "Compressed $count files in $dir"
}

# Usage
# compress_tarball logs/ logs.tar.gz 6
# extract_tarball_safe archive.tar.gz extracted/
# compress_parallel data/ data.tar.gz
# batch_gzip /var/log/app/

Variants

Technology	Approach	Notes
Python	shutil.make_archive()	One-call ZIP/TAR creation; no streaming or per-entry control
Node.js	adm-zip	In-memory ZIP manipulation; convenient but buffers everything
Java	Apache Commons Compress	Adds BZIP2, LZMA, 7Z, and more formats beyond java.util.zip
All	zstd / 7z CLI	Better ratios than DEFLATE; different formats, not interchangeable

What Works

1. Stream archives instead of buffering them: transferTo, pipeline, and with/try blocks keep memory flat regardless of input size
2. Use compression level 6 as the balanced default; level 9 adds ~30–50% more CPU time for single-digit gains on text
3. Validate every extracted path component-wise and cap total uncompressed size; zip-slip and zip bombs are both archive-side attacks
4. Prefer GZIP for single streams and ZIP/TAR for bundles; pick the format by who extracts it, not by which library is nearest
5. List archive contents before extracting untrusted uploads (tar -tf, zf.namelist())

Common Mistakes

1. Loading entire archives into memory when the libraries all support streaming
2. Checking extracted paths with string startswith: a sibling-prefix directory like dest-evil bypasses it
3. Forgetting finalize() or closeEntry(), producing truncated or corrupt archives
4. Compressing already-compressed formats (JPEG, MP4, other archives): wasted CPU, sometimes larger output
5. Ignoring filename encodings when archives cross platforms — ZIP predates UTF-8, and non-ASCII names can mangle between Windows and Linux

Further Reading

• Python zipfile documentation: ZipFile, compression types, and extraction API
• Python tarfile documentation: modes, filters, and the data extraction filter
• OWASP notes on zip-slip and archive attacks: path traversal in archive extraction