StackPractices
ES
beginner By Mathias Paulenko

Copy and Move Files Safely in Python, JS, Java, and Bash

Learn to copy and move files across platforms with Python, JavaScript, Java, and Bash. Includes atomic moves, checksums, symlinks, and batch patterns.

file-handling python javascript java bash
Topics: file handling

Overview

Copying and moving files looks simple until a partial transfer, permission error, or cross-device rename corrupts data. This recipe shows ready-to-use patterns in Python, JavaScript, Java, and Bash that cover overwrites, checksums, symlinks, and atomic moves.

When to Use

When to Avoid

  • Your use case is continuous replication (not one-shot copies), where rsync --daemon or lsyncd are purpose-built for keeping directories in sync over time.
  • You’re moving large objects into cloud storage; the provider’s SDK or CLI handles multipart uploads better.
  • End users need a graphical file manager; this is a scripting recipe.

Solution

Python

import shutil
from pathlib import Path
import hashlib

def safe_copy(src, dest, *, overwrite=False, verify=True, follow_symlinks=False):
    """Copy a file, preserving metadata and optionally verifying integrity."""
    src, dest = Path(src), Path(dest)
    if not src.exists():
        raise FileNotFoundError(f"Source not found: {src}")
    if dest.exists() and not overwrite:
        raise FileExistsError(f"Destination exists: {dest}")
    dest.parent.mkdir(parents=True, exist_ok=True)

    if src.is_symlink() and not follow_symlinks:
        dest.symlink_to(Path(src).readlink())
    else:
        shutil.copy2(src, dest)

    if verify and not src.is_symlink():
        src_hash = hashlib.sha256(src.read_bytes()).hexdigest()
        dest_hash = hashlib.sha256(dest.read_bytes()).hexdigest()
        if src_hash != dest_hash:
            dest.unlink()
            raise IOError(f"Checksum mismatch: {src} -> {dest}")
    return dest

def safe_move(src, dest, *, overwrite=False):
    """Move a file, falling back to copy+delete across filesystems."""
    src, dest = Path(src), Path(dest)
    if not src.exists():
        raise FileNotFoundError(f"Source not found: {src}")
    if dest.exists() and not overwrite:
        raise FileExistsError(f"Destination exists: {dest}")
    dest.parent.mkdir(parents=True, exist_ok=True)

    try:
        shutil.move(str(src), str(dest))
    except shutil.Error:
        safe_copy(src, dest, overwrite=overwrite, verify=True)
        src.unlink()
    return dest

def batch_copy(src_dir, dest_dir, pattern="*", *, overwrite=False):
    """Copy all files matching a pattern from src_dir to dest_dir."""
    src_dir, dest_dir = Path(src_dir), Path(dest_dir)
    dest_dir.mkdir(parents=True, exist_ok=True)
    copied = []
    for file in src_dir.glob(pattern):
        if file.is_file():
            copied.append(safe_copy(file, dest_dir / file.name, overwrite=overwrite))
    return copied

JavaScript

const fs = require('fs').promises;
const path = require('path');
const crypto = require('crypto');

async function sha256(file) {
  const data = await fs.readFile(file);
  return crypto.createHash('sha256').update(data).digest('hex');
}

async function copyWithChecksum(src, dest) {
  // COPYFILE_FICLONE attempts a copy-on-write clone when supported
  await fs.copyFile(src, dest, fs.constants.COPYFILE_FICLONE);
  if (await sha256(src) !== await sha256(dest)) {
    await fs.unlink(dest);
    throw new Error(`Checksum mismatch: ${src} -> ${dest}`);
  }
}

async function moveWithFallback(src, dest) {
  try {
    await fs.rename(src, dest);  // atomic when same filesystem
  } catch (err) {
    if (err.code === 'EXDEV') {
      const stat = await fs.stat(src);
      if (stat.isDirectory()) {
        await fs.cp(src, dest, { recursive: true });  // Node 16.7+
        await fs.rm(src, { recursive: true });
      } else {
        await copyWithChecksum(src, dest);
        await fs.unlink(src);
      }
    } else {
      throw err;
    }
  }
}

Java

import java.nio.file.*;
import java.security.MessageDigest;
import java.util.ArrayList;
import java.util.List;

public class FileCopier {

    public static void copyWithAttributes(Path src, Path dest, boolean overwrite) throws Exception {
        List<CopyOption> options = new ArrayList<>();
        options.add(StandardCopyOption.COPY_ATTRIBUTES);
        if (overwrite) options.add(StandardCopyOption.REPLACE_EXISTING);
        Files.copy(src, dest, options.toArray(new CopyOption[0]));
    }

    public static void moveWithFallback(Path src, Path dest, boolean overwrite) throws Exception {
        List<CopyOption> options = new ArrayList<>();
        if (overwrite) options.add(StandardCopyOption.REPLACE_EXISTING);

        try {
            // ATOMIC_MOVE only works within the same filesystem
            options.add(StandardCopyOption.ATOMIC_MOVE);
            Files.move(src, dest, options.toArray(new CopyOption[0]));
        } catch (AtomicMoveNotSupportedException e) {
            options.remove(StandardCopyOption.ATOMIC_MOVE);
            copyWithAttributes(src, dest, overwrite);
            Files.deleteIfExists(src);
        }
    }

    public static String sha256(Path file) throws Exception {
        MessageDigest md = MessageDigest.getInstance("SHA-256");
        byte[] hash = md.digest(Files.readAllBytes(file));
        StringBuilder sb = new StringBuilder();
        for (byte b : hash) sb.append(String.format("%02x", b));
        return sb.toString();
    }
}

Bash

#!/usr/bin/env bash
set -euo pipefail

safe_copy() {
    local src="$1"
    local dest="$2"
    local overwrite="${3:-false}"

    [[ -f "$src" ]] || { echo "ERROR: Source not found: $src"; return 1; }

    if [[ -f "$dest" && "$overwrite" != "true" ]]; then
        echo "ERROR: Destination exists: $dest"
        return 1
    fi

    mkdir -p "$(dirname "$dest")"
    cp -p "$src" "$dest"

    local src_sum dest_sum
    src_sum=$(sha256sum "$src" | cut -d' ' -f1)
    dest_sum=$(sha256sum "$dest" | cut -d' ' -f1)

    if [[ "$src_sum" != "$dest_sum" ]]; then
        rm -f "$dest"
        echo "ERROR: Checksum mismatch after copy"
        return 1
    fi

    echo "OK: $src -> $dest (verified)"
}

batch_copy() {
    local src_dir="$1"
    local dest_dir="$2"
    local pattern="${3:-*}"

    mkdir -p "$dest_dir"
    local count=0
    for file in "$src_dir"/$pattern; do
        [[ -f "$file" ]] || continue
        safe_copy "$file" "$dest_dir/$(basename "$file")" true && count=$((count + 1))
    done
    echo "Copied $count files"
}

Explanation

flowchart diagram: Source File

A copy duplicates content and optionally metadata. A move on the same filesystem is a quick, atomic rename of the inode. Cross-device moves have to copy the bytes first and then delete the source; if something fails in the middle, you can end up with a partial file or a duplicate.

The Java ATOMIC_MOVE flag and Node’s fs.rename only guarantee atomicity when the source and destination live on the same filesystem. For critical writes, write a temp file in the destination directory, then rename it over the target so readers never see a partial file.

Variants

TechnologyApproachBest For
Pythonshutil + pathlibCross-platform scripts and data pipelines
JavaScriptfs.promises / fs.cpNode.js tooling and CI scripts
Javajava.nio.file.FilesProduction services that need typed options
Bashcp, mv, sha256sumQuick sysadmin tasks and cron jobs

Other useful variants include sendfile/copy_file_range for kernel-level copies on Linux, fs-extra for filtered recursive copies in Node.js, and Apache Commons IO FileUtils for higher-level Java batch helpers.

Best Practices

  • Check overwrites with COPYFILE_EXCL or an explicit overwrite flag so you don’t replace data silently.
  • For critical files, write a temp file in the same directory and then move it over the final name so a crash never leaves a partial file.
  • Verify checksums on large files, network copies, or anything where corruption would cost you.
  • Create parent directories before you write, or you’ll hit a “No such file or directory” error.
  • Decide your symlink policy up front: follow links for backups, copy the link itself to preserve the structure.
  • Handle EXDEV/cross-device moves with a copy-and-delete fallback.

Common Mistakes

  • Overwriting existing files before you confirm or make a backup.
  • Assuming a move is atomic when the source and destination live on different filesystems or partitions.
  • Building paths with raw string concatenation ("/" + folder + "/" + name) instead of pathlib, path.join, or Path.resolve. This breaks on Windows because of backslash separators.
  • Ignoring symbolic links so a backup captures the link instead of the content, or the content instead of the link.
  • Moving files that another process is still writing to.
  • Forgetting to create the destination directory and getting a cryptic “No such file or directory” error.

Summary

A move on the same filesystem is a quick atomic rename; a cross-device move is copy-then-delete and can fail halfway. Always run SHA-256 on large or critical copies, and if the hash doesn’t match, delete the destination and retry. Handle EXDEV (Node) and AtomicMoveNotSupportedException (Java) with a copy-and-delete fallback. For all-or-nothing updates, write a temp file in the destination directory, then rename it over the target so readers never see a partial file. Decide your symlink policy up front: follow links for backups, copy the link itself to preserve structure. Create parent directories before writing, or you’ll hit a cryptic “No such file or directory” error. For large files, kernel-level copies (sendfile, copy_file_range, COPYFILE_FICLONE) beat chunked loops.

See Also

Frequently Asked Questions

Is move always atomic?

A move is only atomic on the same filesystem, and cross-device moves are copy-then-delete operations that can be interrupted. When readers need an all-or-nothing update, write a temp file and rename it into place.

How do I copy directories recursively?

In Python, use shutil.copytree(). In JavaScript, use fs.cp(src, dest, { recursive: true }) (Node 16.7+) or fs-extra.copy(). In Java, use Files.walkFileTree() or Apache Commons IO FileUtils.copyDirectory(). In Bash, cp -r is fine for quick copies, but rsync -a preserves permissions and can resume interrupted transfers.

Should I follow symlinks when copying?

The short answer: it depends on whether you're doing a backup or preserving structure. For backups, follow symlinks so you capture the actual content rather than a dangling reference. To preserve the exact directory structure, copy the symlink itself. Python, Java, and Node all expose flags for this.

What should I do if a checksum fails?

Delete the destination copy, check the source for corruption, and retry. If the hash doesn't match, the destination is corrupt and keeping it around wastes disk space, so start over.

How do I handle permission errors?

Permission errors are the most common copy failure I see in production. The fix is boring but reliable: before you start, check that the source file is readable and the destination folder is writable. Python raises PermissionError, Node sets err.code === 'EACCES', and Java throws AccessDeniedException. If you're copying into a shared directory, make sure the target user owns the path or has group write access. Don't run as root, though, because it masks permission issues that will surface later in production, so run the script as the service user instead.

What's the fastest way to copy large files?

Once you're past 1 GB, the checksum loop becomes the bottleneck, not the copy itself. Skip it and lean on kernel-level calls. On Linux, sendfile or copy_file_range bypass userspace entirely. Node's fs.copyFile with COPYFILE_FICLONE tries copy-on-write on btrfs or XFS reflink, which costs nothing. Python's shutil.copy2 falls back to a chunked read/write loop, which is slower. For batches of large files, rsync -a --checksum is hard to beat because it skips files that already match.