Watch File Changes

Overview

File system watchers let your program react the moment a file is created, modified, deleted, or renamed. No polling loop burning CPU on stat() calls. Every OS exposes a kernel-level notification API: inotify on Linux, FSEvents on macOS, and ReadDirectoryChangesW on Windows. The libraries in this recipe wrap those APIs so events get pushed to your process instead of you going looking for them.

Below is the idiomatic watcher for Python, JavaScript, and Java, plus production-grade versions with debouncing, recursive registration, and the error handling that the minimal examples leave out.

When to Use

Reach for a watcher when:

• You’re building dev tooling — hot-reload servers, build-on-save, test reruns.
• A directory acts as a drop zone: file uploads, exports, or incoming logs that should trigger a pipeline.
• You need near-instant reaction; a cron job or a scan loop every N seconds is either too slow or too expensive.

Skip it when:

• The directory lives on a network share (NFS, SMB, SSHFS). Most network filesystems don’t propagate kernel events between machines, so you’ll need polling instead.
• The tree is huge. Recursive watches consume one handle per directory, and limits like Linux’s fs.inotify.max_user_watches will fail loudly (or worse, drop events) once exceeded.
• You’re inside a container. Docker bind mounts don’t forward host filesystem events to the guest on all setups, so watchers often appear dead inside containers.

Solution

Python — watchdog

watchdog is the standard cross-platform watcher for Python. It picks the best backend per OS automatically.

import time
from watchdog.observers import Observer
from watchdog.events import FileSystemEventHandler

class Handler(FileSystemEventHandler):
    def on_modified(self, event):
        if not event.is_directory:
            print(f"Modified: {event.src_path}")

observer = Observer()
observer.schedule(Handler(), path='./watched', recursive=True)
observer.start()

try:
    while True:
        time.sleep(1)
except KeyboardInterrupt:
    observer.stop()
observer.join()

JavaScript — fs.watch

Node’s built-in fs.watch needs no dependency, but treat it as a thin wrapper: recursive only works on Windows and macOS in older Node versions (Linux support landed in Node 20), and the event names it reports vary by platform. For anything beyond a quick script, use chokidar; the advanced example below shows why.

const fs = require('fs');

// Watch a file or directory
// NOTE: { recursive: true } requires Node 20+ on Linux
const watcher = fs.watch('./watched', { recursive: true }, (eventType, filename) => {
    console.log(`${eventType}: ${filename}`);
});

// Cleanup
process.on('SIGINT', () => watcher.close());

Java — WatchService

Java NIO’s WatchService has been in the JDK since Java 7. It maps to the same native APIs; see the official Oracle tutorial for the full contract.

import java.nio.file.*;

public class FileWatcher {
    public static void watch(Path path) throws Exception {
        WatchService watchService = FileSystems.getDefault().newWatchService();
        path.register(watchService,
                StandardWatchEventKinds.ENTRY_CREATE,
                StandardWatchEventKinds.ENTRY_MODIFY,
                StandardWatchEventKinds.ENTRY_DELETE);

        while (true) {
            WatchKey key = watchService.take();
            for (WatchEvent<?> event : key.pollEvents()) {
                if (event.kind() == StandardWatchEventKinds.OVERFLOW) {
                    continue; // events were lost; rescan the directory
                }
                System.out.println(event.kind() + ": " + event.context());
            }
            if (!key.reset()) break; // key no longer valid
        }
    }
}

Explanation

A watcher registers interest with the kernel once, and the kernel pushes events to your process. That’s why it’s cheap compared to polling: no syscalls until something actually happens.

Three details matter more than the happy path:

• Events can be lost. Every kernel API has a bounded queue. If your process is slow to drain it, Linux reports IN_Q_OVERFLOW and Java reports OVERFLOW. Always handle that event, usually by rescanning the directory, or you’ll silently miss changes.
• Editors write atomically. Many editors save by writing a temp file and renaming it over the original. What looks like one “save” arrives as create + delete + rename on some platforms, so a naive handler fires several times per keystroke-save. That’s what the debounce logic in the advanced examples is for.
• Watches don’t survive the boundary. Network filesystems and container bind mounts break the “kernel pushes to you” model because the events happen in a different kernel. On those, the only reliable option is polling: chokidar supports it via usePolling: true and Java has FileAlterationMonitor.

Variants

Technology	Approach	Notes
Python	watchdog	Cross-platform, handles edge cases like rapid renames
Python	watchfiles	Rust core (notify-rs), async-friendly, powers uvicorn --reload
JavaScript	chokidar	More reliable than fs.watch; normalizes events and supports polling
Java	Apache Commons IO FileAlterationMonitor	Polling fallback for older JDKs or network drives
Linux/Bash	inotifywait	Raw inotify from shell scripts; see the advanced example

What Works

1. Debounce rapid events — editors and build tools often trigger several writes per logical save.
2. Always handle the error event, OVERFLOW, and WatchService exceptions; a silent watcher is worse than no watcher.
3. Use recursive watches sparingly; each directory costs a handle.
4. Filter by file extension to ignore temp files (.tmp, .swp, editor lock files).
5. Run watchers on a dedicated thread or process so a slow handler can’t stall event delivery. If you’re processing large files or streams in the handler, hand the path off to a queue and return immediately.

Common Mistakes

1. Assuming modify fires once per save: editors may trigger many events, or atomic-write sequences.
2. Not cleaning up watchers on shutdown, which leaks file descriptors and eventually hits max_user_watches.
3. Watching network drives with native APIs that can’t see remote changes.
4. Ignoring rename/moved events — they arrive as separate create + delete on some OSes.
5. Processing a file the moment create fires, before the writer has closed it. Wait for modify to stop arriving, or use chokidar’s awaitWriteFinish.

Advanced Solutions

Python: debounced watcher with event coalescing

Rapid editor saves collapse into one callback after a quiet period. Extension filtering happens before the debounce so temp files never schedule a timer.

import time
import threading
from watchdog.observers import Observer
from watchdog.events import FileSystemEventHandler
from typing import Callable

class DebouncedEventHandler(FileSystemEventHandler):
    """Coalesces rapid file events into a single callback after a quiet period."""

    def __init__(self, callback: Callable[[str, str], None],
                 debounce_seconds: float = 0.3,
                 extensions: list[str] | None = None):
        self.callback = callback
        self.debounce = debounce_seconds
        self.extensions = extensions or []
        self._pending: dict[str, dict] = {}
        self._lock = threading.Lock()
        self._timer: threading.Timer | None = None

    def _should_process(self, path: str) -> bool:
        if not self.extensions:
            return True
        return any(path.endswith(ext) for ext in self.extensions)

    def _on_event(self, event_type: str, src_path: str):
        if not self._should_process(src_path):
            return
        with self._lock:
            self._pending[src_path] = {
                "type": event_type,
                "time": time.time(),
            }
            if self._timer:
                self._timer.cancel()
            self._timer = threading.Timer(self.debounce, self._flush)
            self._timer.start()

    def _flush(self):
        with self._lock:
            for path, info in self._pending.items():
                self.callback(info["type"], path)
            self._pending.clear()

    def on_created(self, event):
        if not event.is_directory:
            self._on_event("created", event.src_path)

    def on_modified(self, event):
        if not event.is_directory:
            self._on_event("modified", event.src_path)

    def on_deleted(self, event):
        if not event.is_directory:
            self._on_event("deleted", event.src_path)

    def on_moved(self, event):
        if not event.is_directory:
            self._on_event("moved", event.dest_path)

def handle_change(event_type: str, path: str):
    print(f"[{event_type}] {path}")

observer = Observer()
handler = DebouncedEventHandler(
    callback=handle_change,
    debounce_seconds=0.3,
    extensions=[".py", ".js", ".json", ".yaml"],
)
observer.schedule(handler, path="./src", recursive=True)
observer.start()

try:
    while True:
        time.sleep(1)
except KeyboardInterrupt:
    observer.stop()
observer.join()

JavaScript: chokidar with filtering and stable-write detection

Chokidar normalizes the quirks fs.watch leaves to you — including awaitWriteFinish, which waits until a file stops changing before reporting it. That alone removes most “read a half-written file” bugs.

const chokidar = require('chokidar');

const watcher = chokidar.watch('./src', {
    ignored: /(^|[\/\\])\./,  // ignore dotfiles
    persistent: true,
    ignoreInitial: true,
    followSymlinks: false,
    awaitWriteFinish: {
        stabilityThreshold: 500,
        pollInterval: 100,
    },
});

const debounce = new Map();
function debouncedRun(file, fn, delay = 300) {
    if (debounce.has(file)) clearTimeout(debounce.get(file));
    debounce.set(file, setTimeout(() => {
        fn(file);
        debounce.delete(file);
    }, delay));
}

watcher
    .on('add', file => debouncedRun(file, f => {
        if (f.endsWith('.csv')) processCSV(f);
    }))
    .on('change', file => debouncedRun(file, f => {
        if (f.endsWith('.js')) rebuildBundle(f);
        if (f.endsWith('.css')) recompileStyles(f);
    }))
    .on('unlink', file => {
        console.log(`Deleted: ${file}`);
        cleanupCache(file);
    })
    .on('error', err => console.error('Watcher error:', err))
    .on('ready', () => console.log('Initial scan complete. Watching for changes...'));

function processCSV(file) { console.log(`Processing CSV: ${file}`); }
function rebuildBundle(file) { console.log(`Rebuilding: ${file}`); }
function recompileStyles(file) { console.log(`Recompiling CSS: ${file}`); }
function cleanupCache(file) { console.log(`Cleaning cache for: ${file}`); }

// Cleanup on exit
process.on('SIGINT', () => watcher.close().then(() => process.exit(0)));

Java: recursive watch with thread pool

WatchService only watches the directories you register, so recursion means walking the tree up front and registering new directories as they’re created. The OVERFLOW check isn’t optional: under load the kernel drops events, and that’s your signal to rescan.

import java.io.IOException;
import java.nio.file.*;
import java.nio.file.attribute.BasicFileAttributes;
import java.util.concurrent.*;
import java.util.concurrent.atomic.AtomicBoolean;

public class RecursiveWatcher {

    private final WatchService watchService;
    private final ExecutorService executor;
    private final AtomicBoolean running = new AtomicBoolean(true);
    private final ConcurrentHashMap<WatchKey, Path> keys = new ConcurrentHashMap<>();

    public RecursiveWatcher() throws Exception {
        this.watchService = FileSystems.getDefault().newWatchService();
        this.executor = Executors.newSingleThreadExecutor();
    }

    public void registerAll(Path start) throws Exception {
        Files.walkFileTree(start, new SimpleFileVisitor<>() {
            @Override
            public FileVisitResult preVisitDirectory(Path dir, BasicFileAttributes attrs) throws IOException {
                WatchKey key = dir.register(watchService,
                    StandardWatchEventKinds.ENTRY_CREATE,
                    StandardWatchEventKinds.ENTRY_MODIFY,
                    StandardWatchEventKinds.ENTRY_DELETE);
                keys.put(key, dir);
                return FileVisitResult.CONTINUE;
            }
        });
    }

    public void start() {
        executor.submit(() -> {
            while (running.get()) {
                try {
                    WatchKey key = watchService.poll(1, TimeUnit.SECONDS);
                    if (key == null) continue;

                    Path dir = keys.get(key);
                    if (dir == null) {
                        key.reset();
                        continue;
                    }
                    for (WatchEvent<?> event : key.pollEvents()) {
                        if (event.kind() == StandardWatchEventKinds.OVERFLOW) {
                            continue; // events lost; rescan dir if exactness matters
                        }
                        Path fullPath = dir.resolve((Path) event.context());
                        System.out.println(event.kind() + ": " + fullPath);

                        // Auto-register new subdirectories
                        if (event.kind() == StandardWatchEventKinds.ENTRY_CREATE
                                && Files.isDirectory(fullPath)) {
                            registerAll(fullPath);
                        }
                    }
                    if (!key.reset()) {
                        keys.remove(key); // directory no longer accessible
                    }
                } catch (InterruptedException e) {
                    Thread.currentThread().interrupt();
                    break;
                } catch (Exception e) {
                    e.printStackTrace();
                }
            }
        });
    }

    public void stop() throws Exception {
        running.set(false);
        executor.shutdown();
        executor.awaitTermination(5, TimeUnit.SECONDS);
        watchService.close();
    }
}

// Usage
// RecursiveWatcher watcher = new RecursiveWatcher();
// watcher.registerAll(Path.of("./src"));
// watcher.start();
// Runtime.getRuntime().addShutdownHook(new Thread(watcher::stop));

Bash: inotifywait for simple watching

Linux-only, no library needed — inotify-tools exposes inotify directly. The debounce below is a simplified single-machine approach; for anything concurrent use a real lock or a proper watcher.

#!/usr/bin/env bash
set -euo pipefail

# Watch a directory with inotifywait (Linux only)
# Requires: apt install inotify-tools

WATCH_DIR="${1:-./watched}"
DEBOUNCE_SECONDS=0.3

echo "Watching: $WATCH_DIR"

inotifywait -m -r --format '%w%f|%e' \
    -e create,modify,delete,move \
    --exclude '\.(swp|tmp|log)' \
    "$WATCH_DIR" | while IFS='|' read -r file event; do
        # Debounce: skip if same file+event seen recently (simplified)
        CACHE_KEY="${event}:${file}"
        if [[ -f /tmp/.watch_cache ]] && grep -q "^${CACHE_KEY}$" /tmp/.watch_cache 2>/dev/null; then
            continue
        fi
        echo "${CACHE_KEY}" >> /tmp/.watch_cache
        sleep "$DEBOUNCE_SECONDS"
        sed -i "/^${CACHE_KEY//\//\\/}$/d" /tmp/.watch_cache 2>/dev/null || true

        echo "[$(date +%H:%M:%S)] $event: $file"

        # Trigger action based on extension
        case "$file" in
            *.py) echo "  -> Python file changed, running lint..." ;;
            *.js) echo "  -> JS file changed, rebuilding bundle..." ;;
            *.csv) echo "  -> CSV file added, processing..." ;;
        esac
    done

Companion code: The watch-file-changes companion repo has all four advanced examples as runnable files.