Production Logging in Python, JavaScript and Java

Overview

When production breaks at 3 AM, logs are usually the only evidence you’ve got. Logging captures what your application did — events, errors, state changes — and it’s often the only way to reconstruct an incident after the fact.

A production-grade setup is structured (JSON, not free text), level-based (DEBUG through FATAL), and carries context such as timestamps, request IDs, and user IDs — while keeping secrets and personal data out. This recipe covers working setups in Python, JavaScript, and Java, plus the operational pieces that make logs actually useful: rotation, aggregation, alerting, and redaction.

When to Use

Use this recipe when:

• You need to debug a production issue you can’t reproduce locally.
• You need one place to watch errors, latency, and business events.
• You need audit trails of user actions for security reviews or compliance. See the Security Guide for audit requirements.
• You need to trace one request across several services with a correlation ID. For audit-grade request/response capture, see API Logging and Audit Trails.
• You plan to build dashboards and alerts on top of log data.

Skip the structured pipeline for a throwaway script or a one-off migration — print is fine there. Invest in real logging once the code runs unattended.

Solution

Python (Loguru)

from loguru import logger
import sys

logger.remove()
# Human-readable output for local development
logger.add(sys.stdout, format="{time:HH:mm:ss} | {level} | {message}", level="INFO")
# JSON lines with rotation and async writes for production collectors
logger.add("app.log", serialize=True, rotation="10 MB", retention="7 days",
           level="DEBUG", enqueue=True)

logger.debug("Processing user {}", user_id)
logger.info("User {} logged in", user_id)
logger.warning("Rate limit at 90% for API key {}", api_key[:4])

try:
    raise ConnectionError("database timeout")
except ConnectionError:
    logger.exception("Database connection failed")

# Attach context that travels with every subsequent log
logger.bind(request_id="abc-123", duration_ms=45).info("Request completed")

serialize=True writes one JSON object per line, which Filebeat, Promtail, or the CloudWatch agent can parse without custom grok rules. enqueue=True moves the disk write off the request thread — see How It Works for why that matters.

JavaScript (Winston)

const winston = require('winston');

const logger = winston.createLogger({
  level: process.env.LOG_LEVEL || 'info',
  format: winston.format.combine(
    winston.format.timestamp(),
    winston.format.errors({ stack: true }),
    winston.format.json()
  ),
  transports: [
    new winston.transports.Console(),
    new winston.transports.File({ filename: 'app.log', maxsize: 10_000_000, maxFiles: 5 }),
  ],
});

logger.debug('Processing user', { userId });
logger.info('User logged in', { userId });
logger.warn('Rate limit at 90%', { apiKey: apiKey.slice(0, 4) });
logger.error('Database connection failed', { error });

Metadata passed as the second argument lands as JSON fields, so requestId becomes queryable instead of being buried in the message text.

Java (SLF4J + Logback)

import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
import org.slf4j.MDC;

public class UserService {
    private static final Logger log = LoggerFactory.getLogger(UserService.class);

    public void login(String userId) {
        MDC.put("requestId", RequestContext.currentRequestId());
        try {
            log.info("User {} logged in", userId);
            // ...
        } catch (Exception e) {
            log.error("Login failed for user {}", userId, e);
        } finally {
            MDC.clear();
        }
    }
}

{} placeholders defer string concatenation until the level check passes, so a disabled DEBUG statement costs almost nothing. MDC stores the correlation ID in a thread-local map that Logback prints on every line via %X{requestId} in the encoder pattern.

Log Levels

Level	When to Use	Example
DEBUG	Detailed diagnostic info	Variable values, loop iterations
INFO	Normal application events	Requests processed, jobs completed
WARN	Recoverable anomalies	Deprecated API called, rate limit near
ERROR	Failed operations	Database timeout, file not found
FATAL/CRITICAL	System unusable	Out of memory, disk full

Run production at INFO by default and enable DEBUG per-service or per-request behind a flag. An app that logs everything at DEBUG will drown the signal and can easily multiply your storage bill.

How It Works

Every log call travels the same short pipeline:

The step that hurts in production is the write. A synchronous file append can block the request thread on disk I/O, which is why each framework ships an async path: enqueue=True in Loguru, AsyncAppender in Logback, and async transports in Winston. If you run in containers, prefer logging JSON to stdout and let the platform collect it — no files, no rotation to manage.

Variants

Language	Library	Pick it when
Python	logging (stdlib)	You want zero dependencies and configure handlers yourself
Python	structlog	You want JSON-first structured logs — see Structured JSON Logging with structlog
Python	loguru	You want rotation, retention, and async out of the box (used above)
Node.js	pino	Throughput matters; it’s JSON-first and fast — see High-Performance Logging with pino
Node.js	winston	You need many transports and flexible formats (used above)
Node.js	winston-daily-rotate-file	You rotate by date — see Rotate Logs Daily with Winston
JVM	SLF4J + Logback	The de facto standard; Log4j2 with async loggers for high throughput

For a deeper look at JSON output and correlation fields, see Structured Logging. For shipping to ELK, Loki, or Datadog, see Log Aggregation.

Correlation IDs

A correlation (or request) ID lets you pull every log line for one request out of millions. Generate it at the entry point — an HTTP middleware is the usual place — and attach it to the logger context for the whole request.

• Python: logger.bind(request_id=...) or a contextvars variable read by a formatter.
• Node.js: a child logger (logger.child({ requestId })) created per request.
• Java: MDC.put("requestId", ...) plus %X{requestId} in the Logback pattern, as shown above.

Across services, pass it in a header (X-Request-ID, or the W3C traceparent if you already run OpenTelemetry) so downstream logs join the same trace.

Sensitive Data in Logs

Logs get shipped to third-party aggregators and read by more people than your database. Treat them as semi-public:

• Never log passwords, session tokens, full API keys, or card numbers. Mask to the first 4 characters at most.
• Keep PII (emails, names, IPs) out of extra fields when privacy rules apply, or document them explicitly in your data map.
• Redact at the boundary, before the string reaches the logger:

import re

SENSITIVE = re.compile(r"(token|api_key|password)=\S+", re.IGNORECASE)

def mask(msg: str) -> str:
    return SENSITIVE.sub(r"\1=***", msg)

logger.info(mask(f"fetched profile token={token}"))

The same applies to exceptions: a stack trace can contain connection strings or query parameters. Log the exception object, but scrub the message first if it embeds user input.

What Works

• Emit JSON in production. Plain text is for humans at a terminal; aggregators need fields. Keep a pretty format for local dev and JSON for deploys.
• Attach a correlation ID to every log line in a request. Without it, cross-service debugging is guesswork.
• Log at the right level. DEBUG for dev detail, INFO for normal ops, WARN for anomalies, ERROR for failures that need attention.
• Rotate files. Size- or time-based rotation prevents a busy service from filling the disk.
• Log the exception object, not just the message. logger.exception / { error } / passing e keeps the stack trace.
• Control verbosity with LOG_LEVEL. Reading the level from the environment lets you crank up detail during an incident without redeploying.

Common Mistakes

• Logging everything at INFO, so real problems hide in noise.
• Using print or console.log in production instead of a logging framework.
• Logging raw secrets or PII. See Sensitive Data in Logs and the Security Guide.
• No rotation: one chatty endpoint can fill the disk in hours.
• Catching an exception and logging only str(e), losing the stack trace that tells you where it failed.
• F-string or + concatenation in hot paths: the string is built even when the level is disabled. Use lazy {} or %s placeholders.

Log Aggregation and Monitoring

Raw log files are rarely read directly in production. A shipper forwards them to an aggregator:

Platform	Best for	Shipping method
ELK Stack	Self-hosted, full control	Filebeat / Logstash
Datadog	SaaS with APM integration	Datadog Agent
AWS CloudWatch	AWS-native infrastructure	CloudWatch Agent
Grafana Loki	Kubernetes / Prometheus stack	Promtail or Alloy
Splunk	Enterprise compliance	Universal Forwarder

Alerting rules

Base alerts on log patterns, not just metrics:

• ERROR rate above 1% over a 5-minute window → page the on-call.
• Any FATAL log → immediate alert.
• Log volume drops to zero for 10 minutes → the service may be dead even if health checks pass.

Dashboards

Useful dashboards answer concrete questions: requests per minute, p95 latency, which endpoints produce the most errors, and how the error count trends over 24 hours.

Further Reading

• Loguru documentation — the Python logger used above.
• Winston on GitHub — transports, formats, and custom levels.
• SLF4J manual — MDC, markers, and parameterized logging.
• OWASP Logging Cheat Sheet — what to log for security events and what to exclude.
• Companion repo — the Loguru, Winston, and SLF4J examples from this page as runnable files with their dependencies.