StackPractices
intermediate By Mathias Paulenko

Merge JSON Files in JavaScript

Combine multiple JSON files in Node.js with conflict resolution, ESM/CJS variants, and streaming for large inputs.

Topics: data

Overview

Node.js projects split JSON config across files (one per environment, per module, per locale) and merge them at startup or build time. This recipe is the Node.js deep dive: fs-based multi-file merging, a hardened deepMerge, ESM and async variants, streaming for files too big for memory, and validation of the merged output. For the multi-language version (Python, Java, jq), see Merge JSON Files.

When to Use

  • Combining config/default.json with config/production.json at app startup
  • Aggregating paginated API responses into one payload
  • Merging user settings over defaults without losing nested keys
  • Building a merged locale or dataset file in a build step

If your inputs aren’t JSON yet, convert them first, for example with Convert CSV to JSON. To check what a merge actually changed, run Diff JSON Objects on the output.

Solution

Shallow merge

// CommonJS — top-level spread or Object.assign, flat configs only
const defaults = require("./config/default.json");
const prod = require("./config/production.json");

const config = { ...defaults, ...prod };
// prod keys replace defaults keys at the top level only

Spread replaces nested objects whole: {db: {host, port}} in defaults is gone entirely if prod defines {db: {port}}. Fine for flat files, wrong for nested config.

Deep merge (dependency-free)

const isPlainObject = (v) =>
  v !== null && typeof v === "object" && !Array.isArray(v);
const UNSAFE_KEYS = new Set(["__proto__", "constructor", "prototype"]);

function deepMerge(target, source) {
  for (const key of Object.keys(source)) {
    if (UNSAFE_KEYS.has(key)) continue; // prototype-pollution guard
    if (isPlainObject(source[key]) && isPlainObject(target[key])) {
      deepMerge(target[key], source[key]);
    } else {
      target[key] = isPlainObject(source[key])
        ? deepMerge({}, source[key])
        : source[key];
    }
  }
  return target;
}

Three details matter here. typeof null === "object", so the isPlainObject guard keeps a null override from reaching Object.keys and throwing. UNSAFE_KEYS blocks {"__proto__": {...}} from mutating Object.prototype, a real attack when merging user-supplied JSON. And the deepMerge({}, source[key]) call means merged branches are fresh objects, so the sources stay untouched.

Merge a directory of JSON files

const fs = require("fs");
const path = require("path");

function mergeJsonDir(dir) {
  return fs
    .readdirSync(dir)
    .filter((f) => f.endsWith(".json"))
    .sort() // deterministic: later files win
    .reduce((acc, file) => {
      const content = JSON.parse(fs.readFileSync(path.join(dir, file), "utf-8"));
      return deepMerge(acc, content);
    }, {});
}

const config = mergeJsonDir("./config");

.sort() isn’t decorative: readdirSync returns entries in filesystem order, which differs across machines. Sorting makes “last file wins” predictable in CI and on your teammates’ boxes.

Node.js merge pipeline: read sorted JSON files, parse each one, deep-merge in order, dedupe arrays, validate against schema, write output.

Custom conflict resolution

const _ = require("lodash");

function mergeWithResolver(sources, resolver) {
  const result = {};
  for (const source of sources) {
    for (const key of Object.keys(source)) {
      if (key in result && !_.isEqual(result[key], source[key])) {
        result[key] = resolver(key, result[key], source[key]);
      } else {
        result[key] = source[key];
      }
    }
  }
  return result;
}

// Last value wins, but every conflict gets logged
const merged = mergeWithResolver(
  [base, staging, production],
  (key, oldVal, newVal) => {
    console.warn(`Conflict on "${key}": ${JSON.stringify(oldVal)} -> ${JSON.stringify(newVal)}`);
    return newVal;
  }
);

_.isEqual does the deep comparison the naive version skipped; without it the resolver only ever sees top-level primitives.

Explanation

A merge is two decisions: how deep to combine, and who wins when keys collide. Shallow merge answers “replace everything below the top level”; deep merge walks the tree. Conflict strategy is the second axis: last-write-wins is what deepMerge above does implicitly; the resolver version makes it a choice you can log, veto, or turn into an error for strict CI checks.

Arrays deserve a special callout: deepMerge replaces them (the override’s list wins). For data merges you usually want concatenation plus deduplication by a stable key. Lodash _.mergeWith handles that with a customizer.

Two Node gotchas hide in the file-reading step. require("./x.json") caches its parsed result in the module registry, so editing the file won’t be picked up until restart; fs.readFileSync + JSON.parse re-reads every time, which is what hot-reload workflows expect. And JSON.parse throws on a UTF-8 BOM; files saved by some Windows editors carry one, so strip a leading \uFEFF before parsing if you can’t control how the inputs are produced.

Variants

ESM with JSON modules and async fs

// ESM (node >= 20.10 for import attributes)
import defaults from "./config/default.json" with { type: "json" };
import fs from "node:fs/promises";

async function mergeJsonDirAsync(dir) {
  const files = (await fs.readdir(dir))
    .filter((f) => f.endsWith(".json"))
    .sort();
  let merged = {};
  for (const file of files) {
    const content = JSON.parse(await fs.readFile(new URL(`${dir}/${file}`, import.meta.url), "utf-8"));
    merged = deepMerge(merged, content);
  }
  return merged;
}

await at module top level is legal ESM; in CommonJS wrap it in an async function main(). JSON import ... with { type: "json" } is the cleanest option for a fixed set of files (static analysis and bundlers can see it), but it can’t list a directory, so fs stays necessary for glob-style merging. Mixing both worlds works too: import the defaults statically, then deep-merge the environment-specific files you discovered with fs on top.

Arrays: concatenate instead of replace

const merged = _.mergeWith({}, a, b, (objVal, srcVal) => {
  if (Array.isArray(objVal) && Array.isArray(srcVal)) {
    return objVal.concat(srcVal);
  }
  // undefined → fall back to default merge behavior
});

Streaming with JSONStream for files that don’t fit in memory

const fs = require("fs");
const JSONStream = require("JSONStream");

// Merge huge array files without loading them whole
function streamMergeArrays(inputs, output) {
  const ws = fs.createWriteStream(output);
  ws.write("[");
  let first = true;
  let pending = inputs.length;
  for (const file of inputs) {
    fs.createReadStream(file)
      .pipe(JSONStream.parse("*"))
      .on("data", (item) => {
        ws.write((first ? "" : ",") + JSON.stringify(item));
        first = false;
      })
      .on("end", () => {
        if (--pending === 0) ws.end("]");
      });
  }
}

Streaming concatenates arrays only; deep object merging still needs the merged object in memory, because JSON gives no random access to keys. For 10k+ element datasets consider a worker thread so the merge doesn’t block the event loop; worker_threads runs the merge on a separate core and posts back the result.

Concat and dedupe arrays by a stable key

function mergeRecords(sources, key = "id") {
  const seen = new Map();
  for (const source of sources) {
    for (const item of source) {
      seen.set(item[key], item); // later files overwrite same-id records
    }
  }
  return [...seen.values()];
}

const users = mergeRecords([batch1, batch2, batch3], "id");

Map keeps insertion order, so the output order is stable and the last file to define an id wins the slot.

Tolerate a broken file instead of failing the whole merge

function mergeJsonDirSafe(dir) {
  const files = fs.readdirSync(dir).filter((f) => f.endsWith(".json")).sort();
  let merged = {};
  const skipped = [];
  for (const file of files) {
    try {
      merged = deepMerge(merged, JSON.parse(fs.readFileSync(path.join(dir, file), "utf-8")));
    } catch (err) {
      skipped.push({ file, error: err.message });
    }
  }
  return { merged, skipped };
}

One malformed file shouldn’t kill a batch merge. Collect the failures and surface them: silently skipping is worse than failing, since a dropped production.json looks exactly like a successful merge.

Validate the merged output with AJV

const Ajv = require("ajv");
const schema = require("./config.schema.json");

const merged = mergeJsonDir("./config");
const valid = new Ajv({ allErrors: true }).validate(schema, merged);
if (!valid) {
  console.error(ajv.errors);
  process.exit(1);
}

allErrors: true reports every violation in one pass, worth it in a build step where a merged config failing silently becomes a runtime incident.

In practice that validation belongs in CI, not at boot. Run the merge and the AJV check as a build step and fail the pipeline on any parse error or schema violation: node merge.js && node validate.js. The merge is deterministic and cheap, so gating on it costs seconds and removes a whole class of “worked locally” deploy surprises, where a broken production.json only blows up at runtime.

ApproachNestedArraysDepsUse when
{...a, ...b}NoReplaces—Flat configs
deepMerge (above)YesReplaces—Most config merges
_.mergeYesReplaceslodashBattled-tested default
_.mergeWithYesCustomizerlodashConcat/dedupe arrays
JSONStreamConcat onlyConcatJSONStreamFiles > memory

What Works

  • Deep-merge config, concat data: config trees merge deep; record lists concat + dedupe by a stable key (id)
  • Sort before merging: deterministic order makes “last file wins” reproducible across machines
  • Fail on conflicts in CI: swap the resolver to throw: a silent override in production.json masks real mistakes
  • Validate the output: one AJV pass on the merged object catches missing keys before runtime
  • Keep sources immutable: merge into {}, never into a required object: Node caches modules, so a mutated require result poisons every later import of that file

Common Mistakes

  • Spread on nested config: {...a, ...b} drops a’s nested keys silently
  • typeof as an object check: typeof null === "object" breaks naive recursive merges; guard null and Array explicitly
  • Skipping __proto__: unfiltered keys let an untrusted file pollute Object.prototype for the whole process
  • try/catch around required JSON: require("./x.json") throws on invalid JSON just like JSON.parse; same handling applies
  • Top-level await in CJS: legal only in ESM or inside async functions; a .js file with require is CJS regardless of the code’s looks
  • Mutating a required object: the module cache shares it process-wide; copy first

Further Reading

Frequently Asked Questions

Can I import a JSON file directly in Node.js?

Yes — import cfg from "./app.json" with { type: "json" } on Node ≥ 20.10 (older: assert instead of with). It parses once at module load and is ideal for fixed config files. For directories you still need fs.readdir + fs.readFile, since imports can't be dynamic file lists.

How do I merge JSON files asynchronously?

fs/promises plus sequential deepMerge, or Promise.all to read files in parallel and merge after. Keep the merge step sequential: it's CPU-bound anyway, so parallelizing it gains nothing.

Is deepMerge safe against prototype pollution?

Only if it filters __proto__, constructor, and prototype keys, like the version above. A bare recursive merge assigns obj["__proto__"], which mutates Object.prototype process-wide. Alternatively validate input JSON against a schema before merging.

Should I write my own deep merge or use lodash?

_.merge is battle-tested for null, arrays, Dates, and circular structures; reach for it unless the dependency budget is zero. If you roll your own, copy the guards from the example above; the null/array/__proto__ checks are the part naive versions get wrong.

How do I merge files too large for memory?

Concatenate arrays through JSONStream (above); object deep-merging can't stream, since it needs the whole merged tree. For very large object graphs, split the file by top-level key first with Flatten and Unflatten Objects or move the data into a real store.

Does deepMerge mutate its inputs?

The version above doesn't: merged branches are built into fresh objects. Watch out for variants that merge into target[key] in place. And never merge into a required JSON object, because Node's module cache would share the mutation with every consumer.