Merge JSON Files in JavaScript
Combine multiple JSON files in Node.js with conflict resolution, ESM/CJS variants, and streaming for large inputs.
Overview
Node.js projects split JSON config across files (one per environment, per module, per locale) and merge them at startup or build time. This recipe is the Node.js deep dive: fs-based multi-file merging, a hardened deepMerge, ESM and async variants, streaming for files too big for memory, and validation of the merged output. For the multi-language version (Python, Java, jq), see Merge JSON Files.
When to Use
- Combining
config/default.jsonwithconfig/production.jsonat app startup - Aggregating paginated API responses into one payload
- Merging user settings over defaults without losing nested keys
- Building a merged locale or dataset file in a build step
If your inputs aren’t JSON yet, convert them first, for example with Convert CSV to JSON. To check what a merge actually changed, run Diff JSON Objects on the output.
Solution
Shallow merge
// CommonJS — top-level spread or Object.assign, flat configs only
const defaults = require("./config/default.json");
const prod = require("./config/production.json");
const config = { ...defaults, ...prod };
// prod keys replace defaults keys at the top level only
Spread replaces nested objects whole: {db: {host, port}} in defaults is gone entirely if prod defines {db: {port}}. Fine for flat files, wrong for nested config.
Deep merge (dependency-free)
const isPlainObject = (v) =>
v !== null && typeof v === "object" && !Array.isArray(v);
const UNSAFE_KEYS = new Set(["__proto__", "constructor", "prototype"]);
function deepMerge(target, source) {
for (const key of Object.keys(source)) {
if (UNSAFE_KEYS.has(key)) continue; // prototype-pollution guard
if (isPlainObject(source[key]) && isPlainObject(target[key])) {
deepMerge(target[key], source[key]);
} else {
target[key] = isPlainObject(source[key])
? deepMerge({}, source[key])
: source[key];
}
}
return target;
}
Three details matter here. typeof null === "object", so the isPlainObject guard keeps a null override from reaching Object.keys and throwing. UNSAFE_KEYS blocks {"__proto__": {...}} from mutating Object.prototype, a real attack when merging user-supplied JSON. And the deepMerge({}, source[key]) call means merged branches are fresh objects, so the sources stay untouched.
Merge a directory of JSON files
const fs = require("fs");
const path = require("path");
function mergeJsonDir(dir) {
return fs
.readdirSync(dir)
.filter((f) => f.endsWith(".json"))
.sort() // deterministic: later files win
.reduce((acc, file) => {
const content = JSON.parse(fs.readFileSync(path.join(dir, file), "utf-8"));
return deepMerge(acc, content);
}, {});
}
const config = mergeJsonDir("./config");
.sort() isn’t decorative: readdirSync returns entries in filesystem order, which differs across machines. Sorting makes “last file wins” predictable in CI and on your teammates’ boxes.
Custom conflict resolution
const _ = require("lodash");
function mergeWithResolver(sources, resolver) {
const result = {};
for (const source of sources) {
for (const key of Object.keys(source)) {
if (key in result && !_.isEqual(result[key], source[key])) {
result[key] = resolver(key, result[key], source[key]);
} else {
result[key] = source[key];
}
}
}
return result;
}
// Last value wins, but every conflict gets logged
const merged = mergeWithResolver(
[base, staging, production],
(key, oldVal, newVal) => {
console.warn(`Conflict on "${key}": ${JSON.stringify(oldVal)} -> ${JSON.stringify(newVal)}`);
return newVal;
}
);
_.isEqual does the deep comparison the naive version skipped; without it the resolver only ever sees top-level primitives.
Explanation
A merge is two decisions: how deep to combine, and who wins when keys collide. Shallow merge answers “replace everything below the top level”; deep merge walks the tree. Conflict strategy is the second axis: last-write-wins is what deepMerge above does implicitly; the resolver version makes it a choice you can log, veto, or turn into an error for strict CI checks.
Arrays deserve a special callout: deepMerge replaces them (the override’s list wins). For data merges you usually want concatenation plus deduplication by a stable key. Lodash _.mergeWith handles that with a customizer.
Two Node gotchas hide in the file-reading step. require("./x.json") caches its parsed result in the module registry, so editing the file won’t be picked up until restart; fs.readFileSync + JSON.parse re-reads every time, which is what hot-reload workflows expect. And JSON.parse throws on a UTF-8 BOM; files saved by some Windows editors carry one, so strip a leading \uFEFF before parsing if you can’t control how the inputs are produced.
Variants
ESM with JSON modules and async fs
// ESM (node >= 20.10 for import attributes)
import defaults from "./config/default.json" with { type: "json" };
import fs from "node:fs/promises";
async function mergeJsonDirAsync(dir) {
const files = (await fs.readdir(dir))
.filter((f) => f.endsWith(".json"))
.sort();
let merged = {};
for (const file of files) {
const content = JSON.parse(await fs.readFile(new URL(`${dir}/${file}`, import.meta.url), "utf-8"));
merged = deepMerge(merged, content);
}
return merged;
}
await at module top level is legal ESM; in CommonJS wrap it in an async function main(). JSON import ... with { type: "json" } is the cleanest option for a fixed set of files (static analysis and bundlers can see it), but it can’t list a directory, so fs stays necessary for glob-style merging. Mixing both worlds works too: import the defaults statically, then deep-merge the environment-specific files you discovered with fs on top.
Arrays: concatenate instead of replace
const merged = _.mergeWith({}, a, b, (objVal, srcVal) => {
if (Array.isArray(objVal) && Array.isArray(srcVal)) {
return objVal.concat(srcVal);
}
// undefined → fall back to default merge behavior
});
Streaming with JSONStream for files that don’t fit in memory
const fs = require("fs");
const JSONStream = require("JSONStream");
// Merge huge array files without loading them whole
function streamMergeArrays(inputs, output) {
const ws = fs.createWriteStream(output);
ws.write("[");
let first = true;
let pending = inputs.length;
for (const file of inputs) {
fs.createReadStream(file)
.pipe(JSONStream.parse("*"))
.on("data", (item) => {
ws.write((first ? "" : ",") + JSON.stringify(item));
first = false;
})
.on("end", () => {
if (--pending === 0) ws.end("]");
});
}
}
Streaming concatenates arrays only; deep object merging still needs the merged object in memory, because JSON gives no random access to keys. For 10k+ element datasets consider a worker thread so the merge doesn’t block the event loop; worker_threads runs the merge on a separate core and posts back the result.
Concat and dedupe arrays by a stable key
function mergeRecords(sources, key = "id") {
const seen = new Map();
for (const source of sources) {
for (const item of source) {
seen.set(item[key], item); // later files overwrite same-id records
}
}
return [...seen.values()];
}
const users = mergeRecords([batch1, batch2, batch3], "id");
Map keeps insertion order, so the output order is stable and the last file to define an id wins the slot.
Tolerate a broken file instead of failing the whole merge
function mergeJsonDirSafe(dir) {
const files = fs.readdirSync(dir).filter((f) => f.endsWith(".json")).sort();
let merged = {};
const skipped = [];
for (const file of files) {
try {
merged = deepMerge(merged, JSON.parse(fs.readFileSync(path.join(dir, file), "utf-8")));
} catch (err) {
skipped.push({ file, error: err.message });
}
}
return { merged, skipped };
}
One malformed file shouldn’t kill a batch merge. Collect the failures and surface them: silently skipping is worse than failing, since a dropped production.json looks exactly like a successful merge.
Validate the merged output with AJV
const Ajv = require("ajv");
const schema = require("./config.schema.json");
const merged = mergeJsonDir("./config");
const valid = new Ajv({ allErrors: true }).validate(schema, merged);
if (!valid) {
console.error(ajv.errors);
process.exit(1);
}
allErrors: true reports every violation in one pass, worth it in a build step where a merged config failing silently becomes a runtime incident.
In practice that validation belongs in CI, not at boot. Run the merge and the AJV check as a build step and fail the pipeline on any parse error or schema violation: node merge.js && node validate.js. The merge is deterministic and cheap, so gating on it costs seconds and removes a whole class of “worked locally” deploy surprises, where a broken production.json only blows up at runtime.
| Approach | Nested | Arrays | Deps | Use when |
|---|---|---|---|---|
{...a, ...b} | No | Replaces | — | Flat configs |
deepMerge (above) | Yes | Replaces | — | Most config merges |
_.merge | Yes | Replaces | lodash | Battled-tested default |
_.mergeWith | Yes | Customizer | lodash | Concat/dedupe arrays |
JSONStream | Concat only | Concat | JSONStream | Files > memory |
What Works
- Deep-merge config, concat data: config trees merge deep; record lists concat + dedupe by a stable key (
id) - Sort before merging: deterministic order makes “last file wins” reproducible across machines
- Fail on conflicts in CI: swap the resolver to
throw: a silent override inproduction.jsonmasks real mistakes - Validate the output: one AJV pass on the merged object catches missing keys before runtime
- Keep sources immutable: merge into
{}, never into arequired object: Node caches modules, so a mutatedrequireresult poisons every later import of that file
Common Mistakes
- Spread on nested config:
{...a, ...b}dropsa’s nested keys silently typeofas an object check:typeof null === "object"breaks naive recursive merges; guardnullandArrayexplicitly- Skipping
__proto__: unfiltered keys let an untrusted file polluteObject.prototypefor the whole process try/catcharoundrequired JSON:require("./x.json")throws on invalid JSON just likeJSON.parse; same handling applies- Top-level
awaitin CJS: legal only in ESM or insideasyncfunctions; a.jsfile withrequireis CJS regardless of the code’s looks - Mutating a
required object: the module cache shares it process-wide; copy first
Further Reading
Frequently Asked Questions
Can I import a JSON file directly in Node.js?
Yes — import cfg from "./app.json" with { type: "json" } on Node ≥ 20.10 (older: assert instead of with). It parses once at module load and is ideal for fixed config files. For directories you still need fs.readdir + fs.readFile, since imports can't be dynamic file lists.
How do I merge JSON files asynchronously?
fs/promises plus sequential deepMerge, or Promise.all to read files in parallel and merge after. Keep the merge step sequential: it's CPU-bound anyway, so parallelizing it gains nothing.
Is deepMerge safe against prototype pollution?
Only if it filters __proto__, constructor, and prototype keys, like the version above. A bare recursive merge assigns obj["__proto__"], which mutates Object.prototype process-wide. Alternatively validate input JSON against a schema before merging.
Should I write my own deep merge or use lodash?
_.merge is battle-tested for null, arrays, Dates, and circular structures; reach for it unless the dependency budget is zero. If you roll your own, copy the guards from the example above; the null/array/__proto__ checks are the part naive versions get wrong.
How do I merge files too large for memory?
Concatenate arrays through JSONStream (above); object deep-merging can't stream, since it needs the whole merged tree. For very large object graphs, split the file by top-level key first with Flatten and Unflatten Objects or move the data into a real store.
Does deepMerge mutate its inputs?
The version above doesn't: merged branches are built into fresh objects. Watch out for variants that merge into target[key] in place. And never merge into a required JSON object, because Node's module cache would share the mutation with every consumer.
Related Resources
Merge JSON Files
How to merge multiple JSON files into a single object or array in Python, Java, and JavaScript.
RecipeDiff JSON Objects
How to compare two JSON objects and find differences in Python, Java, and JavaScript.
RecipeFlatten and Unflatten Nested Objects
How to convert nested objects to flat key-value pairs and back again, with dot-notation, bracket notation, and custom separator support.
RecipeConvert JSON to CSV
How to convert JSON data to CSV format in Python, Java, and JavaScript — flatten nested objects, unwind arrays, and produce Excel-ready output.
RecipeConvert CSV to JSON
Turn CSV files into structured JSON using Python, JavaScript, and Java. Pick the right library for quick one-off conversions or large streaming pipelines.
RecipeParse CSV Files
How to parse CSV files in Python, Java, and JavaScript with practical code examples.