How to Merge Multiple JSON Files Using Python
We are given multiple JSON files and our task is to merge those multiple JSON files into a single JSON file with the help of different approaches in Python. In this article, we will see how we can merge multiple JSON files in Python.
JSON Files
Below are the two JSON files that we will use in our article to merge them into a single JSON file.
f.json
{"key1":"value1"}
s.json
{"key2": "value2"}
Merging Multiple JSON Files in Python
Below are some of the ways by which we can merge multiple JSON files in Python:
Merge Multiple JSON Files Using json Module
In this example, a Python function merge_json_files
is defined to combine data from multiple JSON files specified by file_paths
into a list called merged_data
. The merged data is then written to a new JSON file named “merged.json,” and a confirmation message is printed.
Python3
import json def merge_json_files(file_paths, output_file): merged_data = [] for path in file_paths: with open (path, 'r' ) as file : data = json.load( file ) merged_data.append(data) with open (output_file, 'w' ) as outfile: json.dump(merged_data, outfile) file_paths = [ "f.json" , "s.json" ] output_file = "merged.json" merge_json_files(file_paths, output_file) print (f "Merged data written to '{output_file}'" ) |
Output:
merged.json
[{"key1": "value1"}, {"key2": "value2"}]
Merge Multiple JSON Files Using List Comprehension
In this example, the merge_json_files
function reads JSON data from multiple files specified by file_paths
using a list comprehension. The data is then stored in a list named merged_data
. Subsequently, the combined data is written to a new JSON file, “merged.json,” using the json.dump()
method. Finally, the merged data list is printed for confirmation.
Python
import json def merge_json_files(file_paths): merged_data = [json.load( open (path, 'r' )) for path in file_paths] return merged_data file_paths = [ "f.json" , "s.json" , "e.json" , "t.json" ] output_file = "merged.json" merged_data = merge_json_files(file_paths) with open (output_file, 'w' ) as outfile: json.dump(merged_data, outfile) print (merged_data) |
Output:
merged.json
[{"key1": "value1"}, {"key2": "value2"}, {"key2": "value2"}, {"key2": "value2"}]
Merge Multiple JSON Files Using os Module with json Module
In this example, the merge_json_files
function reads and merges JSON files from the specified directory ("./files"
). The combined data is then written to a new JSON file named “merged.json,” and the merged data is printed for confirmation.
Python3
import json import os def merge_json_files(directory_path): merged_data = [] for filename in os.listdir(directory_path): if filename.endswith( '.json' ): with open (os.path.join(directory_path, filename), 'r' ) as file : data = json.load( file ) merged_data.append(data) return merged_data directory_path = "./files" output_file = "merged.json" merged_data = merge_json_files(directory_path) with open (output_file, 'w' ) as outfile: json.dump(merged_data, outfile) print (merged_data) |
Output:
merged.json
[{"key1": "value1"}, {"key2": "value2"}]
Merge Multiple JSON Files Using glob Module with json Module
In this example, the merge_json_files
function utilizes the glob
module to obtain a list of JSON file paths from the specified directory ("./files"
). It then reads and merges the JSON data from these files into a list named merged_data
. The combined data is subsequently written to a new JSON file, “merged.json,” and the merged data is printed for confirmation.
Python3
import json import glob def merge_json_files(directory_path): merged_data = [] file_paths = glob.glob(directory_path + '/*.json' ) for path in file_paths: with open (path, 'r' ) as file : data = json.load( file ) merged_data.append(data) return merged_data directory_path = "./files" output_file = "merged.json" merged_data = merge_json_files(directory_path) with open (output_file, 'w' ) as outfile: json.dump(merged_data, outfile) print (merged_data) |
Output:
merged.json
[{"key1": "value1"}, {"key2": "value2"}]
Merge Multiple JSON Files Using Pandas Library
In this example, the merge_json_files
function reads JSON data from multiple files specified by file_paths
and merges them into a Pandas DataFrame named merged_data
. The combined data is then written to a new JSON file, “merged.json,” using the to_json
method with the ‘records’ orientation. Finally, the merged DataFrame is printed for confirmation.
Python3
import pandas as pd import json def merge_json_files(file_paths): merged_data = pd.DataFrame() for path in file_paths: with open (path, 'r' ) as file : data = json.load( file ) row = pd.DataFrame([data]) merged_data = pd.concat([merged_data, row], ignore_index = True ) return merged_data file_paths = [ "f.json" , "s.json" , "e.json" , "t.json" ] output_file = "merged.json" merged_data = merge_json_files(file_paths) merged_data.to_json(output_file, orient = 'records' ) print (merged_data) |
Output:
merged.json
[{"key1": "value1", "key2": null}, {"key1": null, "key2": "value2"}, {"key1": null, "key2": "value2"}, {"key1": null, "key2": "value2"}]