Extracting Information From Multiple Json Files To Single Csv File In Python

I have a JSON file with multiple dictionaries: {'team1participants': [ { 'stats': { 'item1': 3153, 'totalScore': 0, ... }

Solution 1:

You can make your data tidy so that each row is a unique observation.

teams = []
items = []
scores = []
for team in d:
    for item in d[team]:
        teams.append(team)
        items.append(item['stats']['item1'])
        scores.append(item['stats']['totalScore'])


# Using Pandas.
import pandas as pd

df = pd.DataFrame({'team': teams, 'item': items, 'score': scores})
>>> df
    item   score               team
018532  team2participants
1215235  team2participants
2125031  team2participants
331530  team1participants
421235  team1participants
512531  team1participants

You could also use a list comprehension instead of a loop.

results = [[team, item['stats']['item1'], item['stats']['totalScore']] 
           for team in d for item in d[team]]
df = pd.DataFrame(results, columns=['team', 'item', 'score'])

You can then do a pivot table, for example:

>>> df.pivot_table(values='score ', index='team ', columns='item', aggfunc='sum').fillna(0)
item               12531853212331531250321523
team                                                       
team1participants      105000
team2participants      020015

Also, now that it is a dataframe, it is easy to save it as a CSV.

df.to_csv(my_file_name.csv)