Machine Learning - Reading the JSON data using Python in 3 ways

  Dear Learners,

This blog is to just give a glimpse to show how a simple python code can be used to extract the data from Web API(JSON ) into a tabular (pandas) format. As a learner myself had experienced many hurdles to find out the ways to read JSON effectively therefore blogging here to share my approaches to the problem .For further references , please visit my Git link given below and also i have shared my Linked In profile and Youtube Channel.

Here are the 3 different ways to do it :

Approach 1 :

#Importing Python Libraries

import requests

import pandas as pd

from pandas.io.json import json_normalize

#used this example URL for example json

url = 'https://api.github.com/repos/pandas-dev/pandas/issues'

resp = requests.get(url)

resp

#Read the JSON

data = resp.json()

#Declare a  Dict Variable

data_flat = {}

#Define the Custom Function Named flatten_dict which is resposible to flatten the JSON:

def flatten_dict(obj,name=''):

    for k,v in obj.items():

                        if((type(v) is str) or (type(v) is int) or (type(v) is bool) or (isinstance(v, type(None)))):

                            if(isinstance(v, type(None))):

                                key_val = name + '_'+ str(k)

                                data_flat[key_val]=None

                            else:

                                key_val = name + '_' + str(k)

                                data_flat[key_val]=v

                        elif(type(v) is dict):

                            name =str(k)

                            flatten_dict(v,name)

                            name = ''

                        elif(type(v) is list):

                            for i in range(len(v)):

                                name=str(k) +'_'+ str(float(i))

                                flatten_dict(v[i],name)

                                name=''

                        else:

                            name = ''

    return data_flat

#Declare List Variable 

listing = []

#Declare DataFrame Variable

df1= pd.DataFrame()

#Traverse the entire data set and populate the respective dictionaries in it.Finally populate the df1 

for i in range(len(data)):

    data_local = flatten_dict(data[i])

    listing.append(data_local)

    l_df= pd.DataFrame([listing[i]])   

    df1=df1.append(l_df,sort=False)

Request:

If you think it has added value to you or helped in anyway , please like ,comment ,ask questions', share and subscribe to support and encourage me to write more for the larger humanity.

For you always

Ramdhan Sharma

Git

LinkedIn

Youtube