How to create Data Frame from list of lists where the inner list has data in a dictionary-CodePudding

I have a list mylist where its elements are to be fetched from different files and is available in the format below.

mylist = [[{'id':2,'name':"Ted"},{'id':7,'name':"Jack"}],
          [{'id':13,'name':"Jim"},{'id':19,'name':"Tom"}],
          [{'id':73,'name':"Sid"},{'id':89,'name':"Harry"}]]

How do I create a dataframe from this list which appears like below?

    id    name
0    2    Ted
1    7    Jack
2   13    Jim
3   19    Tom
4   73    Sid
5   89    Harry

CodePudding user response：

You can first flatten the list of lists into a single list of dictionaries (you can use itertools.chain for that) and cast it to the DataFrame constructor:

from itertools import chain
out = pd.DataFrame(chain.from_iterable(mylist))

Output:

   id   name
0   2    Ted
1   7   Jack
2  13    Jim
3  19    Tom
4  73    Sid
5  89  Harry

CodePudding user response：

You only have to flatten your 2 level list into a simple iterable:

df = pd.DataFrame(d for lst in mylist for d in lst)

It gives as expected:

   id   name
0   2    Ted
1   7   Jack
2  13    Jim
3  19    Tom
4  73    Sid
5  89  Harry

CodePudding user response：

Without any extra package and a rather straightforward approach is:

cols = ['id', 'name']
data = []
for item in mylist:
    data.append(item[0].values())
    data.append(item[1].values())

df = pd.DataFrame(data, columns = cols)

It turns your dictionaries to a list data of lists. Output:

   id   name
0   2    Ted
1   7   Jack
2  13    Jim
3  19    Tom
4  73    Sid
5  89  Harry