Unnest dictionary with nested Dictionary or list of dictionaries-CodePudding

This question is related to Flatten nested dictionaries, compressing keys. I used that idea and modified to unnest list of dictionary but unable to do.

My input will be a dictionary which can have nested dictonaries or nested lists (list of dictionaries) as shown in the input to the function, flatten_dict_list

Below is my source code:

def flatten_dict_list(d, parent_key='', sep='.'):
    items = []
    for k, v in d.items():
        print("***")
        new_key = parent_key   sep   k if parent_key else k
        print(new_key)

        if isinstance(v, list):
          for z in v:
              if isinstance(z, collections.abc.MutableMapping):
                  print(type(z))
                  print(z)
                  items.extend(flatten_dict_list(z, new_key, sep=sep).items())
              else:
                  print(type(z))
                  print(z)
                  items.append((new_key, z))
        else:
          if isinstance(v, collections.abc.MutableMapping):
             print(type(v))
             print(v)
             items.extend(flatten_dict_list(v, new_key, sep=sep).items())
          else:
             print(type(v))
             print(v)
             items.append((new_key, v))
    return dict(items)

print(flatten_dict_list({'a': 1, 'c': [{'a': 2, 'b': {'x': 5, 'y' : 10}}, {'test': 9999}, [{'s': 2}, {'t': 100}]], 'd': [1, 2, 3]}))

Expected output is: {'a': 1, 'c.a': 2, 'c.b.x': 5, 'c.b.y': 10, 'c.test': 9999, 'c.s': 2, 'c.t': 100, 'd': 3}

My actual result is: {'a': 1, 'c.a': 2, 'c.b.x': 5, 'c.b.y': 10, 'c.test': 9999, 'c': [{'s': 2}, {'t': 100}], 'd': 3}

CodePudding user response：

Given that:

you don't care that both {'a': {'b': 1, 'c': 2}} and {'a': [{'b': 1}, {'c': 2}]} will map to the same {'a.b': 1, 'a.c': 2}
there won't be anything other than dictionaries in the lists in the input data structure

This seems like a fairly clean solution:

def _compound_key_value(xs, prefix):
    if isinstance(xs, list):
        for x in xs:
            yield from _compound_key_value(x, prefix)
    elif isinstance(xs, dict):
        for k, v in xs.items():
            for p, r in _compound_key_value(v, prefix):
                yield prefix   (k,)   p, r
    else:
        yield prefix, xs


def flatten_dict_list(dl):
    return {'.'.join(k): v for k, v in _compound_key_value(dl, ())}


print(flatten_dict_list({'a': 1, 'c': [{'a': 2, 'b': {'x': 5, 'y' : 10}}, {'test': 9999}, [{'s': 2}, {'t': 100}]], 'd': [1, 2, 3]}))

Output

{'a': 1, 'c.a': 2, 'c.b.x': 5, 'c.b.y': 10, 'c.test': 9999, 'c.s': 2, 'c.t': 100, 'd': 3}

Note that it is recursive, like the solution you started out with, so I also assumed maximum recursion depth would not become an issue.