Remove duplicates nested list Python dependend on position-CodePudding

in a nested list like the one below, I'd like to remove duplicates depending on the position.

[[1,2,3,4], [2,1,3,4], [1,2,3,4], [1,3,2,4]]

So every sub-list contains the same numbers, but in different order. If the order of the numbers is the same, I would like to delete this duplicate. So the list above should look like:

[[1,2,3,4], [2,1,3,4], [1,3,2,4]]

I have tried to write some code by myself, but since I am a beginner, there is no working result. I have tried:

result = []
for i in test_list:
    if i not in result:
        result.append(i)
return result

tpls = [tuple(x) for x in test_list]
dct = list(dict.fromkeys(tpls))
dup_free = [list(x) for x in test_list]
return dup_free

Thanks!

EDIT: the nested list is just a part of the full list. Full list:

[[3, 4, 1, 2], [3, 4, 1, 2], [3, 4, 1, 2], [3, 4, 1, 2], [3, 4, 1, 2], [3, 4, 1, 2], [3, 4, 1, 2], [3, 4, 1, 2], [3, 4, 1, 2], [3, 4, 1, 2], [3, 4, 1, 2], [3, 4, 1, 2]]

CodePudding user response：

lst = [[1,2,3,4], [2,1,3,4], [1,2,3,4], [1,3,2,4]]

new_lst = []
for a in lst:
    if a not in new_lst: # check if current element already in the new_lst or not.
        new_lst.append(a)
print(new_lst)

OUTPUT

[[1, 2, 3, 4], [2, 1, 3, 4], [1, 3, 2, 4]]

CodePudding user response：

You can use pandas for this purpose.

import pandas as pd

aaa = {'date': ['2022-05-02', '2022-04-29', ' 2022-04-29', '2022-04-28 ', '2022-04-28 ', '2022-04-28 '],
       'id': ['A', 'A', 'B', 'A', 'B', 'C'], 'number': [397177, 53876, 191214, 75824, 483860, 51731]}

df = pd.DataFrame(aaa)
df = df.drop_duplicates().values

Output

[[1 2 3 4]
 [2 1 3 4]
 [1 3 2 4]]

CodePudding user response：

The first solution is correct you need to add it into method to use return or to print the resu

CodePudding user response：

First one is working run this code :

X = [[1,2,3,4], [2,1,3,4], [1,2,3,4], [1,3,2,4]]

def Rem_Dup(L) :
    result = []
    for i in L:
        if i not in result:
            result.append(i)
    return result

print(Rem_Dup(X))

CodePudding user response：

For each sublist sublst in the original nested list, convert into tuple for lookup in the dictionary dct. Add to the resulting list without dups nodups only if we have not seen it:

lst = [[1,2,3,4], [2,1,3,4], [1,2,3,4], [1,3,2,4]]
nodups = []
dct = {}

for sublst in lst:
    tup = tuple(sublst)
    if tup not in dct:
        dct[tup] = 1
        nodups.append(sublst)
    else:
        pass
print(nodups)
# [[1, 2, 3, 4], [2, 1, 3, 4], [1, 3, 2, 4]]

CodePudding user response：

You can do it in one line with a list comprehension:

a = [[1,2,3,4], [2,1,3,4], [1,2,3,4], [1,3,2,4]]
[j for i, j in enumerate(a) if j not in a[i 1:]]

Output:

[[2, 1, 3, 4], [1, 2, 3, 4], [1, 3, 2, 4]]

Note: This will choose the last occurrence of that sublist in the original list if you care about the ordering

Edit: If you care about ordering, you can go through the list backwards and then reverse the result:

[j for i, j in enumerate(reversed(a)) if j not in list(reversed(a))[i 1:]][::-1]

Output:

[[1, 2, 3, 4], [2, 1, 3, 4], [1, 3, 2, 4]]