How to Remove Duplicates from a List in Python

Removing duplicates from a list means keeping only one copy of each value.

In Python, there are a few simple ways to do this:

Use dict.fromkeys() to remove duplicates and keep the original order
Use set() if you only care about unique values
Use a loop if you want to understand the logic or need more control

For most beginners, dict.fromkeys() is a good default choice.

Quick fix

items = [1, 2, 2, 3, 1, 4]
unique_items = list(dict.fromkeys(items))
print(unique_items)

Output:

[1, 2, 3, 4]

This removes duplicates and keeps the original order.

What removing duplicates means

A duplicate is a value that appears more than once in a list.

For example:

items = [1, 2, 2, 3, 1, 4]

In this list:

2 appears more than once
1 appears more than once

If you remove duplicates, you keep one copy of each value:

[1, 2, 3, 4]

The important thing to remember is this:

Some methods keep the original order
Some methods do not keep the original order

If order matters, choose your method carefully.

Use `dict.fromkeys()` to keep order

This is one of the easiest ways to remove duplicates while keeping the original order.

items = [1, 2, 2, 3, 1, 4]
unique_items = list(dict.fromkeys(items))

print(unique_items)

Output:

[1, 2, 3, 4]

How it works

dict.fromkeys(items) creates a dictionary using the list values as keys
Dictionary keys must be unique
Duplicate values are removed automatically
list(...) turns the keys back into a list

This works well for:

strings
numbers
tuples

Example with strings:

names = ["Ana", "Bob", "Ana", "Cara", "Bob"]
unique_names = list(dict.fromkeys(names))

print(unique_names)

Output:

['Ana', 'Bob', 'Cara']

If you want to understand sets better, see Python sets explained and the set() function reference.

Use `set()` when order does not matter

A set stores only unique values.

That means you can remove duplicates like this:

items = [1, 2, 2, 3, 1, 4]
unique_items = list(set(items))

print(unique_items)

Possible output:

[1, 2, 3, 4]

This method is:

short
common
often fast

But there is one important warning:

The final order may not match the original list

For example:

letters = ["b", "a", "c", "a", "b"]
unique_letters = list(set(letters))

print(unique_letters)

The output may be in a different order than the original list.

Use this only when order does not matter.

Use a loop to understand the logic

A loop is easy to read and helps you understand what duplicate removal is doing.

items = [1, 2, 2, 3, 1, 4]
unique_items = []

for item in items:
    if item not in unique_items:
        unique_items.append(item)

print(unique_items)

Output:

[1, 2, 3, 4]

What the code does

Start with an empty list called unique_items
Loop through each item in the original list
Check whether the item is already in unique_items
If not, add it with append()

This keeps the original order and is beginner-friendly.

If you want a quick refresher on adding values to lists, see Python list append() method.

When to use this method

Use a loop when:

you want to understand the logic clearly
your list contains values that do not work well with set() or dict.fromkeys()
you need custom rules for what counts as a duplicate

This approach is usually slower for large lists because item not in unique_items checks the list again each time. For small beginner examples, that is often fine.

Handle lists with unhashable items

set() and dict.fromkeys() work best with hashable values such as:

strings
numbers
tuples

Some values are unhashable, including:

lists
dictionaries

That means this will fail:

items = [[1, 2], [1, 2], [3, 4]]
unique_items = list(set(items))

You will get an error because lists cannot be used as set values.

In cases like this, use a loop:

items = [[1, 2], [1, 2], [3, 4]]
unique_items = []

for item in items:
    if item not in unique_items:
        unique_items.append(item)

print(unique_items)

Output:

[[1, 2], [3, 4]]

Example with dictionaries

A list of dictionaries usually needs a custom rule.

items = [
    {"id": 1, "name": "Ana"},
    {"id": 2, "name": "Bob"},
    {"id": 1, "name": "Ana"}
]

unique_items = []

for item in items:
    if item not in unique_items:
        unique_items.append(item)

print(unique_items)

Output:

[{'id': 1, 'name': 'Ana'}, {'id': 2, 'name': 'Bob'}]

Sometimes you do not want to compare whole dictionaries. You may only want to treat items as duplicates if the "id" matches. In that case, you need a custom check.

Choose the right method

Here is a simple guide:

Use dict.fromkeys() if you want to keep order
Use set() if order does not matter
Use a loop if you want beginner-friendly logic or need custom checks

In short:

Best simple default: list(dict.fromkeys(items))
Shortest option: list(set(items))
Most flexible option: a loop

If you are also checking values before adding them, how to check if a value exists in a list in Python may help.

Common mistakes

Here are some common problems when removing duplicates from a list.

Using `set()` and expecting the original order to stay the same

This is a very common mistake.

items = [3, 1, 2, 1, 3]
unique_items = list(set(items))
print(unique_items)

The values will be unique, but the order may change.

If you need order, use:

unique_items = list(dict.fromkeys(items))

Trying `set()` on a list that contains other lists or dictionaries

This will usually fail because lists and dictionaries are unhashable.

Bad example:

items = [[1, 2], [1, 2]]
unique_items = list(set(items))

Use a loop instead.

Removing duplicates without assigning the result

This does not change the original list by itself:

items = [1, 2, 2, 3]
list(dict.fromkeys(items))
print(items)

Output:

[1, 2, 2, 3]

You need to store the result:

items = [1, 2, 2, 3]
unique_items = list(dict.fromkeys(items))
print(unique_items)

Thinking duplicate removal changes the original list automatically

Most duplicate-removal approaches create a new list.

If you want to keep the result, assign it to a variable.

You can also replace the original variable:

items = [1, 2, 2, 3]
items = list(dict.fromkeys(items))
print(items)

If you want to preserve the original list before changing it, see how to copy a list in Python.

Helpful debugging checks

If your result is not what you expected, these checks can help:

print(items)
print(type(items))
print(unique_items)
print(len(items), len(unique_items))
for item in items:
    print(repr(item))

These help you:

confirm the input list
check the data type
compare the original and new lists
see whether values only look the same but are actually different

FAQ

How do I remove duplicates and keep the original order?

Use list(dict.fromkeys(your_list)). It is short and keeps order in modern Python.

What is the shortest way to remove duplicates?

Use list(set(your_list)), but only if you do not care about order.

Why did my list change order after removing duplicates?

You probably used set(). Sets do not keep list order.

Can I remove duplicates from a list of dictionaries?

Not directly with set() in most cases. Use a loop and define your own rule for duplicates.