How to Get Unique Values from a List in Python

Extracting unique values from a list, removing duplicates while keeping only distinct elements, is one of the most common operations in Python. Whether you're cleaning data, processing user inputs, or preparing datasets, knowing how to efficiently deduplicate a list is essential.

For example, given [1, 2, 1, 1, 3, 4, 3, 3, 5], the unique values are [1, 2, 3, 4, 5].

This guide covers several approaches, highlighting the trade-off between preserving order and performance.

Using dict.fromkeys() (Recommended)

The most Pythonic and efficient way to get unique values while preserving the original order is dict.fromkeys(). Since dictionary keys must be unique (and since Python 3.7+, dictionaries maintain insertion order), this approach removes duplicates and keeps the first occurrence of each element.

a = [1, 2, 1, 1, 3, 4, 3, 3, 5]

unique = list(dict.fromkeys(a))
print(unique)

Output:

[1, 2, 3, 4, 5]

dict.fromkeys(a) creates a dictionary where each element from the list becomes a key. Duplicate elements are naturally discarded since keys can't repeat. Converting the keys back to a list produces the unique values in their original order.

Why this method is recommended

• Preserves insertion order (guaranteed in Python 3.7+)
• O(n) time complexity: each lookup and insertion into the dict is O(1) on average
• Clean one-liner: no imports needed

Using set() (Fastest, No Order Guarantee)

Converting a list to a set is the fastest way to remove duplicates because sets are optimized for uniqueness checks. However, sets are unordered, so the original element order is not preserved.

a = [1, 2, 1, 1, 3, 4, 3, 3, 5]

unique = list(set(a))
print(unique)

Output (order may vary):

[1, 2, 3, 4, 5]

Order is not guaranteed

While the output might appear ordered with small integers (due to how CPython implements sets internally), this is not guaranteed:

a = ['banana', 'apple', 'cherry', 'apple', 'banana']
print(list(set(a)))
# Possible output: ['cherry', 'banana', 'apple'] : order varies!

If order matters, use dict.fromkeys() instead.

Preserving Order with set + sorted()

If you need sorted unique values (not original order, but sorted), combine set with sorted():

a = [5, 3, 1, 4, 2, 3, 1, 5]

unique_sorted = sorted(set(a))
print(unique_sorted)

Output:

[1, 2, 3, 4, 5]

Using collections.OrderedDict

Before Python 3.7 guaranteed dictionary insertion order, OrderedDict was the standard way to remove duplicates while preserving order. It still works perfectly and makes the intent explicit:

from collections import OrderedDict

a = [1, 2, 1, 1, 3, 4, 3, 3, 5]

unique = list(OrderedDict.fromkeys(a))
print(unique)

Output:

[1, 2, 3, 4, 5]

note

In Python 3.7+, regular dict preserves insertion order, so dict.fromkeys() achieves the same result without the import. Use OrderedDict if you need to support Python 3.6 or want to make the ordering intent explicit in your code.

Using a for Loop

A manual loop gives you the most control and is easy to understand. It checks each element before adding it to the result list:

a = [1, 2, 1, 1, 3, 4, 3, 3, 5]

unique = []
for item in a:
    if item not in unique:
        unique.append(item)

print(unique)

Output:

[1, 2, 3, 4, 5]

This preserves order and is clear in its intent, but it has a performance drawback.

Performance concern with large lists

The if item not in unique check scans the entire unique list each time, making this approach O(n²). For large lists, this becomes significantly slower than dict.fromkeys() or set().

Fix: use a set for O(1) lookups.

a = [1, 2, 1, 1, 3, 4, 3, 3, 5]

seen = set()
unique = []
for item in a:
    if item not in seen:
        seen.add(item)
        unique.append(item)

print(unique)

This preserves order with O(n) time complexity, the best of both worlds.

Using List Comprehension with a Set

A concise version of the optimized loop approach using list comprehension:

a = [1, 2, 1, 1, 3, 4, 3, 3, 5]

seen = set()
unique = [x for x in a if x not in seen and not seen.add(x)]
print(unique)

Output:

[1, 2, 3, 4, 5]

This works because seen.add(x) returns None (which is falsy), so not seen.add(x) is always True. The side effect of adding x to seen happens during the comprehension.

note

While clever, this approach uses a side effect inside a list comprehension, which some consider unpythonic. The explicit loop with a set or dict.fromkeys() is generally preferred for readability.

Working with Unhashable Elements

All the methods above require elements to be hashable (e.g., numbers, strings, tuples). If your list contains unhashable elements like dictionaries or lists, you need a different approach:

a = [{'name': 'Alice'}, {'name': 'Bob'}, {'name': 'Alice'}]

unique = []
for item in a:
    if item not in unique:
        unique.append(item)

print(unique)

Output:

[{'name': 'Alice'}, {'name': 'Bob'}]

This O(n²) loop is the only option for unhashable types since they cannot be added to sets or used as dictionary keys.

Comparison of Approaches

Method	Preserves Order	Time Complexity	Handles Unhashable?	Best For
dict.fromkeys()	✅	O(n)	❌	Most use cases (recommended)
set()	❌	O(n)	❌	When order doesn't matter
sorted(set())	Sorted order	O(n log n)	❌	When you need sorted results
OrderedDict.fromkeys()	✅	O(n)	❌	Python 3.6 compatibility
Loop with set tracking	✅	O(n)	❌	Custom logic during dedup
Loop (list-only check)	✅	O(n²)	✅	Unhashable elements

Conclusion

For getting unique values from a list in Python, dict.fromkeys() is the best default choice: it's fast, preserves order, and requires no imports.

• Use set() when you do not care about order and want maximum speed.
• For large lists where you need both order and performance with custom logic
• Use a loop with a set for tracking.
• When working with unhashable elements like dictionaries or nested lists, a simple for loop with membership checking is your only option.