Python sets: unique items, set operations and frozenset

A set is an unordered collection of unique items. If you add the same value twice, only one copy is kept. That single property makes sets ideal for removing duplicates, testing membership quickly, and comparing groups of things with operations borrowed from mathematics: union, intersection and difference. Every output here comes from running the code on a real interpreter.

Creating a set

s = {3, 1, 2, 3}
print(s, len(s))                        # {1, 2, 3} 3
print(set([1, 1, 2]), set("hello") == {"h", "e", "l", "o"})     # {1, 2} True

Curly braces with items make a set; set(iterable) converts any iterable and drops duplicates. The one trap: {} is an empty dictionary, not an empty set. Use set():

print(type({}), type(set()))     # <class 'dict'> <class 'set'>

Rules for what a set can hold

Set members must be hashable (immutable in practice): numbers, strings, tuples of hashables. Lists, dictionaries and other sets can't be members.

try:
    {[1]}
except TypeError as e:
    print(e)                      # unhashable type: 'list'
print({(1, 2), (1, 2)})           # {(1, 2)}

Also note that values that are equal count as the same item. 1, 1.0 and True are equal, so they collapse into one:

print({1, True, 1.0})             # {1}

No order, no indexes

A set has no positions: s[0] raises an error, and the printed order is up to Python. For small integers it often looks sorted, but never rely on that, and for strings the order can differ from run to run. If you need a predictable order, use sorted(s), which returns a list.

Adding and removing

s = {1, 2, 3}
s.add(4)
s.update([5, 6], {7})              # add many at once
print(sorted(s))                   # [1, 2, 3, 4, 5, 6, 7]
s.remove(7)                        # error if missing
s.discard(99)                      # silent if missing
print(sorted(s))                   # [1, 2, 3, 4, 5, 6]
try:
    s.remove(99)
except KeyError as e:
    print("KeyError", e)           # KeyError 99

Method	What it does
add(x)	Adds one item
update(*iterables)	Adds all items from the iterables
remove(x)	Removes x; KeyError if absent
discard(x)	Removes x if present; never an error
pop()	Removes and returns an arbitrary item
clear()	Empties the set
copy()	Shallow copy

c = {1, 2, 3, 4}.copy()
c.add(100)
c.clear()
print(c, len(c))                   # set() 0

An empty set prints as set(). Because of the ordering rule, pop() removes some element, not "the first".

Set operations

Take two sets:

a = {1, 2, 3, 4}
b = {3, 4, 5}

Operation	Operator	Method	Result for a, b
Union (in either)	a | b	a.union(b)	{1, 2, 3, 4, 5}
Intersection (in both)	a & b	a.intersection(b)	{3, 4}
Difference (in a, not b)	a - b	a.difference(b)	{1, 2}
Symmetric difference (in exactly one)	a ^ b	a.symmetric_difference(b)	{1, 2, 5}

print(a | b, a & b, a - b, b - a, a ^ b)
# {1, 2, 3, 4, 5} {3, 4} {1, 2} {5} {1, 2, 5}

Note that difference depends on the order (a - b differs from b - a) while the others do not. The method forms accept any iterable, not only sets: a.union([9, 10]) works but a | [9, 10] does not.

Comparing sets

print({1, 2} <= a, {1, 2} < a, a >= {1, 2}, a.issubset({1, 2, 3, 4, 5}), a.isdisjoint({9}))
# True True True True True

<= means "is a subset of", < a proper subset (a subset that is not equal), >= a superset, and isdisjoint means "nothing in common". {1, 2} == {2, 1} is True: order is irrelevant.

In-place versions

a |= b, a &= b, a -= b and a ^= b, or update, intersection_update, difference_update and symmetric_difference_update, change the set instead of creating a new one:

a = {1, 2, 3, 4}
a.intersection_update({1, 2, 3})
print(a)                             # {1, 2, 3}

Aliases are not copies

x = {1, 2, 3}
y = x
y.add(4)
print(x)                             # {1, 2, 3, 4}

y = x points both names at the same set. Use x.copy() for an independent one.

Membership is fast

Testing item in my_set takes about the same time however big the set is, because it uses a hash table. Testing item in my_list has to scan. For large collections the difference is huge:

import timeit
l = list(range(20000)); st = set(l)
print(timeit.timeit(lambda: 19999 in l, number=200) > timeit.timeit(lambda: 19999 in st, number=200))
# True

If you check membership repeatedly, convert the list to a set first.

Set comprehensions

Like list comprehensions, but with braces:

print({x % 3 for x in range(10)})                         # {0, 1, 2}
print(sorted({c for c in "hello world" if c in "aeiou"}))   # ['e', 'o']

frozenset: the immutable set

A frozenset can't be changed after creation, which makes it hashable, so it can be a dictionary key or a member of another set.

fs = frozenset([1, 2])
print(fs, {fs: "k"}[fs])               # frozenset({1, 2}) k
try:
    fs.add(3)
except AttributeError as e:
    print(e)                            # 'frozenset' object has no attribute 'add'

Practical uses

Remove duplicates (the order is lost):

print(sorted(set([3, 1, 3, 2, 1])))     # [1, 2, 3]

Remove duplicates but keep the order, using a dictionary:

words = ["apple", "bob", "apple", "cat", "bob"]
print(len(words), len(set(words)), list(dict.fromkeys(words)))
# 5 3 ['apple', 'bob', 'cat']

Find common and different items, for example subscribers who opened both of two emails (a & b), only the first (a - b) or either (a | b).

Check for overlap with isdisjoint, and validate that a list contains no repeated item with len(set(items)) == len(items).

Find items in one list but not another:

a = [1, 2, 3, 4]; b = [3, 4, 5]
print([x for x in a if x in set(b)])    # [3, 4]

(The set is built once per check here; build it before the comprehension if the list is large.)

Sets compared with the other collections

	list	tuple	set	dict
Ordered	yes	yes	no	insertion order
Duplicates	yes	yes	no	unique keys
Changeable	yes	no	yes	yes
Fast in	no	no	yes	yes (keys)

A worked example

Two classes sign up for workshops. Which students attend both, which only the first, and how many different students are there?

maths = {"Asha", "Ben", "Chen", "Dev"}
art = {"Chen", "Dev", "Esha"}
print(sorted(maths & art), sorted(maths - art), len(maths | art))
# ['Chen', 'Dev'] ['Asha', 'Ben'] 5

The same question with two lists would need nested loops and a lot more code. Sets turn it into three short expressions, and the result is correct even when a name appears twice in the input.

Common mistakes

• Writing {} and expecting an empty set.
• Relying on the printed order.
• Putting a list or dict inside a set.
• Using remove when the item might be missing (use discard).
• Expecting set[0] to work.
• Losing order when removing duplicates with set().

Practice

Given two lists of student names for two courses, print who is in both, who is in only the first, and the total number of distinct students. Then write a function that returns True if all characters of a word are unique. Finally, time membership tests on a list and a set of 100,000 numbers.

Continue

See how sets fit with the other data types, compare with tuples, and explore the quick reference in the Python cheat sheet. The interview questions include several on mutability and hashing.