Generators in Python Explained

Generators let you produce values one at a time instead of building a full list all at once.

This is useful when:

  • you have many values
  • you only need to loop through them once
  • you want to create values only when needed

A Python function becomes a generator function when it uses yield.

Quick example

def count_up_to(limit):
    number = 1
    while number <= limit:
        yield number
        number += 1

for value in count_up_to(3):
    print(value)

Output:

1
2
3

Use yield inside a function to create a generator. The values are produced one at a time when you loop over it.

What a generator is

A generator is an object that gives values one at a time.

Unlike a list, it does not create and store all results in memory immediately. Instead, it waits until Python asks for the next value.

Generators are helpful when:

  • you are working with lots of data
  • values should be created as needed
  • you only need to loop through the results once

If you are new to this topic, it also helps to understand the idea of an iterable in Python and an iterator in Python, because generators are closely related to both.

How yield works

yield sends back one value and pauses the function.

When Python asks for the next value, the function continues from where it stopped.

That is the main difference between yield and return:

  • return ends the function completely
  • yield pauses the function and allows it to continue later

Example:

def simple_generator():
    print("Start")
    yield 10
    print("Middle")
    yield 20
    print("End")

gen = simple_generator()

print(next(gen))
print(next(gen))

Output:

Start
10
Middle
20

Notice what happens:

  • the function does not run all at once
  • it runs until the first yield
  • then it pauses
  • next(gen) starts it again from the same place

After all values are yielded, the generator is exhausted.

Generator function vs normal function

A normal function gives one final result with return.

A generator function can give many results over time with yield.

Normal function

def get_numbers():
    return [1, 2, 3]

result = get_numbers()
print(result)

Output:

[1, 2, 3]

Generator function

def get_numbers():
    yield 1
    yield 2
    yield 3

result = get_numbers()
print(result)

Possible output:

<generator object get_numbers at 0x...>

Calling a generator function does not run all its code immediately. It returns a generator object.

You can then loop over it:

def get_numbers():
    yield 1
    yield 2
    yield 3

for number in get_numbers():
    print(number)

Output:

1
2
3

Looping through a generator

The easiest way to use a generator is with a for loop.

def count_up_to(limit):
    number = 1
    while number <= limit:
        yield number
        number += 1

for value in count_up_to(5):
    print(value)

Output:

1
2
3
4
5

You can also use next() to get one value at a time:

def count_up_to(limit):
    number = 1
    while number <= limit:
        yield number
        number += 1

gen = count_up_to(3)

print(next(gen))
print(next(gen))
print(next(gen))

Output:

1
2
3

If you call next() again after the generator is finished, Python raises StopIteration.

gen = count_up_to(1)

print(next(gen))
print(next(gen))

For beginners, a for loop is usually the best choice because it stops automatically when the generator is done. If you want to understand this error better, see StopIteration in Python explained.

Generator expressions

A generator expression is similar to a list comprehension, but it uses round brackets instead of square brackets.

List comprehension

numbers = [1, 2, 3, 4]
doubled = [x * 2 for x in numbers]
print(doubled)

Output:

[2, 4, 6, 8]

Generator expression

numbers = [1, 2, 3, 4]
doubled = (x * 2 for x in numbers)

for value in doubled:
    print(value)

Output:

2
4
6
8

A generator expression creates values lazily. That means the values are produced when needed, not all at once.

Use this when you only need to loop through the results.

If you want to compare this more directly with list creation, see list comprehensions in Python explained.

When to use generators

Generators are a good choice when you want to process data step by step.

Use generators when:

  • you are working with large amounts of data
  • you only need each value once
  • you want to avoid storing everything at the same time
  • you are reading data gradually, such as lines from a file

Example:

def even_numbers(limit):
    for number in range(2, limit + 1, 2):
        yield number

for value in even_numbers(10):
    print(value)

Output:

2
4
6
8
10

Use a list instead when:

  • you need indexing like items[0]
  • you need to reuse the values many times
  • you want all results available immediately

Limits beginners should know

Generators are useful, but they are not the same as lists.

Important limits:

  • a generator does not support indexing like items[0]
  • once fully used, a generator cannot be restarted
  • if you need the values again, create a new generator
  • you can convert a generator to a list with list(generator) if needed

Example of converting to a list:

def count_up_to(limit):
    number = 1
    while number <= limit:
        yield number
        number += 1

gen = count_up_to(4)
numbers = list(gen)

print(numbers)

Output:

[1, 2, 3, 4]

Be careful: converting a generator to a list uses up the generator.

gen = count_up_to(3)

print(list(gen))
print(list(gen))

Output:

[1, 2, 3]
[]

The second result is empty because the generator was already exhausted.

Common mistakes

These are some common beginner mistakes with generators:

  • confusing yield with return
  • trying to reuse an exhausted generator
  • trying to index a generator like a list
  • expecting a generator expression to print all values by itself
  • using next() too many times and hitting StopIteration

Useful debugging checks:

def count_up_to(limit):
    number = 1
    while number <= limit:
        yield number
        number += 1

generator_obj = count_up_to(3)

print(generator_obj)
print(type(generator_obj))
print(next(generator_obj))
print(list(generator_obj))

Possible output:

<generator object count_up_to at 0x...>
<class 'generator'>
1
[2, 3]

This example shows an important detail:

  • next(generator_obj) uses the first value
  • list(generator_obj) only gets the remaining values

If you are confused about why generators work this way, reading about iterators and iterable objects explained can help.

FAQ

What is the difference between a generator and a list?

A list stores all values immediately. A generator produces values one at a time when needed.

Does yield end the function?

No. yield pauses the function. The function continues later from the same place.

Can I loop through a generator more than once?

Not the same exhausted generator. Create a new generator if you need to loop again.

Should beginners use generators?

Yes, for simple cases. Start with basic yield examples and use them when values should be produced one at a time.

See also