What are Iterable, Iterator, and Iteration in Python?

Summary: In this tutorial, we will learn what iterable, iterator, and iteration are in Python and how they differ from each other with the help of examples.

When we fetch one item of something one after another it is called iteration. For example, when we use for loop on a list or string to go over its items, it is iteration.

An iterator is an object in Python that has a stream of data and __next__ method to return the next value in the iteration.

Whereas iterable is a Python object that has a __iter__ method which returns an iterator object or a __getitem__ method which implements efficient element access using integer indices.

To do the iteration we use the for loop (or other constructs such as while) with an iterable object that returns an iterator.

Consider the following code as an example:

>>> l = [1, 2, 3, 4, 5]
>>> for x in l:
...    print(x*2, end=' ')
2 4 6 8 10

Here, the list l is an iterable object. When we use it with for statement, the for statement automatically calls the __iter__ method on l and creates a temporary unnamed variable to hold the returned iterator object for the duration of the loop.

The for statement then uses the __next__ method on this iterator object to do the iteration i.e. fetch one item after another.

We cannot directly use constructs such as list, while, etc, with an iterator object. We use them with an iterable that returns an iterator.

It is then the iterator object that is responsible for returning the successive item from the stream of data.

To get this clear, let’s try writing a custom iterator and iterable in Python.

Custom Iterator and Iterable in Python

An iterator must have the __next__ method to return successive items from the stream of data. So, we create a class and define __next__ method to create a custom iterator as follows:

class MyIterator:
    def __init__(self):
        self.index = 0
        
    def __next__(self):
        self.index += 1
        return self.index

Here, we have successfully created a custom iterator that returns the next natural number on each successive call.

If we use the next() method on the iterator object, we see that __next__ returns the successive natural number on each call.

>>> iterator = MyIterator()
>>>
>>> next(iterator)
1
>>> next(iterator)
2
>>> next(iterator)
3
>>> next(iterator)
4

The next() is a buit-in function in Python that retrieve the next item from the iterator by calling its __next__ method.

The above custom iterator produces an infinite stream of natural numbers. To be finite, it must raise the StopIteration exception.

class MyIterator:
    def __init__(self):
        self.index = 0
        
    def __next__(self):
        if self.index >= 5:
            raise StopIteration
            
        self.index += 1
        return self.index

Now if we use next() on the iterator more than 5 times, we get the StopIteration exception.

>>> iterator = MyIterator()
>>>
>>> next(iterator)
1
>>> next(iterator)
2
>>> next(iterator)
3
>>> next(iterator)
4
>>> next(iterator)
5
>>> next(iterator)
Traceback (most recent call last):
    StopIteration

This exception is used by the constructs such as for, while, list, etc, to stop the iteration.

However as discussed, we cannot directly use the iterator object with the for statement because for and other constructs in Python accept an iterable that return an iterator.

>>> for x in iterator:
...     print(x)
TypeError: 'MyIterator' object is not iterable

To make this work, we have to return this iterator object from an iterable.

As we know iterable is an object that has __iter__ method which returns an iterator object, so we create a class with __iter__ method which returns the above iterator.

class MyIterable:
    def __iter__(self):
        self.index = 0
        return self
        
    def __next__(self):
        if self.index >= 5:
            raise StopIteration
            
        self.index += 1
        return self.index

Now if we use for statement with the object of this class, we see the iteration happens without any error.

>>> iterable = MyIterable()
>>> for x in iterable:
...     print(x, end=' ')
1 2 3 4 5

The MyIterable class is similar to the MyIterator class, the only difference is that it has an additional __iter__ method that returns the ‘self’ object i.e. iterator object.

It can be confusing how ‘self‘ is an iterator object, because we have defined __iter__ in the class to make it an iterable.

It is because in Python we cannot use iterator without having __iter__ method. This makes the iterator an iterable in Python.

Is an iterator always iterable?

Every iterator in Python is iterable because an iterator is required to have an __iter__ method to return the iterator object.

However, the vice versa is not true. We can create iterable without the iterator by using the __getitem__ method in Python.

For example in the following custom iterable class, we have not defined an __iter__ method to return an iterator object, but have defined a __getitem__ method to return the item at the given integer index needed by the for statement:

class MyIterable:
    def __getitem__(self, index):
        if index >= 5:
            raise StopIteration
    
        return index*2

If we do the iteration on the object of this class we will get a sequence of integer indices multiplied by 2 as the output.

>>> iterable = MyIterable()
>>> for x in iterable:
...     print(x, end=' ')
0 2 4 6 8

In summary, the iterable is a Python object that returns an iterator for constructs such as for, while, list, etc, to do the iteration in Python. An iterable has the __iter__ method to return the iterator object and an iterator has the __next__ method to return the successive items in the stream.