Summary: In this tutorial, we will learn what iterable, iterator, and iteration are in Python and how they differ from each other with the help of examples.
When we fetch one item of something one after another it is called iteration. For example, when we use for loop on a list or string to go over its items, it is iteration.
An iterator is an object in Python that has a stream of data and
__next__ method to return the next value in the iteration.
Whereas iterable is a Python object that has a
__iter__ method which returns an iterator object or a
__getitem__ method which implements efficient element access using integer indices.
To do the iteration we use the
for loop (or other constructs such as
while) with an iterable object that returns an iterator.
Consider the following code as an example:
>>> l = [1, 2, 3, 4, 5] >>> for x in l: ... print(x*2, end=' ') 2 4 6 8 10
Here, the list
l is an iterable object. When we use it with
for statement, the
for statement automatically calls the
__iter__ method on
l and creates a temporary unnamed variable to hold the returned iterator object for the duration of the loop.
for statement then uses the
__next__ method on this iterator object to do the iteration i.e. fetch one item after another.
We cannot directly use constructs such as
while, etc, with an iterator object. We use them with an iterable that returns an iterator.
It is then the iterator object that is responsible for returning the successive item from the stream of data.
To get this clear, let’s try writing a custom iterator and iterable in Python.
Custom Iterator and Iterable in Python
An iterator must have the
__next__ method to return successive items from the stream of data. So, we create a class and define
__next__ method to create a custom iterator as follows:
class MyIterator: def __init__(self): self.index = 0 def __next__(self): self.index += 1 return self.index
Here, we have successfully created a custom iterator that returns the next natural number on each successive call.
If we use the
next() method on the iterator object, we see that
__next__ returns the successive natural number on each call.
>>> iterator = MyIterator() >>> >>> next(iterator) 1 >>> next(iterator) 2 >>> next(iterator) 3 >>> next(iterator) 4
The next() is a buit-in function in Python that retrieve the next item from the iterator by calling its
The above custom iterator produces an infinite stream of natural numbers. To be finite, it must raise the
class MyIterator: def __init__(self): self.index = 0 def __next__(self): if self.index >= 5: raise StopIteration self.index += 1 return self.index
Now if we use
next() on the iterator more than 5 times, we get the
>>> iterator = MyIterator() >>> >>> next(iterator) 1 >>> next(iterator) 2 >>> next(iterator) 3 >>> next(iterator) 4 >>> next(iterator) 5 >>> next(iterator) Traceback (most recent call last): StopIteration
This exception is used by the constructs such as
list, etc, to stop the iteration.
However as discussed, we cannot directly use the iterator object with the
for statement because
for and other constructs in Python accept an iterable that return an iterator.
>>> for x in iterator: ... print(x) TypeError: 'MyIterator' object is not iterable
To make this work, we have to return this iterator object from an iterable.
As we know iterable is an object that has
__iter__ method which returns an iterator object, so we create a class with
__iter__ method which returns the above iterator.
class MyIterable: def __iter__(self): self.index = 0 return self def __next__(self): if self.index >= 5: raise StopIteration self.index += 1 return self.index
Now if we use
for statement with the object of this class, we see the iteration happens without any error.
>>> iterable = MyIterable() >>> for x in iterable: ... print(x, end=' ') 1 2 3 4 5
MyIterable class is similar to the
MyIterator class, the only difference is that it has an additional
__iter__ method that returns the ‘self’ object i.e. iterator object.
It can be confusing how ‘
self‘ is an iterator object, because we have defined
__iter__ in the class to make it an iterable.
It is because in Python we cannot use iterator without having
__iter__ method. This makes the iterator an iterable in Python.
Is an iterator always iterable?
Every iterator in Python is iterable because an iterator is required to have an
__iter__ method to return the iterator object.
However, the vice versa is not true. We can create iterable without the iterator by using the
__getitem__ method in Python.
For example in the following custom iterable class, we have not defined an
__iter__ method to return an iterator object, but have defined a
__getitem__ method to return the item at the given integer index needed by the
class MyIterable: def __getitem__(self, index): if index >= 5: raise StopIteration return index*2
If we do the iteration on the object of this class we will get a sequence of integer indices multiplied by 2 as the output.
>>> iterable = MyIterable() >>> for x in iterable: ... print(x, end=' ') 0 2 4 6 8
In summary, the iterable is a Python object that returns an iterator for constructs such as
list, etc, to do the iteration in Python. An iterable has the
__iter__ method to return the iterator object and an iterator has the
__next__ method to return the successive items in the stream.