Python Generators

Python Generator Tutorial for Beginners :

This tutorial should help you learn, create, and use Python Generator functions and expressions. It also covers some essential facts, such as why and when to use them in programs. And what makes a generator different from an iterator and a regular function.

Moreover, you would also get to know about the Python yield statement in this tutorial. It is a part of the Generator function and replaces the return keyword. Whenever the control hits the yield call, the function goes in a suspended state. However, it resumes from the same context point if called again.

A generator in Python is a function with unique abilities. We can either suspend or resume it at run-time. It returns an iterator object which we can step through and access a single value in each iteration.

How to Create a Generator in Python?

Python generator gives an alternative and simple approach to return iterators. The procedure to create the generator is as simple as writing a regular function.

There are two straightforward ways to create generators in Python.

Generator Function :

We write a generator in the same style as we write a user-defined function.

The difference is that we use the yield statement instead of the return. It notifies Python interpreter that the function is a generator and returns an iterator.

Generator Expression :

Python allows writing generator expressions to create anonymous generator functions.

This procedure is similar to a lambda function creating an anonymous function.

The syntax of a generator expression is the same as of list comprehension in Python. However, the former uses the round parentheses instead of square brackets.

While executing the above example, firstly, the list comprehension returns the list containing the square roots of all elements. So we get the conclusive result here.

Next, the generator expression produces an iterator object which gives one result at a time. The size of the list is four. So we have four successive next() method calls which print the square root of respective list elements.

Since we called the next() method one more time, it caused the StopIteration exception. Please check from the below output.

How to Use Generator in Python?

We now know how to create a generator. Let’s now focus on using it in programs. In the above coding snippets, we’ve used the next() method on the iterator returned by the generator function.

Using next() Method :

It (next()) is the most common way we can request a value from the generator function. Calling the next() method triggers its execution, which in turn gives a result back to the caller.

Return vs. Yield

The return is a final statement of a function. It provides a way to send some value back. While returning, its local stack also gets flushed. And any new call will begin execution from the very first statement.

On the contrary, the yield preserves the state between subsequent function calls. It resumes execution from the point where it gave back the control to the caller, i.e., right after the last yield statement. Also, read the detailed explanation in the below post.

Generator vs. Function

We have listed down a few facts to let you understand the difference between a generator and a regular function.

• A generator uses the yield statement to send a value back to the caller whereas a function does it using the return.
• The generator function can have one or more than one yield call.

• The yield call pauses the execution and returns an iterator, whereas the return statement is the last one to be executed.
• The next() method call triggers the execution of the generator function.
• Local variables and their states retain between successive calls to the next() method.
• Any additional call to the next() will raise the StopIteration exception if the there is no further item to process.

When to use a Generator?

There are many use cases where generators can be useful. We have mentioned some of them here:

• Generators can help to process large amounts of data. They can let us do the calculation when we want, also known as the lazy evaluation. The stream processing uses this approach.
• We can also stack the generators one by one and use them as pipes as we do with the Unix pipes.
• The generators can also let us establish concurrency.
• We can utilize Generators for reading a vast amount of large files. It will help in keeping the code cleaner and leaner by splitting the entire process into smaller entities.

• Generators are super useful for web scraping and help increasing crawl efficiency. They can allow us to fetch the single page, do some operation, and move on to the next. This approach is far more efficient and straightforward than retrieving all pages at once and then use another loop to process them.

Why use Generators?

Generators provide many programming-level benefits and extend many run-time advantages which influence programmers to use them.

Programmer-friendly :

It seems like a complicated concept, but the truth is that you can easily incorporate them into programs. They are a perfect alternative for the iterators.

Generator Pipeline :

With the help of generators, we can create a pipeline of different operations. It is a cleaner way to sub-divide responsibilities among various components and then integrates them to achieve the desired result

Now, it’s up to your imagination how well and what you like to accomplish with this cool Python Generator feature.

Summary :

Generators can produce a sequence on the fly and allow us to access one of its items whenever we need it. While doing so, it doesn’t consume a lot of memory and still gives us the ability to work with infinite streams of data. All in all, it is a trickier programming concept and worth trying in projects.

Finally, we highly recommend you reading about the iterators. They are one of the basic building blocks in Python and also an older sibling to generators. You can go through the details in the below tutorial.