Closures and Decorators

Closures and Decorators in Python

This article will attempt to explain the related concepts of Closures and Decorators in the Python program language. It is essential to understand that functions, like literally everything else in Python, are, in fact, objects. As such, they can be passed to other functions as arguments. This fact opens up some interesting possibilities (such as the concept of closures and decorators).

It is critical to have a handle on closures to understand decorators, so we will start there.

Closures

A closure allows the reference of the scope of an outer function, from an inner function to be accessed by that inner function when it has been returned and is being used outside of the outer function. That sounds more complicated than it is. Here is an example:

def divide(divisor):
    def inner(dividend):
        return dividend/divisor
    return inner


div_2 = divide(2)
div_3 = divide(3)

print(div_2(6))
print(div_3(6))

Output

3
2

In the above example, we called the divide function twice. The first time we called it with a divisor argument set to 2 and assigned the value, it returned to div_2. The second time we called it, we used a divisor argument of 3 and assigned the value it returned to div_3. You will also note that in the inner function, we referenced the outer functions divisor argument as part of the calculation. Finally, or firstly depending on how you read the code, we returned the inner function. Important: Note this is not calling the inner function but returning the function itself.

When we assign div_2 to divide(2), we are actually returning a function with a signature of inner(dividend) to and assigning that function to div_2. For div_3, it is the same signature. But, the vital point about closures is that the environment from the enclosing function is available to the inner function – even after the outer function has finished running. So the copy of inner that is assigned to div_2 knows about the divisor argument and retains that reference even after divide completes. Additionally, as demonstrated by the second call to divide, with the divisor of 3 assigned to div_3, this variable is its own reference based on the call in which the inner function was returned.

Finally, we utilize these two references to inner, now in div_2 and div_3, and provide a dividend for the first time. The dividend we are providing (in both cases) is the number six (this is arbitrary and could have been different dividends). You will note from the output our original divisor is used in the division, and we get the correct results of 3 (6/2) and 2 (6/3) when we call these functions.

Decorators

Decorators make use of closures to add functionality to the decorated function. We can use a decorator to modify the behavior of a function call to a specific function without changing the original function or modifying the calling function.

Let’s start with a function to divide two numbers.

def divide(dividend, divisor):
    return float(dividend)/divisor

print(divide(2,3))

Output:

0.666666666667

The above works – but what if we instead provided zero as the second parameter to divide?

print(divide(2,0))

Output:

Traceback (most recent call last):
 File "/Users/jeff/Python_Tutorials/CleanCode/clsoures_scratch.py", line 18, in <module>
 print(divide(2,0))
 File "/Users/jeff/Python_Tutorials/CleanCode/clsoures_scratch.py", line 16, in divide
 return float(dividend)/divisor
ZeroDivisionError: float division by zero

Let’s say that we wanted to handle this error, but for some reason, we couldn’t do it in the calling code or modify the divide function. We could do something like this:

def checkDiv0(func):
    def inner_func(dividend, divisor):
        if divisor != 0:
            return func(dividend, divisor)
        else:
            return "You cannot divide by zero!"
    return inner_func


@checkDiv0
def divide(dividend, divisor):
    return float(dividend)/divisor


print(divide(2, 3))
print(divide(2, 0))

Output:

0.666666666667
You cannot divide by zero!

Let’s unpack this. Starting with the

print(divide(2, 3)

statement. In this statement, we call divide, but Python provides us a bit of syntactic sugar (as the cool kids say) in that by providing the @checkDiv0 directive directly preceding the divide function definition we are in fact not calling divide anymore – we are calling checkDiv0 with the divide function as the argument. From there, we call return inner_func; that is the function that is actually being called with the arguments provided. The divisor is then checked to ensure it isn’t zero and then, and only then, is the actual divide function called in the line

return func(dividend, divisor)

. Finally, our result is calculated and returned to inner_func, and inner_func returns the value to our print statement where it is displayed. You will note that when we use the func
identifier in the line

return func(dividend, divisor)

we are using the concept of an enclosure, described above. Because by the time we call inner_func checkDiv0 is has ended. But, its reference to divide is retained, and that is how we are calling back to our original function.

If we pass zero as an argument, as we do in this statement

print(divide(2, 0))

the same thing happens. checkDiv0 is called and returns the inner function, which is in turn called with our arguments, the divisor being zero now. When the initial if condition fails to return true

 if divisor != 0

we drop into the else block. In this case, instead of calling our original divide function (which would result in an exception) we return a string that indicates that "You cannot divide by zero!". The message is then printed to the screen in our print statement. In this case, our decorator actually prevented the divide function from ever executing.

A slightly more complex example of a decorator

In the above example, we checkDiv0, itself, took no arguments other than the function reference. If you wanted to have the decorator function take arguments as well, you could do that; we will explore an example that does that here. Let’s first start by implementing a trivial base function that wraps random.randint and returns a random number between 1 and 10.

import random

def randInt():
    return random.randint(1, 10)

print(randInt())

Output: (your output may vary)

8

Now let’s say we wanted to use this function, but we wanted to get the square of random integers, rather than just the random integers themselves. We additionally wish to return this in a tuple so we can see what the original random number was as well as the calculated square. To do this, we need to nest another function within our decorator function. The outer function will accept the argument to the decorator. The next level function will accept the function we are decorating, and finally, the third level function will actually be the function that is returned for our code to run.

import random


def pow(exponent):
    def pow_inner(func):
        def inner():
            funcVal = func()
            return funcVal, funcVal**exponent
        return inner
    return pow_inner

@pow(2)
def randInt():
    return random.randint(1, 10)


print(randInt())

Output: (your output may vary)

(8, 64)

In this example, we provide an argument 2 to a function we defined to decorate our RandInt() function – this caused the value to be squared and a tuple containing the function value and the squared function value to be returned. It should be obvious we could provide any exponent we wanted to this function. But the original spec indicated we only wanted to square the function, and it might be helpful to allow the decorator to simply be @pow to square the number and to accept an argument optionally.

To do this, we need to evaluate the argument to pow. If that argument is a callable (a function), then we know an exponent wasn’t provided, and we can simply call pow_inner with the argument supplied to pow – but first we must set exponent to a default value (we will use 2). Otherwise, we keep the logic the same.

import random


def pow(exponent):
    def pow_inner(func):
        def inner():
            funcVal = func()
            return funcVal, funcVal**exponent
        return inner

    if callable(exponent):
        func = exponent
        exponent = 2
        return pow_inner(func)
    else:
        return pow_inner


@pow
def randInt():
    return random.randint(1, 10)


print(randInt())

Output: (your output may vary)

(5, 25)

Note we can still provide an exponent – in this example, we will cube the results (the code, aside from the

@pow(3)

line is exactly the same as above).

import random


def pow(exponent):
    def pow_inner(func):
        def inner():
            funcVal = func()
            return funcVal, funcVal**exponent
    return inner

    if callable(exponent):
        func = exponent
        exponent = 2
        return pow_inner(func)
    else:
        return pow_inner


@pow(3)
def randInt():
    return random.randint(1, 10)


print(randInt())

Output: (your output may vary)

(10, 1000)

So now we have used a decorator that optionally accepts an argument (it could take more than one) to change the functionality of a function that we haven’t modified and avoided changing anything about how we were initially calling that function.

Conclusion

If I am honest, I struggle with identifying a use case where I would use decorators or enclosures in non-trivial examples in my own code. But I do use the built-in decorators in Python quite a lot. One such example is @contextmanager in the contextlibs module, which you can use in your code to create a context manager against a resource simply by including the decorator and yielding the resource. Python also implements @property to help you create property methods in your classes, and many other language features are implemented as decorators. Having an understanding of what is actually happening when you use this syntax is, for me at least, helpful in grokking what is happening with my code.