第七章－函数装饰器和闭包.md

CHAPTER 7 Function decorators and closures

第七章 函数装饰器和必包

There’s been a number of complaints about the choice of the name “decorator” for this feature. The major one is that the name is not consistent with its use in the GoF book[1]. The name decorator probably owes more to its use in the compiler area — a syntax tree is walked and annotated. — PEP 318 — Decorators for Functions and Methods

对于这个功能来说，就存在很多关于“装饰器”取名上的抱怨。最重要的一条就是这个名称和GoF真本书中所讲的不一致［注释1］。

[1]. That’s the 1995 “Design Patterns” book by the so-called Gang of Four.

Function decorators let us “mark” functions in the source code to enhance their behavior is some way. This is powerful stuff, but mastering it requires understanding closures.

函数装饰器让我们在源码中“标记”函数以便在某种程度上增强这些函数的行为。这东西很强大，但是精通它需要理解闭包。

One of the newest reserved keywords in Python is nonlocal, introduced in Python 3.0. You can have a profitable life as a Python programmer without ever using it if you adhere to a strict regimen of class-centered object orientation. However, if you want to imple‐ ment your own function decorators, you must know closures inside out, and then the need for nonlocal becomes obvious.

在Python中的最新保留关键字是在3.0版本中引入的nonlocal。

Aside from their application in decorators, closures are also essential for effective asynchronous programming with callbacks, and for coding in a functional style whenever it makes sense.

除了

The end goal of this chapter is to explain exactly how function decorators work, from the simplest registration decorators to the rather more complicated parametrized ones. However, before we reach that goal we need to cover:

本章的最终目标是明明白白解释函数装饰器是如何工作的，从最简单的注册装饰器到

• How Python evaluates decorator syntax
• How Python decides whether a variable is local
• Why closures exist and how they work
• What problem is solved by nonlocal

• Python如何计算装饰器语法
• Python如何确定一个变量是否为本地变量
• 为什么会存在闭包，以及它们如何工作的
• 什么问题可以由nolocal来解决

With this grounding, we can tackle further decorator topics:

有了这个基础，我们可以进一步深入的装饰器话题：

• Implementing a well-behaved decorator
• Interesting decorators in the standard library • Implementing a parametrized decorator

• 实现一个良好行为的装饰器
• 标准库中装饰器的意义
• 实现一个参数化的装饰器

We start with a very basic introduction to decorators, and then proceed with the rest of the items listed here.

我们从介绍非常简单的装饰器开始，然后继续此处列出的内容。

Decorators 101 装饰器基础

A decorator is a callable that takes another function as argument (the decorated func‐ tion) [2]. The decorator may perform some processing with the decorated function, and returns it or replaces it with another function or callable object.

装饰器是一个接受另外函数做为参数（被装饰的函数）的可调用对象［2］。装饰器可能会对被装饰器函数执行一些操作，然后返回它或者将其替换为另外的函数、抑或可调用对象。

[2]: Python also supports class decorators. They are covered in Chapter 21.
注释［2］：Python还支持类装饰器。它们出现在第21章。

In other words, assuming an existing decorator named decorate, this code:

换句话说，假设存在一个名称为decorate的装饰器，即代码：

@decorate
def target():
    print('running target()')

Has the same effect as writing this:
具有相同效果的代码可以这样编写：

def target():
    print('running target()')
    
target = decorate(target)

The end result is the same: at the end of either of these snippets, the target name does not necessarily refer to the original target function, but to whatever function is returned by decorate(target).

最终的结果是一样的：这些代码片段中其中一个的结尾，变量名target没有引用原始target的必要，而是由decorate(target)返回的任意函数。

To confirm that the decorated function is replaced, see the console session in Example 7-1.

要确认被装饰函数的替换，参见例子7－1中的控制台会话。

Example 7-1. A decorator usually replaces a function with a different one.
例子7-1. 装饰器通常把函数替换为另外一个函数。

>>> def deco(func):
...     def inner():
...			print('running inner()')
...     return inner #1
>>> @deco
... def target(): #2
...     print('running target()')
...
>>> target() #3
running inner()
>>> target #4
<function deco.<locals>.inner at 0x10063b598>

1: deco returns its inner function object.
deco返回其内部的函数对象。

2: target is decorated by deco.
taget被deco所装饰。

3: Invoking the decorated target actually runs inner. 调用被装饰器target，而实际上执行的则是inner。

4: Inspection reveals that target is a now a reference to inner.
检查显示target现在是一个到inner的引用。

Strictly speaking, decorators are just syntactic sugar. As we just saw, you can always simply call a decorator like any regular callable, passing another function. Sometimes that is actually convenient, especially when doing metaprogramming — changing pro‐ gram behavior at run-time.

严格上来讲，装饰器只是语法糖而已。就像我们刚才看到的那样，你随时都可以像调用普通可调用对象那样简单的调用一个装饰器，传递另外的函数。有时候这样做实际上非常方便，特别是在执行元编程——在运行时改变程序的行为。

To summarize: the first crucial fact about decorators is that they have the power to replace the decorated function with a different one. The second crucial fact is that they are executed immediately when a module is loaded. This is explained next.

总结：关于装饰器的第一个重要因素是它们有能力使用一个不同的函数来替换被装饰器的函数。第二个重要的因素是它们在模块载入时就被立即执行了。这种情况在下面有所解释。

When Python executes decorators

Python执行装饰器的时机

A key feature of decorators is that they run right after the decorated function is defined. That is usually at import time, i.e. when a module is loaded by Python. Consider registration.py in Example 7-2.

装饰器的一个关键特征是它们在被装饰器函数被定义之后就可以正确的运行。这通常发生在导入时，比如，一个模块被Python载入时。思考例子7－2中的registration.py。

Example 7-2. The registration.py module
例子7－2。 registration.py模块。

registry = []


def register(func):
	  print('running register(%s)' % func) 
    registry.append(func)
	  return func


@register
def f1():
    print('running f1()')
  

@register
def f2():
    print('running f2()')
    

def f3():
    print('running f3()')


def main():
    print('running main()') 
    print('registry ->', registry) 
    f1()
    f2()
    f3()
    
if __name__=='__main__': 
    main()

1: registry will hold references to functions decorated by @register
registry通过@register能够拥有被装饰函数的引用。

2: register takes a function as argument.
register将函数作为参数。

3: Display what function is being decorated, for demonstration.
说明什么函数正在被装饰。

4: Include func in registry.
在register内继承函数。

5: Return func: we must return a function, here we return the same received as argument.
返回func：我们必须返回一个函数，这里我们返回

6: f1 and f2 are decorated by @register.
f1和f2由@register装饰。

7: f3 is not decorated.
f3未被装饰。

8: main displays the registry, then calls f1(), f2() and f3().
main显示registry，然后调用f1(), f2() 和 f3()。

9: main() is only invoked if registration.py runs as a script.
如果registration.py当做脚本运行，那么main()是唯一被调用的。

The output of running registration.py as a script looks like this:

把registration.py当做脚本运行的输出结果如下：

$ python3 registration.py
running register(<function f1 at 0x100631bf8>)
running register(<function f2 at 0x100631c80>)
running main()
registry -> [<function f1 at 0x100631bf8>, <function f2 at 0x100631c80>] running f1()
running f2()
running f3()

Note that register runs (twice) before any other function in the module. When reg ister is called, it receives as argument the function object being decorated, eg. <func tion f1 at 0x100631bf8>.

注意register在模块中其他任意函数之前运行（了两次）。当register被调用时，

After the module is loaded, the registry holds references to the two decorated functions: f1 and f2. These functions, as well as f3, are only executed when explicitly called by main.

在模块载入之后，registry拥有对两个被装饰函数引用：f1和f2。这些函数，包括f3也是如此，它们都尽在明确调用main`时被执行。

If registration.py is imported (and not run as a script), the output is this:

如果导入registration.py，其输出如下：

>>> import registration
running register(<function f1 at 0x10063b1e0>) running register(<function f2 at 0x10063b268>)

At this time, if you look at the registry, here is what you get:

这时候，如果你去浏览registry，下面就是你会得到的结果：

>>> registration.registry
[<function f1 at 0x10063b1e0>, <function f2 at 0x10063b268>]

The main point of Example 7-2 is to emphasize that function decorators are executed as soon as the module is imported, but the decorated functions only run when they are explicitly invoked. This highlights the difference between what Pythonistas call import time and run time.

例子7-2的重点在于强调函数装饰器在模块被导入时就立刻被执行了，但是被装饰的函数尽在它们被显式地调用时才会运行。这就是Python支持者们提出所谓的import time 和 run time之间的重要区别。

Considering how decorators are commonly employed in real code, Example 7-2 is unusual in two ways:

考虑下装饰器通常是如何使用在真实代码中的，例子7-2的不一般体现在下面两个方面：

• The decorator function is defined in the same module as the decorated functions. A real decorator is usually defined in one module and applied to functions in other modules.
• The register decorator returns the same function passed as argument. In practice, most decorators define an inner function and return it.

• 装饰器函数和被装饰函数定义在相同的模块中。真实的装饰器通常定义在一个模块中，然后在另外一个模块中应用到函数上。
• register装饰器返回相同的被作为参数传递的函数。实际上，多数的装饰器定义一个inner函数，并返回它。

Even though the register decorator in Example 7-2 returns the decorated function unchanged, that technique is not useless. Similar decorators are used in many Python Web frameworks to add functions to some central registry, for example, a registry mapping URL patterns to functions that generate HTTP responses. Such registration decorators may or may not change the decorated function. The next section shows a practical example.

尽管例子7-2中的register装饰器返回的装饰函数并未改变，可实现技术也不是毫无用处。类似的装饰器被用在很多Python Web框架中，以便将函数加入到注册中心里，例如，映射URL模式到生成HTTP响应函数的注册。此类注册装饰器改变或者不改变被装饰器函数。接下来的小节演示了一个实际的例子。

Decorator-enhanced Strategy pattern

使用了装饰器的增强策略模式

A registration decorator is a good enhancement to the e-commerce promotional dis‐ count from “Case study: refactoring Strategy” on page 168.

注册装饰器能够很好的改善在168页中的“案例探究：重构策略”的商业促销降价。

Recall that our main issue with Example 6-6 is the repetition of the function names in their definitions and then in the promos list used by the best_promo function to determine the highest discount applicable. The repetition is problematic because someone may add a new promotional strategy function and forget to manually add it to the promos list — in which case best_promo will silently ignore the new strategy, introducing a subtle bug in the system. Example 6-8 solves this problem with a registration decorator.

回想我们在例子6－6中遇到的主要问题是定义中的函数名称重复，在promos列表中使用best_promo来确定最高的降价额度。重复表明有问题存在，因为当用户添加了一个新的降价促销策略函数而且又忘记将该函数手动添加到promos列表，在这种情况下best_promo会不带任何提示地忽略新的策略，并在系统中留下一个不易察觉的错误。例子6-8利用注册装饰器解决了这个问题。

Example 7-3. The promos list is filled by the promotion decorator.

例子7-3. promos列表使用了promotion装饰器进行过滤。

promos = []	＃1

def promotion(promo_func): ＃2
    promos.append(promo_func) 
    return    promo_func


@promotion
def fidelity(order):	＃3
	 """
   5% discount for customers with 1000 or more fidelity points
   对忠诚点数大于或者等于1000的消费者，执行5%的折扣
   """ 
   return   order.total() * .05 if order.customer.fidelity >= 1000 else 0

	 
@promotion
def bulk_item(order):
	 """
   10% discount for each LineItem with 20 or more units
   """ 
   discount = 0
	 for item in order.cart:
	 	 if item.quantity >= 20:
		 discount += item.total() * .1
		 return discount
		 

@promotion
def large_order(order):
	 """
   7% discount for orders with 10 or more distinct items
   对于购买10种以上不同商品的订单执行7%的折扣
   """ 
   distinct_items = {item.product for item in order.cart}
	 if len(distinct_items) >= 10:
	 	  return order.total() * .07 return 0


def best_promo(order):＃4
	 """Select best discount available"""
	 return max(promo(order) for promo in promos)

1：The promos list starts empty.
promos开始是空的。

2: promotion decorator returns promo_func unchanged, after adding it to the promos list.
promotion装饰器返回了未被改变的promo_func，然后将其加入到promos列表。

3: Any function decorated by @promotion will be added to promos.
任何由@promotion装饰器的函数都将被加入到promos。

4: No changes needed to best_promos, as it relies on the promos list 不需要对best_promo应用改变，因为它依据于promo列表

This solution has several advantages over the others presented in “Case study: refactoring Strategy” on page 168:

这个解决方案相较于出线在168页“案例探究：重构策略”中的其它解决方案有多个优点：

• The promotion strategy functions don’t have to use special names (like the _promo suffix).
• The @promotion decorator highlights the purpose of the decorated function, and also makes it easy to temporarily disable a promotion: just comment out the decorator.
• Promotional discount strategies may be defined in other modules, anywhere in the system, as long as the @promotion decorator is applied to them.

• promotion策略函数不必使用特殊名称（比如，_promo后缀）
• @promotion装饰器突出了被装饰函数的用途，也同时让临时禁用促销变得很简单：注释掉装饰器就好了。
• 促销打折策略也可以定义在系统中任意的其它模块中，只要 @promotion 装饰器可以应用到被装饰函数就好

Most decorators do change the decorated function. They usually do it by defining an inner function and returning it to replace the decorated function. Code that uses inner functions almost always depends on closures to operate correctly. To understand clo‐ sures, we need to take a step back a have a close look at how variable scopes work in Python.

多数的装饰器会改变被装饰函数。它们通常通过定义一个内部函数，并将其返回以替换被装饰函数。代码使用内部函数几乎大多数时候都依赖于闭包的正确操作。要理解闭包，我们需要回过头来仔细看一看变量是如何在Python中工作的。

Variable scope rules

变量域规则

In Example 7-4 we define and test a function which reads two variables: a local variable a, defined as function parameter, and variable b that is not defined anywhere in the function:

在例子7-4中我们定义并测试了能够读取两个变量的函数：本地变量a定义为函数参数，变量b并没有定义函数中的任何地方：

Example 7-4. Function reading a local and a global variable.

例子7-4。 函数读取一个本地和全局变量。

>>> def f1(a): 
>>> ... print(a) 
>>> ... print(b)
>>> ...
>>> f1(3)
3
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "<stdin>", line 3, in f1
NameError: global name 'b' is not defined

The error we got is not surprising. Continuing from Example 7-4, if we assign a value to a global b and then call f1, it works:

碰到的这个错误并不令人惊讶。接着例子7-4，如果我们对全局变量b赋予一个值，然后调用f1，函数就会起作用的：

>>>b=6 
>>> f1(3)
>>>  3
6

Now, let’s see an example that may surprise you.

现在，让我们看一个让你震惊的例子。

Take a look at the f2 function in Example 7-5. Its first two lines are the same as f1 in Example 7-4, then it makes an assignment to b, and prints its value. But it fails at the second print, before the assignment is made:

我们来看一看例子7-5中的函数f2.它的前两行和例子7－4中的相同，它对b进行赋值，并打印b的值。但是本地变量b在真正被赋值之前，在进行第二次打印会失败的：

Example 7-5. Variable b is local, because it is assigned a value in the body of the function.

例子7-5. 变量b是本地变量，因为它在函数的主体北被赋予了一个值。

>>> b = 6
>>> def f2(a):
...     print(a)
...     print(b)
...     b = 9
...
>>> f2(3)
3
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "<stdin>", line 3, in f2
UnboundLocalError: local variable 'b' referenced before assignment

Note that the output starts with 3, which proves that the print(a) statement was executed. But the second one, print(b) never runs. When I first saw this I was surprised, thinking that 6 should be printed, because there is a global variable b and the assignment to the local b is made after print(b).

注意输出是从3开始到，它证明了语句print(a)是可以执行的。但是第二个语句，print(b)根本没有运行。我第一次看到这种情况就被惊到了，我以为6应该打印出来，因为有一个全局变量b，而且本地变量b的赋值被放在了print(b)之后。

But the fact is, when Python compiles the body of the function, it decides that b is a local variable because it is assigned within the function. The generated bytecode reflects this decision and will try to fetch b from the local environment. Later, when the call f2(3) is made, the body of f2 fetches and prints the value of the local variable a, but when trying to fetch the value of local variable b it discovers that b is unbound.

但事实情况是，当Python编译函数主体时将b判定为本地变量，因为b实在函数内部赋值的。生成的字节码也反映了这个判定，而且从本地环境中获取b。之后，在调用f2(3)执行过后，答应本地变量a的值，但是在获取本地变量b的值时，Python发现b并未绑定。

This is not a bug, but a design choice: Python does not require you to declare variables, but assumes that a variable assigned in the body of a function is local. This is much better than the behavior of JavaScript, which does not require variable declarations either, but if you do forget to declare that a variable is local (with var), you may clobber a global variable without knowing.

这并不是一个bug，而是基于设计的考量：Python不会要求你申明变量，而是假定在函数主体内赋值的变量是本地变量。这比Javascript的行为好多了，虽然JS也不要求变量声明，但是如果你忘记声明变量为本地的（使用var），那么你会因为未知的全局变量而大受打击。

If we want the interpreter to treat b as a global variable in spite of the assignment within the function, we use the global declaration:

如果你希望解释器将b作为全局变量而不管函数中的赋值，我可以global进行声明：

>>> def f3(a):
...     global b
...     print(a)
...     print(b)
...     b = 9
...
>>> f3(3)
3
6
>>> b
9
>>> def f3(a):
...     global b
...     print(a)
...     print(b)
...     b = 30
...
>>> f3(3)
3
8
>>> b
30
>>>

After this closer look at how variable scopes work in Python, we can tackle closures in the next section, “Closures” on page 192. If you are curious about the bytecode differences between the functions in Example 7-4 and Example 7-5, see the following sidebar.

在仔细观察了Python中的变量域如何工作之后，我们就可以应对下一节的闭包，192页的“闭包”。如果你对例子7-4和例子7-5之间的字节码区别好奇，可以来看看下面的侧栏描述。

Comparing bytecodes 比较字节码

The dis module provides an easy way to disassemble the bytecode of Python functions. Read Example 7-6 and Example 7-7 to see the bytecodes for f1 and f2 from Example 7-4 and Example 7-5.

Example 7-6. Disassembly of the f1 function from Example 7-4.

dis模块提供了一种简单的可以把Python函数的字节码反汇编的方式。阅读例子7-6和例子7-7以查看来自例子7－4和例子7－5的f1和f2点字节码。 例子7-6. 反汇编例子7－4中的f1函数。

>>> dis(f1)
  2           0 LOAD_GLOBAL              0 (print) #1
              3 LOAD_FAST                0 (a) #2
              6 CALL_FUNCTION            1 (1 positional, 0 keyword pair)
              9 POP_TOP

  3          10 LOAD_GLOBAL              0 (print)
             13 LOAD_GLOBAL              1 (b) #3
             16 CALL_FUNCTION            1 (1 positional, 0 keyword pair)
             19 POP_TOP
             20 LOAD_CONST               0 (None)
             23 RETURN_VALUE

#1: Load global name print. #2: Load local name a. #3: Load global name b.

#1: 载入全局变量名print。 #2: 载入本地变量名a。 #3: 载入全局变量名b。

Contrast the bytecode for f1 shown in Example 7-6 with the bytecode for f2 in Example 7-6.
Example 7-7. Disassembly of the f2 function from Example 7-5.

对比例子7-6中出现的f1的字节码和例子7-6中的f2的字节码。 例子7－7. 反汇编例子7－5中的函数f2.

>>> dis(f2)
  2           0 LOAD_GLOBAL              0 (print)
              3 LOAD_FAST                0 (a)
              6 CALL_FUNCTION            1 (1 positional, 0 keyword pair)
              9 POP_TOP

  3          10 LOAD_GLOBAL              0 (print)
             13 LOAD_FAST                1 (b) #1
             16 CALL_FUNCTION            1 (1 positional, 0 keyword pair)
             19 POP_TOP

  4          20 LOAD_CONST               1 (9)
             23 STORE_FAST               1 (b)
             26 LOAD_CONST               0 (None)
             29 RETURN_VALUE

#1: Load local name b. This shows that the compiler considers b a local variable, even if the assignment to b occurs later, because the nature of the variable — whether it is local or not — cannot change body of the function.

#1: 载入本地本地变量名b。这表示编译器把b当做一个本地变量，即便是对b的赋值发生在后面，因为这是变量的特点——不论变量是否为本地变量，都不可能改变函数的主体。

The CPython VM that runs the bytecode is a stack machine, so the operations LOAD and POP refer to the stack. It is beyond the scope of this book to further describe the Python opcodes, but they are documented along with the dis module in dis — Disassembler for Python bytecode.

CPython虚拟机中运行字节代码的是栈机器，所以LOAD和POP操作都引用这个栈。深入描述Python的运算码超出了本书的范围，但是它们在使用了dis模块（反编译Python的字节码）中的dis都被有很好文档说明。

Closures 闭包

In the blogosphere closures are sometimes confused with anonymous functions. The reason why many confuse them is historic: defining functions inside functions is not so common, until you start using anonymous functions. And closures only matter when you have nested functions. So a lot of people learn both concepts at the same time.

在博客圈子里闭包有时候会和匿名函数搞混淆。人们会将它们之间弄混淆是有历史原因的：直到使用匿名函数之前，在函数内部定义函数不是件很普遍的事情。而闭包仅在你使用了嵌套函数时变得重要起来。因此很多人在同一时间学习了这两个概念。

Actually, a closure is function with an extended scope that encompasses non-global variables referenced in the body of the function but not defined there. It does not matter whether the function is anonymous or not, what matters is that it can access non-global variables that are defined outside of its body.

实际上，闭包是一个有着包含在函数体中但是未定义在该函数中的非全局变量引用的扩展作用域。函数是否匿名并不重要，重要的是它可以访问定义在函数体外部的非全局变量。

This is a challenging concept to grasp, and is better approached through an example.

要掌握概念是个挑战，通过实例是个掌握它的好办法。

Consider an avg function to compute the mean of an ever-increasing series of values, for example, the average closing price of a commodity over its entire history. Every day a new price is added, and the average is computed taking into account all prices so far.

考虑一个用来计算不断增长的一系列值的avg函数，例如，商品历史周期内的平均收盘价。每天添加一个新的价格，目前为止均价的计算是要考虑商品全部价格的。

Starting with a clean slate, this is how avg could be used:

透过清晰的均价列表，我们可以看到avg的用法：

>>> avg(10) 10.0
>>> avg(11) 10.5
>>> avg(12) 11.0

Where does avg come from, and where does it keep the history of previous values? For starters, Example 7-8 is a class-based implementation:

avg从来得来的，它又是如何保存前面值的历史？先上道开胃小菜，例子7-8是一个基于类的实现：

Example 7-8. average_oo.py: A class to calculate a running average.
例子7-8. average_oo.py：一个计算运行中的平均数。

class Averager():

    def __init__(self):
        self.series = []

    def __call__(self, new_value): 
        self.series.append(new_value) 
        total = sum(self.series) 
        return total/len(self.series)

The Averager class creates instances that are callable:

创建Averager类的可调用实例：

>>> avg = Averager() 
>>> avg(10)
10.0
>>> avg(11)
10.5
>>> avg(12) 
11.0

Now, Example 7-9 is a functional implementation, using a higher order function make_averager:

现在你来看，例子7-9是个使用高阶函数make_averager的函数化实现：

Example 7-9. average.py: a higher-order function to calculate a running average.
例子7-9. average.py：计算平均数的高阶函数。

def make_averager():  
    series = []
	 
    def averager(new_value): 
        series.append(new_value) 
        total = sum(series) 
        return total/len(series)

    return averager

When invoked, make_averager returns an averager function object. Each time an averager is called, it appends the passed argument to the series, and computes the current average:

当调用make_averager函数时，返回的是averager函数对象。每次averager被调用时，它会将传递的参数追加到series，并计算当前的平均数。

Example 7-10. Testing Example 7-9

>>> avg = make_averager() 
>>> avg(10)
10.0
>>> avg(11)
10.5
>>> avg(12) 
11.0

Note the similarities of the examples: we call Averager() or make_averager() to get a callable object avg that will update the historical series and calculate the current mean. In Example 7-8, avg is an instance of Averager, and in Example 7-9 it is the inner function, averager. Either way, we just call avg(n) to include n in the series and get the updated mean.

注意例子的相似之处：我们调用Averager()或者 make_averager()来获取一个可调用的对象avg，而调用将更新旧的series并计算当前的结果。在例子7-8中，avg是Averager的一个实例，而在例子7-9中，它则是内部函数averager。不论哪种方式，我们都是调用avg(n)来把n包括在series中，并计算更新后的结果。

It’s obvious where the avg of the Averager class keeps the history: the self.series instance attribute. But where does the avg function in the second example find the series?

Averager类的avg保存历史的地方很明显：self.series实例属性。但是第二个例子中的avg函数是在哪里发现的series的呢？

Note that series is a local variable of make_averager because the initialization series = [] happens in the body of that function. But when avg(10) is called, make_averager has already returned, its local scope is long gone.

注意series是make_averager的本地变量，因为初始化series = []发生在这个函数体内。但是当avg(10)被调用时，make_averager已经执行返回了，而且本地作用域早就不存在了。

Figure 7-1. The closure for averager extends the scope of that function to include the binding for the free variable series.

图7-1.

Within averager, series is a free variable. This is a technical term meaning a variable that is not bound in the local scope. See Figure 7-1.

在averrager内部，series是一个自由变量。这是一个技术术语，它意味着变量没有绑定到本地变量域。见图表7-1.

Inspecting the returned averager object shows how Python keeps the names of local and free variables in the __code__ attribute that represents the compiled body of the function:

检查返回averager对象可以发现Python在表示函数编译体的__code__属性中保留了本地和自由变量：

Example 7-11. Inspecting the function created by make_averager in Example 7-9.

例子7-11. 检查在例子7-9中创建的函数。

>>> avg.__code__.co_varnames ('new_value', 'total')
>>> avg.__code__.co_freevars ('series',)

The binding for series is kept in the __closure__ attribute of the returned function avg. Each item in avg.__closure__ corresponds to a name in avg.__code__.co_free vars. These items are cells, and they have an attribute called cell_contents where the actual value can be found. Example 7-12 shows these attributes.

series的绑定保存在了

Example 7-12. Continuing from Example 7-10

>>> avg.__code__.co_freevars
('series',)
>>> avg.__closure__
(<cell at 0x107a44f78: list object at 0x107a91a48>,) >>> avg.__closure__[0].cell_contents
[10, 11, 12]

To summarize: a closure is a function that retains the bindings of the free variables that exist when the function is defined, so that they can be used later when the function is invoked and the defining scope is no longer available.

总结：闭包一个函数保留已有自由变量的绑定

Note that the only situation in which a function may need to deal with external variables that are nonglobal is when it is nested in another function.

注意

The nonlocal Declaration nolocal声明

Our previous implementation of make_averager was not efficient. In Example 7-9, we stored all the values in the historical series and computed their sum every time averager was called. A better implementation would just store the total and the number of items so far, and compute the mean from these two numbers.

我们之前的make_averager实现没有效率的。在例子7-9中，我们

Example 7-13 is a broken implementation, just to make a point. Can you see where it breaks?
例子7-13是一个不完整的实现，

Example 7-13. A broken higher-order function to calculate a running average without keeping all history

def make_averager(): 
    count = 0
    total = 0
    def averager(new_value): 
        count += 1
        total += new_value 
        return total / count
    return averager

If you try Example 7-13, here is what you get:

如果你尝试运行例子7-13，你会获得如下结果：

>>> avg = make_averager()
>>> avg(10)
Traceback (most recent call last):
...
    UnboundLocalError: local variable 'count' referenced before assignment
>>>

The problem is that the statement count += 1 actually means the same as count = count + 1, when count is a number or any immutable type. So we are actually assigning to count in the body of averager, and that makes it a local variable. The same problem affects the total variable.

问题在于语句count += 1的实际意义和count = count + 1相同，

We did not have this problem in Example 7-9 because we never assigned to the ser ies name; we only called series.append and invoked sum and len on it. So we took advantage of the fact that lists are mutable.

在例子7-9中我们并未遇到这个问题，因为

But with immutable types like numbers, strings, tuples, etc., all you can do is read, but never update. If you try to rebind them, as in count = count + 1, then you are implicitly creating a local variable count. It is no longer a free variable, and therefore it is not saved in the closure.

To work around this, the nonlocal declaration was introduced in Python 3. It lets you flag a variable as a free variable even when it is assigned a new value within the function. If a new value is assigned to a nonlocal variable, the binding stored in the closure is changed. A correct implementation of our newest make_averager looks like Example 7-14.

Example 7-14. Calculate a running average without keeping all history (fixed with the use of nonlocal)

例子7-14。

def make_averager(): 
    count = 0
    total = 0

    def averager(new_value): 
        nonlocal count, total 
        count += 1
        total += new_value 
        return total / count
    return averager

Getting by without nonlocal in Python 2

The lack of nonlocal in Python 2 requires workarounds, one of which is described in the third code snippet of PEP 3104 — Ac‐ cess to Names in Outer Scopes, which introduced nonlocal. Es‐ sentially the idea is to store the variables the inner functions need to change (e.g., count, total) as items or attributes of some mu‐ table object, like a dict or a simple instance, and bind that object to a free variable.

Now that we have Python closures covered, we can effectively implement decorators with nested functions.

Implementing a Simple Decorator 实现一个简单装饰器

Example 7-15 is a decorator that clocks every invocation of the decorated function and prints the elapsed time, the arguments passed, and the result of the call.

例子7-15

Example 7-15. A simple decorator to output the running time of functions

例子 7－15。一个输出函数运行时间的简单装饰器。

import time


def clock(func):
    def clocked(*args): #1 
        t0 = time.perf_counter()
        result = func(*args) #2
        elapsed = time.perf_counter() - t0
        name = func.__name__
        arg_str = ', '.join(repr(arg) for arg in args)
        print('[%0.8fs] %s(%s) -> %r' % (elapsed, name, arg_str, result)) return result
    return clocked #3

1. Define inner function clocked to accept any number of positional arguments.

2. This line only works because the closure for clocked encompasses the func free variable.

3. Return the inner function to replace the decorated function.

4. 定义能够接受人意数量位置参数的函数clocked。

5. 该行仅在clocked的闭包包含在func自由变量时起作用

6. 返回内部函数来替换被装饰的函数

Example 7-16 demonstrates the use of the clock decorator.
例子 7-16 clock装饰器的使用说明。

Example 7-16. Using the clock decorator
例子 716. 使用clock装饰器

# clockdeco_demo.py
import time
from clockdeco import clock


@clock
def snooze(seconds):
    time.sleep(seconds)


@clock
def factorial(n):
    return 1 if n < 2 else n*factorial(n-1)


if __name__=='__main__':
    print('*' * 40, 'Calling snooze(.123)') snooze(.123)
    print('*' * 40, 'Calling factorial(6)') 
    print('6! =', factorial(6))

The output of running Example 7-16 looks like this:

例子7-16的输出内容如下：

$ python3 clockdeco_demo.py
**************************************** Calling snooze(123)
[0.12405610s] snooze(.123) -> None
**************************************** Calling factorial(6)
[0.00000191s] factorial(1) -> 1
[0.00004911s] factorial(2) -> 2
[0.00008488s] factorial(3) -> 6
[0.00013208s] factorial(4) -> 24
[0.00019193s] factorial(5) -> 120
[0.00026107s] factorial(6) -> 720 6!=720

How It Works 工作原理

Remember that this code:

请记住这段代码：

@clock
def factorial(n):
    return 1 if n < 2 else n*factorial(n-1)

Actually does this:

实际上执行的代码如下：

def factorial(n):
    return 1 if n < 2 else n*factorial(n-1)


factorial = clock(factorial)

So, in both examples, clock gets the factorial function as its func argument (see Example 7-15). It then creates and returns the clocked function, which the Python interpreter assigns to factorial behind the scenes. In fact, if you import the clockde co_demo module and check the __name__ of factorial, this is what you get:

所以，这这两个例子中，clock都是把factorial函数作为自己的func参数（参见例子7-15）。然后它创建和返回Python解释器在后台赋值给factorial的函数clocked。实际上，如果你导入clockdeco_demo模块，并查看factorial的__name__，你会得到如下内容：

>>> import clockdeco_demo
>>> clockdeco_demo.factorial.__name__ 
'clocked'
>>>

So factorial now actually holds a reference to the clocked function. From now on, each time factorial(n) is called, clocked(n) gets executed. In essence, clocked does the following:

所以现在实际上factorial拥有了到clocked函数的引用。从现在起，每次factorial(n)被调用时，clocked(n)都会被执行。

1. Records the initial time t0.

2. Calls the original factorial, saving the result.

3. Computes the elapsed time.

4. Formats and prints the collected data.

5. Returns the result saved in step 2.

This is the typical behavior of a decorator: it replaces the decorated function with a new function that accepts the same arguments and (usually) returns whatever the decorated function was supposed to return, while also doing some extra processing.

这是装饰器的典型行为：使用一个新的接受了相同参数的

In Design Patterns by Gamma et al., the short description of the Decorator pattern starts with: “Attach additional responsibilities to an object dynamically.” Function decorators fit that description. But at the implementation level, Python decorators bear little re‐ semblance to the classic Decorator described in the original De‐ sign Patterns work. “Soapbox” on page 213 has more on this subject.

The clock decorator implemented in Example 7-15 has a few shortcomings: it does not support keyword arguments, and it masks the __name__ and __doc__ of the decorated function. Example 7-17 uses the functools.wraps decorator to copy the relevant at‐ tributes from func to clocked. Also, in this new version, keyword arguments are cor‐ rectly handled.

在例子7-15中实现的clock装饰器包含几个缺点：它不支持关键字参数，而且它还掩盖了被装饰函数的__name__和__doc__。例子7-17使用装饰器functools.wraps从func复制了相关属性到clocked。而且，在这个新版本代码中，关键字参数也被正确地处理了。

Example 7-17. An improved clock decorator
例子7-17. 改进过的clock装饰器

# clockdeco2.py
import time
import functools


def clock(func): 
    @functools.wraps(func)
    def clocked(*args, **kwargs):
        t0 = time.time()
        result = func(*args, **kwargs) elapsed = time.time() - t0 name = func.__name__
        arg_lst = []
        if args:
            arg_lst.append(', '.join(repr(arg) for arg in args))
        if kwargs:
            pairs = ['%s=%r' % (k, w) for k, w in sorted(kwargs.items())]
        arg_lst.append(', '.join(pairs))
        arg_str = ', '.join(arg_lst)
        print('[%0.8fs] %s(%s) -> %r ' % (elapsed, name, arg_str, result))
        return result
    return clocked

functools.wraps is just one of the ready-to-use decorators in the standard library. In the next section, we’ll meet two of the most impressive decorators that functools provides: lru_cache and singledispatch.

functools.wraps只是标准库中的开箱即用的装饰器其中之一。在下一节，我们会见到最令人映像深刻的由functools提供的两个装饰器：lru_cache 和 singledispatch。

Decorators in the Standard Library 标准库中的装饰器

Python has three built-in functions that are designed to decorate methods: property, classmethod, and staticmethod. We will discuss property in “Using a Property for Attribute Validation” on page 604 and the others in “classmethod Versus staticmethod” on page 252.

Python拥有三个被设计用来装饰方法的内建函数：property, classmethod, 和 staticmethod。我们会在604页中的“为属性验证使用Property”讨论propert，在252页上面的“classmethod与staticmethod”讨论其他内容。

Another frequently seen decorator is functools.wraps, a helper for building well- behaved decorators. We used it in Example 7-17. Two of the most interesting decorators in the standard library are lru_cache and the brand-new singledispatch (added in Python 3.4). Both are defined in the functools module. We’ll cover them next.

另外经常见到的装饰器为functools.wraps，

Memoization with functools.lru_cache

A very practical decorator is functools.lru_cache. It implements memoization: an optimization technique that works by saving the results of previous invocations of an expensive function, avoiding repeat computations on previously used arguments. The letters LRU stand for Least Recently Used, meaning that the growth of the cache is limited by discarding the entries that have not been read for a while.

一个非常实用的装饰器就是functools.lru_cache。它实现了记忆：

A good demonstration is to apply lru_cache to the painfully slow recursive function to generate the nth number in the Fibonacci sequence, as shown in Example 7-18.

比较好的一个说明是运用lru_cache及其慢的速度去递归函数以生成斐波那契序列中的第n个数字，一如例子7-18所示。

Example 7-18. The very costly recursive way to compute the nth number in the Fibonacci series

from clockdeco import clock


@clock
def fibonacci(n): 
    if n<2:
        return n
    return fibonacci(n-2) + fibonacci(n-1)


if __name__=='__main__': print(fibonacci(6))

Here is the result of running fibo_demo.py. Except for the last line, all output is generated by the clock decorator:

这里是运行fibo_demo.py的输出结果。除了最后一行之外，所有的输出都是由clock装饰器生成的：

    $ python3 fibo_demo.py
    [0.00000095s] fibonacci(0) -> 0
    [0.00000095s] fibonacci(1) -> 1
    [0.00007892s] fibonacci(2) -> 1
    [0.00000095s] fibonacci(1) -> 1
    [0.00000095s] fibonacci(0) -> 0
    [0.00000095s] fibonacci(1) -> 1
    [0.00003815s] fibonacci(2) -> 1
    [0.00007391s] fibonacci(3) -> 2
    [0.00018883s] fibonacci(4) -> 3
    [0.00000000s] fibonacci(1) -> 1
    [0.00000095s] fibonacci(0) -> 0
    [0.00000119s] fibonacci(1) -> 1
    [0.00004911s] fibonacci(2) -> 1
    [0.00009704s] fibonacci(3) -> 2
    [0.00000000s] fibonacci(0) -> 0
    [0.00000000s] fibonacci(1) -> 1
    [0.00002694s] fibonacci(2) -> 1
    [0.00000095s] fibonacci(1) -> 1
    [0.00000095s] fibonacci(0) -> 0
    [0.00000095s] fibonacci(1) -> 1
    [0.00005102s] fibonacci(2) -> 1
    [0.00008917s] fibonacci(3) -> 2
    [0.00015593s] fibonacci(4) -> 3
    [0.00029993s] fibonacci(5) -> 5
    [0.00052810s] fibonacci(6) -> 8
    8

The waste is obvious: fibonacci(1) is called eight times, fibonacci(2) five times, etc. But if we just add two lines to use lru_cache, performance is much improved. See Example 7-19.

浪费是显而易见的：fibonacci(1)被调用了8次，fibonacci(2)被调用5次，等等。但是如果我们使用lru_cache增加两行代码，性能就会得到很大的提高。参见例子7-19.

Example 7-19. Faster implementation using caching
例子7－19.

import functools
from clockdeco import clock


@functools.lru_cache() # 1
@clock # 2
def fibonacci(n):
    if n<2:
        return n
    return fibonacci(n-2) + fibonacci(n-1)


if __name__=='__main__':
    print(fibonacci(6))

1. Note that lru_cache must be invoked as a regular function—note the parentheses in the line: @functools.lru_cache(). The reason is that it accepts configuration parameters, as we’ll see shortly.

2. This is an example of stacked decorators: @lru_cache() is applied on the function returned by @clock.

3. 注意lru_cache必须和像普通函数一样被调用——注意圆括号：@functools.lru_cache()。原因是它接受配置参数，我们很快就会看到的。

4. 这是一个堆叠式装饰器的例子：@lru_cache()应用在由@clock返回的函数之上。

Execution time is halved, and the function is called only once for each value of n:

$ python3 fibo_demo_lru.py
  [0.00000119s] fibonacci(0) -> 0
  [0.00000119s] fibonacci(1) -> 1
  [0.00010800s] fibonacci(2) -> 1
  [0.00000787s] fibonacci(3) -> 2
  [0.00016093s] fibonacci(4) -> 3
  [0.00001216s] fibonacci(5) -> 5
  [0.00025296s] fibonacci(6) -> 8

In another test, to compute fibonacci(30), Example 7-19 made the 31 calls needed in 0.0005s, while the uncached Example 7-18 called fibonacci 2,692,537 times and took 17.7 seconds in an Intel Core i7 notebook.

Besides making silly recursive algorithms viable, lru_cache really shines in applications that need to fetch information from the Web.

It’s important to note that lru_cache can be tuned by passing two optional arguments. Its full signature is:

functools.lru_cache(maxsize=128, typed=False)

The maxsize argument determines how many call results are stored. After the cache is full, older results are discarded to make room. For optimal performance, maxsize should be a power of 2. The typed argument, if set to True, stores results of different argument types separately, i.e., distinguishing between float and integer arguments that are nor‐ mally considered equal, like 1 and 1.0. By the way, because lru_cache uses a dict to store the results, and the keys are made from the positional and keyword arguments used in the calls, all the arguments taken by the decorated function must be hashable.

maxsize参数决定了要存储多少次调用结果。在缓存满了之后，先前旧的结果被丢弃以腾出空间来。

Now let’s consider the intriguing functools.singledispatch decorator.

Generic Functions with Single Dispatch

Imagine we are creating a tool to debug web applications. We want to be able to generate HTML displays for different types of Python objects.

想象一下，我们正在创建一个调试web应用的工具。我们想要生成显示不同Python类型的HTML。

We could start with a function like this:

我们可以从编写这样一个函数：

import html


def htmlize(obj):
    content = html.escape(repr(obj)) return '<pre>{}</pre>'.format(content)

That will work for any Python type, but now we want to extend it to generate custom displays for some types:

该函数可以适用任何Python类型，但是现在我们想要扩展它

• str: replace embedded newline characters with '
  \n' and use

  tags instead of

  .
  
  

• int: show the number in decimal and hexadecimal.

• list: output an HTML list, formatting each item according to its type.

• str：使用'
  \n'和

  标签而不是

  标签来替换内嵌的换行字符。
  
  

• int：显示十进制和十六进制的数字

• list：输出一个HTML列表，

The behavior we want is shown in Example 7-20.

我们想要的行为出现在例子7-20.

Example 7-20. htmlize generates HTML tailored to different object types

例子7-20. htmlize将订制的HTML生成不同对象类型。

>>> htmlize({1, 2, 3})   #1
'<pre>{1, 2, 3}</pre>'
>>> htmlize(abs)
'<pre>&lt;built-in function abs&gt;</pre>'  #2
>>> htmlize('Heimlich & Co.\n- a game') 
'<p>Heimlich &amp; Co.<br>\n- a game</p>' 
>>> htmlize(42) #3
'<pre>42 (0x2a)</pre>'
>>> print(htmlize(['alpha', 66, {3, 2, 1}])) #4
<ul>
<li><p>alpha</p></li>
<li><pre>66 (0x42)</pre></li>
<li><pre>{1, 2, 3}</pre></li>
</ul>

1. By default, the HTML-escaped repr of an object is shown enclosed in

   </ pre>.

2. str objects are also HTML-escaped but wrapped in
   with
   line breaks.
3. An int is shown in decimal and hexadecimal, inside
   .
4. Each list item is formatted according to its type, and the whole sequence rendered as an HTML list.

Because we don’t have method or function overloading in Python, we can’t create var‐ iations of htmlize with different signatures for each data type we want to handle dif‐ ferently. A common solution in Python would be to turn htmlize into a dispatch func‐ tion, with a chain of if/elif/elif calling specialized functions like htmlize_str, htmlize_int, etc. This is not extensible by users of our module, and is unwieldy: over time, the htmlize dispatcher would become too big, and the coupling between it and the specialized functions would be very tight.

The new functools.singledispatch decorator in Python 3.4 allows each module to contribute to the overall solution, and lets you easily provide a specialized function even for classes that you can’t edit. If you decorate a plain function with @singledispatch, it becomes a generic function: a group of functions to perform the same operation in different ways, depending on the type of the first argument.[3] Example 7-21 shows how.

functools.singledispatch was added in Python 3.4, but the sin gledispatch package available on PyPI is a backport compatible with Python 2.6 to 3.3.

[3]. This is what is meant by the term single-dispatch. If more arguments were used to select the specific functions, we’d have multiple-dispatch.

Example 7-21. singledispatch creates a custom htmlize.register to bundle several func‐ tions into a generic function