07-closure-deco.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# 函数装饰器与闭包\n",
    "\n",
    "> 有很多人抱怨，把这个特性命名为“装饰器”不好。主要原因是，这个名称与 GoF 书使用的不一致。**装饰器**这个名称可能更适合在编译器领域使用，因为它会遍历并注解语法书。\n",
    "> —“PEP 318 — Decorators for Functions and Methods”\n",
    "\n",
    "本章的最终目标是解释清楚函数装饰器的工作原理，包括最简单的注册装饰器和较复杂的参数化装饰器。  \n",
    "\n",
    "讨论内容：\n",
    "* Python 如何计算装饰器语法\n",
    "* Python 如何判断变量是不是局部的\n",
    "* 闭包存在的原因和工作原理\n",
    "* `nonlocal` 能解决什么问题\n",
    "* 实现行为良好的装饰器\n",
    "* 标准库中有用的装饰器\n",
    "* 实现一个参数化的装饰器"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 装饰器基础\n",
    "\n",
    "装饰器是可调用的对象，其参数是一个函数（被装饰的函数）。\n",
    "\n",
    "装饰器可能会处理被装饰的函数，然后把它返回，或者将其替换成另一个函数或可调用对象。\n",
    "\n",
    "装饰器两大特性：\n",
    "\n",
    "1. 能把被装饰的函数替换成其他函数\n",
    "2. 装饰器在加载模块时立即执行"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# 装饰器通常会把函数替换成另一个函数\n",
    "def decorate(func):\n",
    "    def wrapped():\n",
    "        print('Running wrapped()')\n",
    "    return wrapped\n",
    "\n",
    "@decorate\n",
    "def target():\n",
    "    print('running target()')\n",
    "target()\n",
    "\n",
    "# 以上写法等同于\n",
    "def target():\n",
    "    print('running target()')\n",
    "target = decorate(target)\n",
    "target()\n",
    "\n",
    "# 这里真正调用的是装饰器返回的函数\n",
    "\n",
    "def deco(func):\n",
    "    def inner():\n",
    "        print('running iner()')\n",
    "    return inner\n",
    "\n",
    "@deco\n",
    "def target():\n",
    "    print('running target()')\n",
    "target()\n",
    "# target 现在是 inner 的引用\n",
    "target"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## *Python* 何时执行装饰器\n",
    "\n",
    "装饰器在导入时（模块加载时）立即执行"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {
    "tags": []
   },
   "outputs": [
    {
     "output_type": "stream",
     "name": "stdout",
     "text": "running register <function f1 at 0x111b6fee0>\nrunning register <function f2 at 0x111bdb160>\nrunning main()\nregistry -> [<function f1 at 0x111b6fee0>, <function f2 at 0x111bdb160>]\nrunning f1()\nrunning f2()\nrunning f3()\n"
    }
   ],
   "source": [
    "# registration.py\n",
    "\n",
    "registry = []\n",
    "def register(func):\n",
    "    print('running register {}'.format(func))\n",
    "    registry.append(func)\n",
    "    return func\n",
    "\n",
    "@register\n",
    "def f1():\n",
    "    print('running f1()')\n",
    "\n",
    "@register\n",
    "def f2():\n",
    "    print('running f2()')\n",
    "\n",
    "def f3():\n",
    "    print('running f3()')\n",
    "    \n",
    "def main():\n",
    "    print('running main()')\n",
    "    print('registry ->', registry)\n",
    "    f1()\n",
    "    f2()\n",
    "    f3()\n",
    "\n",
    "if __name__=='__main__':\n",
    "    main()\n",
    "    \n",
    "# python3 registration.py\n",
    "# output:\n",
    "# running register <function f1 at 0x10b4194d0>\n",
    "# running register <function f2 at 0x10b419ef0>\n",
    "# running main()\n",
    "# registry -> [<function f1 at 0x10b4194d0>, <function f2 at 0x10b419ef0>]\n",
    "# running f1()\n",
    "# running f2()\n",
    "# running f3()\n",
    "\n",
    "# import registration\n",
    "# running register <function f1 at 0x10d89e950>\n",
    "# running register <function f2 at 0x10d89e050>\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "通过上面的例子，强调装饰器函数在导入模块式立即执行，而普通函数在被调用时运行。导入时和运行时的区别。\n",
    "\n",
    "- 装饰器函数通常与被装饰函数不在同一模块。\n",
    "- register 装饰器返回的函数没有变化\n",
    "\n",
    "上面的装饰器会原封不动地返回被装饰的函数，而不一定会对函数做修改。 \n",
    "这种装饰器叫注册装饰器，通过使用它来中心化地注册函数，例如把 URL 模式映射到生成 HTTP 响应的函数上的注册处。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 使用装饰器\n",
    "\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "promos = []\n",
    "\n",
    "def promotion(promo_func):\n",
    "    promos.append(promo_func)\n",
    "    return promo_func\n",
    "\n",
    "@promotion\n",
    "def fidelity(order):\n",
    "    \"\"\"积分 >= 1000 提供 5% 折扣\"\"\"\n",
    "    return order.total() * .05 if order.customer.fidelity >= 1000 else 0"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 变量作用域规则"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# 比较两个例子\n",
    "\n",
    "b = 6\n",
    "def f1(a):\n",
    "    print(a)\n",
    "    print(b)\n",
    "f1(3)\n",
    "\n",
    "\n",
    "def f2(a):\n",
    "    print(a)\n",
    "    print(b)\n",
    "    b = 9 # b 此时为局部变量\n",
    "f2(3)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "*Python* 假定在函数体内部的变量为局部变量。如果未在局部变量中找到，会逐级向上查找变量。\n",
    "\n",
    "如果想在函数中赋值时让解释器把 b 当做全局变量，用 global 关键字"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "def f3(a):\n",
    "    global b\n",
    "    print(a)\n",
    "    print(b)\n",
    "    b = 9 \n",
    "f3(3)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 闭包\n",
    "\n",
    "闭包和匿名函数常被弄混。只有涉及到嵌套函数时才有闭包问题。\n",
    "\n",
    "闭包指延伸了作用域的函数，其中包含函数定义体中的引用，但非定义体中定义的非全局变量。和函数是否匿名无关。关键是能访问定义体之外定义的非全局变量。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "class Averager():\n",
    "    def __init__(self):\n",
    "        self.series = []\n",
    "        \n",
    "    def __call__(self, new_value):\n",
    "        self.series.append(new_value)\n",
    "        total = sum(self.series)\n",
    "        return total/len(self.series)\n",
    "    \n",
    "avg = Averager()\n",
    "avg(10)\n",
    "avg(11)\n",
    "avg(12)\n",
    "\n",
    "def make_averager():\n",
    "    series = []  # 自由变量\n",
    "    \n",
    "    def averager(new_value):\n",
    "        series.append(new_value)\n",
    "        total = sum(series)\n",
    "        return total/len(series)\n",
    "\n",
    "    return averager\n",
    "\n",
    "avg = make_averager()\n",
    "avg(10)\n",
    "avg(11)\n",
    "avg(12)\n",
    "\n",
    "avg.__code__.co_varnames\n",
    "avg.__code__.co_freevars\n",
    "avg.__closure__\n",
    "avg.__closure__[0].cell_contents"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "在 averager 函数中，series 是自由变量，指未在本地作用域绑定的变量。\n",
    "\n",
    "通过 `__code__.co_freevars` `__closure__` 查看自由变量和闭包\n",
    "\n",
    "闭包是一种函数，保留定义函数时存在的自由变量的绑定。调用函数时，虽然定义作用域不可用了，但仍能使用那些绑定\n",
    "\n",
    "> 只有嵌套在其他函数中的函数才可能需要处理不在全局作用域的外部变量"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## nonlocal 声明\n",
    "\n",
    "下面一个例子有缺陷："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "def make_averager():\n",
    "    count = 0\n",
    "    total = 0\n",
    "    \n",
    "    def averager(new_value):\n",
    "        count += 1\n",
    "        total += new_value\n",
    "        return total / count\n",
    "\n",
    "    return averager\n",
    "\n",
    "avg = make_averager()\n",
    "avg(10)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "注意 count， total 的赋值语句使它们成为局部变量，在赋值是会隐式创建局部变量，这样它们就不是自由变量了，因此不会保存在闭包中。\n",
    "\n",
    "为解决这个问题，*Python3* 引入了 nonlocal 声明，作用是吧变量标记为自由变量，即使在函数中为变量新值了，也会变成自由变量。在闭包中的绑定也会更新\n",
    "\n",
    "> 对于没有 nonlocal 的 *Python2* PEP3104"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "def make_averager():\n",
    "    count = 0\n",
    "    total = 0\n",
    "    \n",
    "    def averager(new_value):\n",
    "        nonlocal count, total\n",
    "        count += 1\n",
    "        total += new_value\n",
    "        return total / count\n",
    "\n",
    "    return averager\n",
    "\n",
    "avg = make_averager()\n",
    "avg(10)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 实现一个简单的装饰器"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "import time\n",
    "\n",
    "def clock(func):\n",
    "    def clocked(*args):\n",
    "        t0 = time.perf_counter()\n",
    "        result = func(*args)\n",
    "        elapsed = time.perf_counter() - t0\n",
    "        name = func.__name__\n",
    "        arg_str = ', '.join(repr(arg) for arg in args)\n",
    "        print('[%0.8fs] %s(%s) -> % r' %(elapsed, name, arg_str, result))\n",
    "        return result\n",
    "    return clocked\n",
    "\n",
    "@clock \n",
    "def snooze(seconds):\n",
    "    time.sleep(seconds)\n",
    "    \n",
    "@clock\n",
    "def factorial(n):\n",
    "    return 1 if n < 2 else n * factorial(n-1)\n",
    "\n",
    "if __name__=='__main__':\n",
    "    print('*' * 40, 'Calling snooze(.123)')\n",
    "    snooze(.123)\n",
    "    print('*' * 40, 'Calling factorial(6)')\n",
    "    print('6! =', factorial(6))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "装饰器的典型行为：把被装饰的函数替换成新函数，二者接受相同的参数，而且(通常)返回被装装饰函数本该返回的值，同时做一些额外操作"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "factorial.__name__"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "上述实现的 clock 装饰器有几个缺点：不支持关键字参数，而且遮盖了被装饰函数的 `__name__`, `__doc__` 属性\n",
    "\n",
    "functools.wraps 装饰器把相关属性从 func 复制到 clocked 中，还能正确处理关键字函数"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "import time\n",
    "import functools\n",
    "\n",
    "def clock(func):\n",
    "    @functools.wraps(func)\n",
    "    def clocked(*args, **kwargs):\n",
    "        t0 = time.perf_counter()\n",
    "        result = func(*args, **kwargs)\n",
    "        elapsed = time.perf_counter() - t0\n",
    "        name = func.__name__\n",
    "        arg_lst = []\n",
    "        if args:\n",
    "            arg_str = ', '.join(repr(arg) for arg in args)\n",
    "        if kwargs:\n",
    "            pairs = ['%s=%s' % (k, w) for k, w in sorted(kwargs.items())]\n",
    "            arg_lst.append(', '.join(pairs))\n",
    "        arg_str = ', '.join(arg_lst)\n",
    "        print('[%0.8fs] %s(%s) -> % r' %(elapsed, name, arg_str, result))\n",
    "        return result\n",
    "    return clocked\n",
    "\n",
    "if __name__=='__main__':\n",
    "    print('*' * 40, 'Calling snooze(.123)')\n",
    "    snooze(.123)\n",
    "    print('*' * 40, 'Calling factorial(6)')\n",
    "    print('6! =', factorial(6))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 标准库中的装饰器\n",
    "\n",
    "Python 内置的三个装饰器分别为 `property`, `classmethod` 和 `staticmethod`.  \n",
    "\n",
    "但 Python 内置的库中，有两个装饰器很常用，分别为 `functools.lru_cache` 和 [`functools.singledispatch`](https://docs.python.org/3/library/functools.html#functools.singledispatch)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "@clock\n",
    "def fibonacci(n):\n",
    "    if n < 2:\n",
    "        return n\n",
    "    return fibonacci(n-2) + fibonacci(n-1)\n",
    "\n",
    "print(fibonacci(6))\n",
    "\n",
    "@functools.lru_cache() # () 是因为 lru_cache 可以接受配置参数\n",
    "# functools.lru_cache(maxsize=128, typed=False)\n",
    "@clock # 叠放装饰器\n",
    "def fibonacci(n):\n",
    "    if n < 2:\n",
    "        return n\n",
    "    return fibonacci(n-2) + fibonacci(n-1)\n",
    "\n",
    "print(fibonacci(6))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 单分派反函数\n",
    "\n",
    "*Python* 不支持重载方法或函数，所以我们不能使用不同的签名定义 htmlize 的辩题，也无法使用不同的方式处理不同的数据类型。\n",
    "\n",
    "一种常见的方法是把 htmlize 编程一个分派函数，使用 if-elif-else 分别调用专门的函数。但这样不便于模块的拓展，而且臃肿\n",
    "\n",
    "functoos.singledispatch 装饰器可以把整体方案拆分成多个模块，甚至可以为你无法修改的类提供专门的函数。\n",
    "使用 functoos.singledispatch 装饰的普通函数会变成反函数。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# 生成 HTML 显示不同类型的 python 对象\n",
    "import html\n",
    "\n",
    "def htmlize(obj):\n",
    "    content = html.escape(repr(obj))\n",
    "    return '<pre>{}</pre>'.format(content)\n",
    "\n",
    "# htmlize({1, 2, 3})\n",
    "# htmlize(abs)\n",
    "# htmlize('hwimich & Co.\\n- a game')\n",
    "# htmlize(42)\n",
    "# print(htmlize(['alpha', 66, {3, 2, 1}]))\n",
    "\n",
    "from functools import singledispatch\n",
    "from collections import abc \n",
    "import numbers\n",
    "\n",
    "@singledispatch # 标记处理 object 类型的基函数\n",
    "def htmlize(obj):\n",
    "    content = html.escape(repr(obj))\n",
    "    return '<pre>{}</pre>'.format(content)\n",
    "\n",
    "@htmlize.register(str)\n",
    "def _(text):\n",
    "    content = html.escape(text).replace('\\n', '<br>\\n')\n",
    "    return '<p>{0}</p>'.format(content)\n",
    "\n",
    "@htmlize.register(numbers.Integral) # Integral 是 int 的虚拟超类\n",
    "def _(n):\n",
    "    return '<pre>{0} (0x{0:x})</pre>'.format(n)\n",
    "\n",
    "@htmlize.register(tuple)\n",
    "@htmlize.register(abc.MutableSequence)\n",
    "def _(seq):\n",
    "    inner = '</li>\\n<li>'.join(htmlize(item) for item in seq)\n",
    "    return '<ul>\\n<li>' + inner + '</li>\\n<ul>'\n",
    "\n",
    "htmlize({1, 2, 3})\n",
    "htmlize(abs)\n",
    "htmlize('hwimich & Co.\\n- a game')\n",
    "htmlize(42)\n",
    "print(htmlize(['alpha', 66, {3, 2, 1}]))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "只要可能，注册的专门函数应该处理抽象基类(numbers.Integral, abc.MutableSequence)， 不要处理具体实现（int，list）\n",
    "\n",
    "这样代码支持的兼容类型更广泛。\n",
    "\n",
    "使用 singledispatch 可以在系统的任何地方和任何模块注册专门函数。\n",
    "\n",
    "\n",
    "\n",
    "## 叠放装饰器\n",
    "\n",
    "```python\n",
    "@d1\n",
    "@d2\n",
    "def func():\n",
    "    pass\n",
    "\n",
    "# 等同于\n",
    "func = d1(d2(func))\n",
    "```\n",
    "\n",
    "## 参数化装饰器\n",
    "\n",
    "为了方便理解，可以把参数化装饰器看成一个函数：这个函数接受任意参数，返回一个装饰器（参数为 func 的另一个函数）。\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# 参数化的注册装饰器\n",
    "registry = set()\n",
    "\n",
    "# 这是一个工厂函数，用来构建装饰器函数\n",
    "def register(active=True):\n",
    "    # decorate 是真正的装饰器\n",
    "    def decorate(func):\n",
    "        print('running register(active=%s)->decorate(%s)' % (active, func))\n",
    "        if active:\n",
    "            registry.add(func)\n",
    "        else:\n",
    "            registry.discard(func) \n",
    "        return func\n",
    "    return decorate\n",
    "\n",
    "@register(active=False)\n",
    "def f1():\n",
    "    print('running f1()')\n",
    "    \n",
    "@register()\n",
    "def f2():\n",
    "    print('running f2()')\n",
    "    \n",
    "def f3():\n",
    "    print('running f3()')\n",
    "    \n",
    "f1()\n",
    "f2()\n",
    "f3()\n",
    "register()(f3)\n",
    "registry\n",
    "register(active=False)(f2)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 参数化 clock 装饰器\n",
    "\n",
    "为 clock 装饰器添加一个功能，让用户传入一个格式化字符串，控制被装饰函数的输出。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "import time\n",
    "\n",
    "DEFAULT_FMT = '[{elapsed:0.8f}s] {name}({args}) -> {result}'\n",
    "\n",
    "def clock(fmt=DEFAULT_FMT):\n",
    "    def decorate(func):\n",
    "        def clocked(*_args):\n",
    "            t0 = time.time()\n",
    "            _result = func(*_args)\n",
    "            elapsed = time.time() - t0\n",
    "            name = func.__name__\n",
    "            args = ', '.join(repr(arg) for arg in _args)\n",
    "            result = repr(_result)\n",
    "            print(fmt.format(**locals()))\n",
    "            return _result\n",
    "        return clocked\n",
    "    return decorate\n",
    "\n",
    "# @clock()\n",
    "# @clock('{name}: {elapsed}s')\n",
    "@clock('{name}{args} dt={elapsed:0.3f}s')\n",
    "def snooze(seconds):\n",
    "    time.sleep(seconds)\n",
    "\n",
    "for i in range(3):\n",
    "    snooze(.123)"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.7.4"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}