Skip to content

Commit

Permalink
update docs (#7)
Browse files Browse the repository at this point in the history
  • Loading branch information
@reiase
reiase authored May 26, 2023
1 parent 58bc198 commit 6a7e1ac
Show file tree
Hide file tree
Showing 2 changed files with 151 additions and 136 deletions.
139 changes: 73 additions & 66 deletions README.md
View file
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
**H**_yper_**P**_arameter_
Hyperparameter
===========================

<h3 align="center">
Expand All @@ -7,88 +7,99 @@
</p>
</h3>

HyperParameter is a pythonic configuration framework designed to simplify the massive configuration in complex applications. The key feature is a dynamic hierarchical parameter tree composited with scopes. HyperParameter is particularly well suited for machine learning experiments and related systems, which have many parameters and nested codes.
<p align="center">

Key Conceptions
---------------

1. `parameter tree`, a nested python dict with support for default values and object-style API;
1. `param_scope`, a context manager for compositing the ` parameter tree` with nested `param_scope`;
2. `auto_param`, a decorator allowing default parameters of a function or class to be supplied from `param_scope`;
**Hyperparameter, Make configurable AI applications.Build for Python hackers.**

</p>

Quick Start
-----------

A quick example for defining a model with HyperParameter:
`Hyperparameter` uses `auto _ param` decorator to convert keywords arguments into configurable parameters:

```python
@auto_param
def dnn(input, layers=3, activation="relu"):
"""
build a DNN model with the following configurations:
- dnn.layers(default: 3)
- dnn.activation(default: "relu")
"""
for i in range(layers):
input = Linear(input)
input = activation_fn(
activation,
input
)
return input

# call dnn with default configuration
# and create a 3 layer dnn with relu activation
dnn(x)

# passing parameter using param_scope
with param_scope(**{
"dnn.layers": 4,
"dnn.activation": "sigmoid"}):
# create a 4 layer dnn with sigmoid activation
dnn()
from hyperparameter import auto_param

@auto_param("foo")
def foo(x, y=1, z="a"):
return f"x={x}, y={y}, z={z}"
```

Another example for building ML system:
The parameters can be controlled with `param_scope`

```python
@auto_param
def inference(x, backend="tvm"):
...
from hyperparameter import param_scope

with param_scope(backend="onnx"):
inference(x)
foo(1) # x=1, y=1, z='a'
with param_scope(**{"foo.y":2}):
foo(1) # x=1, y=2, z='a'
```

Advanced Usage
--------------
### Nested Scope and Configuration Composition

HyperParameter uses nested `param_scope` for configuration composition :
### Read/Write Parameters

``` python
```python
from hyperparameter import param_scope
# on initialization, the parameter tree is empty: {}
with param_scope(a=1) as ps:
# in the with context, the composited parameter tree is {"a": 1}
ps == {"a": 1}
with param_scope(a=2, b=3) as ps2:
# in the nested scope, the composited parameter tree is {"a": 2, "b": 3}
# param `b` is a new, and param `a` is overwrite by new value
ps2 == {"a": 2, "b": 3}
# when exit the inner scope, the modification of inner scope is cleaned up
# the composited parameter tree is {"a": 1}
ps == {"a": 1}

# create param_scope
with param_scope():
pass

with param_scope("foo.y=1", "foo.z=b"):
pass

with param_scope(**{"foo.y":1, "foo.z":2}):
pass

# read param with default value
with param_scope(**{"foo.y":2}) as ps:
y = ps.foo.y(1)
y = ps.foo.y | 1
y = param_scope.foo.y(1)
y = param_scope.foo.y | 1
foo(1) # x=1, y=2, z='a'

# wite values to param_scope
with param_scope(**{"foo.y":2}) as ps:
ps.foo.y = 2
param_scope.foo.y = 2
```

### Manage Parameters from CMD Line
### Nested Scope

It is recommended to use three-layer configuration for complex programmes:
`Hyperparameter` support nested `param_scope`:

1. `inline default values`;
2. `config file`, which will override `inline default values`;
3. `cmdline arguments` that override both `config file` and `inline default values`;
``` python
from hyperparameter import param_scope

# no param_scope, use the default value defined in foo
foo(1) # x=1, y=1, z='a'

# start a new param_scope
# and set the default value of `foo.y` to `2`
with param_scope(**{"foo.y":2}) as ps:
# found one param_scope `ps`,
# and receive default value of `foo.y` from `ps`
foo(1) # x=1, y=2, z='a'

# start another param_scope
# and set the default value of `foo.y` to `3`
with param_scope(**{"foo.z": "b"}) as ps2:
# found nested param_scope `ps2`,
# and receive default values of `foo.z` from `ps2`
foo(1) # x=1, y=2, z='b'
# `ps2` ends here, and `foo.y` is restored to `2`
foo(1) # x=1, y=2, z='a'
# `ps` ends here, and `foo.y` is restored to `1`
foo(1) # x=1, y=1, z='a'
```

### CMD Line Arguments

An example CLI app:

```python
from hyperparameter import param_scope, auto_param
Expand All @@ -99,17 +110,13 @@ def main(a=0, b=1): # `inline default values`

if __name__ == "__main__":
import argparse
import json
parser = argparse.ArgumentParser()
parser.add_argument("--config", type=str, default=None)

parser.add_argument("-D", "--define", nargs="*", default=[], action="extend")
args = parser.parse_args()

with open(args.config) as f:
cfg = json.load(f) # read config file
with param_scope(**cfg): # scope for `config file`
with param_scope(*args.define): # scope for `cmdline args`
main()
with param_scope(*args.define):
main()
```

Examples
Expand Down
148 changes: 78 additions & 70 deletions README.zh.md
View file
Original file line number Diff line number Diff line change
@@ -1,122 +1,130 @@
**H**_yper_**P**_arameter_
===========================
Hyperparameter
===============

<h3 align="center">
<p style="text-align: center;">
<a href="README.md" target="_blank">ENGLISH</a> | <a href="README.zh.md">中文文档</a>
</p>
</h3>

HyperParameter 是轻量级python代码配置框架，用户仅需对代码添加标记即可实现配置化。特别适用于机器学习模型的参数管理，帮助开发者对接MLOps工具；也适用于大型Python应用的配置管理。=====
<p align="center">

### 一个示例
**Hyperparameter, Make configurable AI applications.Build for Python hackers.**

假设我们开发了一个MLP结构，并使用在某个模型中：
</p>

```python
class MLP(nn.Module):
def __init__(self, units=[64, 32, 16], activation="ReLU", use_bn=False):
...

class MyAwesomeModel(nn.Module):
def __init__(self):
self.mlp = MLP()
...

model = MyAwesomeModel()
```

如果我们想尝试修改MLP结构的参数来提升模型效果，通常要修改其代码，或者在MyAwesomeModel中添加额外参数。前者代码难以维护，而后者需要大量冗余代码。
快速开始
-------

HyperParameter 通过 `auto_param`装饰器自动抽取配置参数：
`Hyperparameter` 使用装饰器 `auto _ param` 自动将keyword参数转化为超参：

```python
from hyperparameter import auto_param

@auto_param
class MLP(nn.Module):
def __init__(self, units=[64, 32, 16], activation="ReLU", use_bn=False):
...
@auto_param("foo")
def foo(x, y=1, z="a"):
return f"x={x}, y={y}, z={z}"
```

上述代码将自动生成三个参数，分别对应`MLP.__init__`方法的keyword参数：

1. `MLP.units`，默认值`[64, 32, 16]`;
2. `MLP.activation`，默认值`ReLU`;
3. `MLP.use_bn`，默认值`False`;

三个参数与keyword参数同名，并被放在`MLP`这个namespace下。参数值通过`param_scope`来控制：
超参数可以通过 `param_scope` 控制：

```python
from hyperparameter import param_scope

with param_scope(**{"MLP.activation": "sigmoid"}):
model = MyAwesomeModel()
foo(1) # x=1, y=1, z='a'
with param_scope(**{"foo.y":2}):
foo(1) # x=1, y=2, z='a'
```

高级用法
-------
### 嵌套scope

我们可以通过嵌套`param_scope`来精细控制参数，当退出一个`param_scope`，该scope对参数的修改自动失效:
### 读写超参

``` python
from hyperparameter import auto_param, param_scope
```python
from hyperparameter import param_scope

@auto_param
def foo(a=1, b=2):
print(f"a={a}, b={b}")

with param_scope(a=3):
foo() # a=3, b=2!
with param_scope(a=2, b=3):
foo() # a=2, b=3!
foo() # a=3, b=2!
# 创建param_scope
with param_scope():
pass

with param_scope("foo.y=1", "foo.z=b"):
pass

with param_scope(**{"foo.y":1, "foo.z":2}):
pass

# 读取超参（待默认值）
with param_scope(**{"foo.y":2}) as ps:
y = ps.foo.y(1)
y = ps.foo.y | 1
y = param_scope.foo.y(1)
y = param_scope.foo.y | 1
foo(1) # x=1, y=2, z='a'

# 写入超参
with param_scope(**{"foo.y":2}) as ps:
ps.foo.y = 2
param_scope.foo.y = 2
```

### 从命令行管理参数
### 嵌套Scope

我们通常推荐使用如下三层结构管理参数:
`Hyperparameter` support nested `param_scope`:

1. 代码的inline默认值，即写在函数或者类定义中的默认值;
2. 配置文件，会覆盖inline默认值;
3. 命令行参数，会覆盖配置文件和inline默认值;
``` python
from hyperparameter import param_scope

我们推荐的命令脚本示例如下：
# 当前没有param_scope，使用函数定义的默认值
foo(1) # x=1, y=1, z='a'

# 开启新的param_scope
# 并将`foo.y`默认值设为`2`
with param_scope(**{"foo.y":2}) as ps:
# 发现一个param_scope `ps`,
# 并从`ps`中获取`foo.y`的默认值
foo(1) # x=1, y=2, z='a'

# 开启另一个新的param_scope
# 并将`foo.y`默认值设为`3`
with param_scope(**{"foo.z": "b"}) as ps2:
# 发现一个嵌套param_scope `ps2`,
# 并从`ps2`中获取`foo.y`的默认值
foo(1) # x=1, y=2, z='b'
# `ps2` 结束，参数清理，`foo.y` 恢复为`2`
foo(1) # x=1, y=2, z='a'
# `ps` 结束，参数清理，`foo.y` 恢复为`1`
foo(1) # x=1, y=1, z='a'
```

### CMD Line Arguments

CLI应用示例代码:

```python
from hyperparameter import param_scope, auto_param

@auto_param
def main(a=0, b=1): # inline默认值
def main(a=0, b=1): # `inline default values`
print(a, b)

if __name__ == "__main__":
import argparse
import json
parser = argparse.ArgumentParser()
parser.add_argument("--config", type=str, default=None)

parser.add_argument("-D", "--define", nargs="*", default=[], action="extend")
args = parser.parse_args()

if args.config is not None: # 加载配置文件
with open(args.config) as f:
cfg = json.load(f)
else:
cfg = {}

with param_scope(**cfg): # 配置文件的scope
with param_scope(*args.define): # 命令行参数的scope
main()
with param_scope(*args.define):
main()
```

更多示例
-------

### [超参调节](examples/sparse_lr/README.md)
--------

该示例展示了如何使用`hyperparameter`来搭建机器学习项目，并保证复现性。
### [超参搜索](examples/sparse_lr/README.md)

### [试验跟踪](examples/mnist/README.md)
该示例展示如何在研究项目中使用hyperparameter，并让模型实验可以复现。
### [实验管理](examples/mnist/README.md)

该例子说明了如何通过`hyperparameter`进行试验管理，并跟踪试验结果。
该示例演示如何使用hyperparameter进行实验管理，并对接mlflow的tracing系统。

0 comments on commit 6a7e1ac

Please sign in to comment.