Skip to content

Commit e1f6ea6

Browse files
committed
add range
1 parent eb33e10 commit e1f6ea6

File tree

5 files changed

+29
-4
lines changed

5 files changed

+29
-4
lines changed

docs/backend-reference/basic.mdx

+13-1
Original file line numberDiff line numberDiff line change
@@ -15,6 +15,7 @@ Overview
1515
| [`Identity`](#identity) | None | `data/any` | `result/any` | `dict["result"]=dict["data"]` | Default backend |
1616
| [`Sequential`](#sequential) | `Sequential::backend` | `data/any` | | | |
1717
| [`Jump`](../Inter-node/graphtraversal.mdx#jump) | `jump` | | | Jump to other (root) node | |
18+
| [`Range`](#range) | `Range::backend`;`range` | `data/any` | | | |
1819

1920

2021

@@ -68,4 +69,15 @@ jump="||node_name_1||node_name_2"
6869
When forwarding, sub-backends are executed in series. If the result of a non-final sub-backend is empty, the execution is terminated and an exception is thrown. If the result of a non-final sub-backend is non-empty, the filter will be executed. The default is `swap`.
6970

7071
### Input Range
71-
If the input data range of all sub-backends is [1,1], then the range of `Sequential` is [1,1]. Otherwise, it is the union of the input ranges of sub-backends whose input ranges are not [1,1]. **Currently, only one sub-backend is allowed to satisfy `max() != 1 && max() != UINT32_MAX` at most**. In `Sequential`, currently only one sub-backend with non-trivial input range is allowed at most, and this sub-backend can be regarded as the main functional implementer of the entire `Sequential`, while other backends are auxiliary or functional backends.
72+
If the input data range of all sub-backends is [1,1], then the range of `Sequential` is [1,1]. Otherwise, it is the union of the input ranges of sub-backends whose input ranges are not [1,1]. **Currently, only one sub-backend is allowed to satisfy `max() != 1 && max() != UINT32_MAX` at most**. In `Sequential`, currently only one sub-backend with non-trivial input range is allowed at most, and this sub-backend can be regarded as the main functional implementer of the entire `Sequential`, while other backends are auxiliary or functional backends.
73+
74+
## Range
75+
Set a new maximum and minimum range [target_min, target_max].
76+
Use dynamic programming to determine whether all the integer 'target' in [target_min, target_max] can be represented as the sum
77+
of multiple numbers within `Range::backend`'s [min, max] range, and provide a result with the largest possible values.
78+
### Initialization
79+
:::tip Parameters
80+
- **range** - Target [min, max]. for example, `range="1,10"`.
81+
- **Range::backend** - Proxy backend.
82+
:::
83+
Initialization fails if the new range cannot be represented by the range in the old range. For example, `target=10, [min, max]=[2,9]` => `10 = 8+2`. And the following situations will fail: `range=[10,10], [min, max]=[8,8]` => `10 != 8`.

docs/showcase/showcase.mdx

+1
Original file line numberDiff line numberDiff line change
@@ -14,6 +14,7 @@ slug: /showcase
1414
| [yolox] | ![](../assets/yolox.svg) | [TensorrtTensor](../backend-reference/torch.mdx#tensorrttensor)<br />[torchpipe.utils.cpp_extension.load] | |
1515
| [PP-OCRv2] | ![](../assets/ppocr.svg) | [MapReduce](../Inter-node/graphtraversal.mdx#mapreduce)<br />[Jump] | |
1616
| [tensorrt's native int8] | | [TensorrtTensor](../backend-reference/torch.mdx#tensorrttensor) | |
17+
| [Llama] | | [Llama](https://github.com/torchpipe/LLM.TensorRT.Serve) | |
1718

1819
[resnet18]: https://github.com/torchpipe/torchpipe/tree/main/examples/resnet18
1920
[yolox]: https://github.com/torchpipe/torchpipe/tree/main/examples/yolox

docs/tools/quantization.mdx

+1-1
Original file line numberDiff line numberDiff line change
@@ -11,7 +11,7 @@ The post-training quantization process of TensorRT mainly includes two steps:
1111
- Prepare quantization data, about 500 pieces. We will save the data in PyTorch format in the backend.
1212
- Launch the quantization process with the prepared data and generate the model.
1313

14-
For more details, please refer to the [example](https://github.com/torchpipe/torchpipe//examples/int8/README_en.md).
14+
For more details, please refer to the [example](https://github.com/torchpipe/torchpipe/tree/develop/examples/int8).
1515

1616
## Training-based Quantization (based on pytorch_quantization)
1717
In the [official notebook](https://github.com/NVIDIA/TensorRT/blob/release/8.6/quickstart/quantization_tutorial/qat-ptq-workflow.ipynb), NVIDIA summarizes how to improve quantization accuracy through training-based quantization in PyTorch.

i18n/zh/docusaurus-plugin-content-docs/current/backend-reference/basic.mdx

+13-1
Original file line numberDiff line numberDiff line change
@@ -17,6 +17,7 @@ displayed_sidebar: api
1717
| [`Identity`](#identity) || `data/any` | `result/any` | `dict["result"]=dict["data"]` | 默认后端 |
1818
| [`Sequential`](#sequential) | `Sequential::backend` | `data/any` | | | |
1919
| [`Jump`](../Inter-node/graphtraversal.mdx#jump) | `jump` | | | 跳转到其他(根)节点 | |
20+
| [`Range`](#range) | `Range::backend`;`range` | `data/any` | | | |
2021

2122

2223

@@ -91,4 +92,15 @@ jump="||node_name_1||node_name_2"
9192

9293
:::caution
9394
被提升输入范围的后端的前向接口必须是线程安全的,并且状态不依赖于线程本身。[torch相关后端](./torch.mdx)不被支持
94-
::: -->
95+
::: -->
96+
97+
## Range
98+
设定新的最大最小范围 [target_min, target_max]
99+
使用动态规划来确定 [target_min, target_max] 范围内的所有整数 'target' 是否可以表示为 `Range::backend`[min, max] 范围内的多个数字之和,并提供可能的最大值结果。
100+
### 初始化
101+
:::tip 参数
102+
- **range** - 目标 [min, max]。例如,`range="1,10"`
103+
- **Range::backend** - 代理的后端。
104+
:::
105+
如果新的范围无法被旧的范围中的范围表示,则初始化失败。例如,`range=[10,10], [min, max]=[2,9]` => `10 = 8+2`
106+
而以下情况会失败:`range=[10,10], [min, max]=[8,8]` => `10 != 8`

i18n/zh/docusaurus-plugin-content-docs/current/tools/quantization.mdx

+1-1
Original file line numberDiff line numberDiff line change
@@ -10,7 +10,7 @@ tensorrt 的训练后量化过程主要包含两步:
1010
- 准备量化数据,500份左右。这部分我们将进入后端的数据按照Pytorch的格式保存下来
1111
- 以准备好的数据启动量化过程,并生成模型。
1212

13-
详细可参见[示例](https://github.com/torchpipe/torchpipe//examples/int8).
13+
详细可参见[示例](https://github.com/torchpipe/torchpipe/tree/develop/examples/int8).
1414

1515
## 训练时量化(基于pytorch_quantization)
1616
[官方notebook](https://github.com/NVIDIA/TensorRT/blob/release/8.6/quickstart/quantization_tutorial/qat-ptq-workflow.ipynb)中,NVIDIA总结了如何在Pytorch中通过训练时量化提升量化精度。

0 commit comments

Comments
 (0)