-
Notifications
You must be signed in to change notification settings - Fork 456
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[Backend] support ipu in paddle inference backend. #437
[Backend] support ipu in paddle inference backend. #437
Conversation
748a405
to
aa21ea5
Compare
测试结果:example测试:测试覆盖fastdeploy readme 中提供的除inceptionV3以外所有模型,用一张图片测试推理。
benchmark测试使用脚本,将其中的运行命令改为:
测试覆盖readme 中提供的除inceptionV3以外所有模型。截取部分输出log如下:
注意: 这里的benchmark结果仅仅用于PR测试,由于后续的硬件变动,当前的数据不具有性能参考意义。 |
fastdeploy/runtime.h
Outdated
* \param[in] batches_per_step the number of batches per run in pipelining. | ||
*/ | ||
void EnableIpu(int device_num = 1, int micro_batch_size = 1, | ||
bool enable_pipelining = false, int batches_per_step = 1); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
UseIpu
与EnableIpu
两个接口合为一个看起来会更方便,类似UseGpu
时,支持同时配置GPU的device_id
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
这里的接口设计是为了和paddle inference中的IPU接口保持了一致。所以没有合并到一起。
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
保持接口简洁, IPU先采用如下两个用户接口设计:
void UseIpu(int device_num = 1, int micro_batch_size = 1,bool enable_pipelining = false, int batches_per_step = 1);
void SetIpuConfig(bool enable_fp16 = false, int replica_num = 1,
float available_memory_proportion = 1.0,
bool enable_half_partial = false);
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Updated
@leiqing1 麻烦帮忙Review下文档的修改 |
2b61fdb
to
e2fa75e
Compare
解决develop 分支冲突出了点差错,git amend 冲突正确的改动上来 |
PR types(PR类型)
Backend
Describe
增加paddle inference backend对IPU的支持。