optim performance and cls/rec batchsize for inference #480
Conversation
liangxhao
changed the title
liangxhao
changed the title
liangxhao
requested review from
SamitHuang,
HaoyangLee,
VictorHe-1 and
heepengpeng
|
|
||
| if image_total > 0: | ||
| self.profiling(profiling_data, image_total) | ||
| print( |
There was a problem hiding this comment.
这个之前是写在log中的，后来被反馈后修改增加了print。
因为其它log一般不重要默认关闭，但是性能信息log却很重要，故需要单独打出来，确保不开log也能看得到。
| @@ -39,7 +78,8 @@ def free_model(self): | |||
| del model | |||
| else: | |||
| del self.model | |||
| self.model = None | |||
|
|
|||
| self.model = None | |||
|
|
|||
| def __del__(self): | |||
| self.free_model() | |||
There was a problem hiding this comment.
将self.free_model()的代码直接移动到__del__里面
There was a problem hiding this comment.
free_model()有时需要单独调用，而不依赖__del__
| @@ -39,7 +78,8 @@ def free_model(self): | |||
| del model | |||
There was a problem hiding this comment.
删除循环，直接使用del self.model[:]或者del self.model，参见：https://stackoverflow.com/questions/12417498/how-to-release-used-memory-immediately-in-python-list
There was a problem hiding this comment.
我这里写错了，self.model是字典，已修复
self.model.clear()
del self.model
gc.collect()
|
|
||
| self._hw_list = [] | ||
| self._bs_list = [] | ||
|
|
||
| self.model = defaultdict() |
There was a problem hiding this comment.
self.model = defaultdict(Model)
There was a problem hiding this comment.
Model是有必选参数的，若使用defaultdict(Model)，命中时会因缺少初始化参数而报错
已修复为：
self.model: Dict[int, Model] = {}
There was a problem hiding this comment.
change input = data["image"] to input_data = data["image"] in function model_infer
| f"total cost {cost_time:.2f}s, FPS: " | ||
| f"{safe_div(image_total, cost_time):.2f}" | ||
| ) | ||
| print(perf_info) |
There was a problem hiding this comment.
这个perf_info 108行也有个log.info，也是需要不开log也打印的吗？
There was a problem hiding this comment.
对，既要确保开log时输入到log文件中，又要确保不开log时打印到屏幕（否则没法测试性能）
| for i in range(batch): | ||
| label, score = output[i] | ||
| if "180" == label and score > self.cls_thresh: | ||
| sub_images[i] = cv2.rotate(sub_images[i], cv2.ROTATE_180) | ||
| input_data.sub_image_list = sub_images | ||
| else: | ||
| # TODO: only support batch=1 | ||
| input_data.infer_result = output[0] | ||
| input_data.infer_result = output | ||
|
|
||
| input_data.data = None |
There was a problem hiding this comment.
delete input_data.data first and then set to None to avoid memory leak
There was a problem hiding this comment.
添加del可能会引起gc概率性的提前执行，导致性能降低
这里是由python引用计数器来自动记录，一般会在cpu相对空闲时（比如数据队列发送等待时）自行gc释放内存
Thank you for your contribution to the MindOCR repo.
Before submitting this PR, please make sure:
Motivation
实现逻辑：在每个模型推理子进程获取参数值，同步到主进程，在主进程发送数据之前对数据切分打包，避免在子进程缓存数据
crnn_resnet34vd, ic15-2077, fps
Test Plan
(How should this PR be tested? Do you require special setup to run the test or repro the fixed bug?)
Related Issues and PRs
(Is this PR part of a group of changes? Link the other relevant PRs and Issues here. Use https://help.github.com/en/articles/closing-issues-using-keywords for help on GitHub syntax)