Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Debug dim scatter #5371

Merged
merged 109 commits into from
Jul 31, 2021
Merged

Debug dim scatter #5371

merged 109 commits into from
Jul 31, 2021

Conversation

MARD1NO
Copy link
Contributor

@MARD1NO MARD1NO commented Jul 2, 2021

相关细节

  1. 针对后向传播,增加了scatter_update的scalar版本(由于torch的scatter_add没有scalar版本,故没有增加)
  2. 对齐torch反向传播要求,即index.shape == source.shape才允许反传
  3. 修改了单测index的生成逻辑,保证index不存在重复元素。原因是经过torch下测试,当存在重复元素时,GPU版的scatter_update出来的结果不一致(由于线程单独执行逻辑),而CPU版本由于其顺序执行的方式,使其输出结果一致。

新增接口

  • oneflow.scatter 的前后向
  • oneflow.scatter_add 的前后向
  • oneflow.F.dim_scatter 的前后向
  • oneflow.F.dim_scatter_add 的前后向
  • oneflow.F.dim_scatter_scalar 的前后向
  • oneflow.F.dim_scatter_add_scalar (只有前向,没有后向,这个 PyTorch 没有)

以上没有后向的算子,是因为 PyTorch 也没有后向。

文档生成:

image
image

@oneflow-ci-bot oneflow-ci-bot removed their request for review July 30, 2021 11:22
@oneflow-ci-bot oneflow-ci-bot self-requested a review July 30, 2021 11:22
@oneflow-ci-bot oneflow-ci-bot requested review from oneflow-ci-bot and removed request for oneflow-ci-bot July 30, 2021 13:16
@oneflow-ci-bot oneflow-ci-bot requested review from oneflow-ci-bot and removed request for oneflow-ci-bot July 30, 2021 14:36
@oneflow-ci-bot oneflow-ci-bot requested review from oneflow-ci-bot and removed request for oneflow-ci-bot July 30, 2021 15:50
@oneflow-ci-bot oneflow-ci-bot requested review from oneflow-ci-bot and removed request for oneflow-ci-bot July 30, 2021 17:31
@github-actions
Copy link
Contributor

CI failed, removing label automerge

1 similar comment
@github-actions
Copy link
Contributor

CI failed, removing label automerge

@oneflow-ci-bot oneflow-ci-bot removed their request for review July 30, 2021 18:51
@doombeaker doombeaker requested review from oneflow-ci-bot and removed request for oneflow-ci-bot July 31, 2021 04:54
@github-actions
Copy link
Contributor

Speed stats:
GPU Name: GeForce GTX 1080 

PyTorch resnet50 time: 140.2ms (= 7009.5ms / 50, input_shape=[16, 3, 224, 224], backward is enabled)
OneFlow resnet50 time: 126.6ms (= 6329.3ms / 50, input_shape=[16, 3, 224, 224], backward is enabled)
Relative speed: 1.11 (= 140.2ms / 126.6ms)

PyTorch resnet50 time: 83.5ms (= 4173.3ms / 50, input_shape=[8, 3, 224, 224], backward is enabled)
OneFlow resnet50 time: 75.2ms (= 3758.0ms / 50, input_shape=[8, 3, 224, 224], backward is enabled)
Relative speed: 1.11 (= 83.5ms / 75.2ms)

PyTorch resnet50 time: 57.4ms (= 2868.3ms / 50, input_shape=[4, 3, 224, 224], backward is enabled)
OneFlow resnet50 time: 47.6ms (= 2381.0ms / 50, input_shape=[4, 3, 224, 224], backward is enabled)
Relative speed: 1.20 (= 57.4ms / 47.6ms)

PyTorch resnet50 time: 48.9ms (= 2443.9ms / 50, input_shape=[2, 3, 224, 224], backward is enabled)
OneFlow resnet50 time: 43.2ms (= 2160.2ms / 50, input_shape=[2, 3, 224, 224], backward is enabled)
Relative speed: 1.13 (= 48.9ms / 43.2ms)

PyTorch resnet50 time: 41.4ms (= 2072.3ms / 50, input_shape=[1, 3, 224, 224], backward is enabled)
OneFlow resnet50 time: 42.3ms (= 2114.5ms / 50, input_shape=[1, 3, 224, 224], backward is enabled)
Relative speed: 0.98 (= 41.4ms / 42.3ms)

@oneflow-ci-bot oneflow-ci-bot removed their request for review July 31, 2021 05:45
@oneflow-ci-bot oneflow-ci-bot merged commit 7948262 into master Jul 31, 2021
@oneflow-ci-bot oneflow-ci-bot deleted the debug_dim_scatter branch July 31, 2021 05:45
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

3 participants