Skip to content

Conversation

@strint
Copy link
Contributor

@strint strint commented Sep 13, 2021

增加一个graph支持的lr的流程

  1. c++ lr kernel
  2. lr proto
  3. python eager lr add graph conf
  4. test in test_graph_lrs.py

待支持的

  • multistep lr
  • step lr
  • cosine annealing lr

除了lambda lr,当前eager的lr,graph都已经支持

@strint strint requested review from oneflow-ci-bot and removed request for oneflow-ci-bot September 13, 2021 11:28
@oneflow-ci-bot oneflow-ci-bot requested review from oneflow-ci-bot and removed request for oneflow-ci-bot September 13, 2021 15:27
@oneflow-ci-bot oneflow-ci-bot requested review from oneflow-ci-bot and removed request for oneflow-ci-bot September 13, 2021 16:25
@oneflow-ci-bot oneflow-ci-bot requested review from oneflow-ci-bot and removed request for oneflow-ci-bot September 13, 2021 17:16
@github-actions
Copy link
Contributor

Speed stats:
GPU Name: GeForce GTX 1080 

OneFlow resnet50 time: 128.0ms (= 6399.0ms / 50, input_shape=[16, 3, 224, 224])
PyTorch resnet50 time: 138.3ms (= 6915.6ms / 50, input_shape=[16, 3, 224, 224])
✔️ Relative speed: 1.08 (= 138.3ms / 128.0ms)

OneFlow resnet50 time: 74.1ms (= 3703.8ms / 50, input_shape=[8, 3, 224, 224])
PyTorch resnet50 time: 85.0ms (= 4251.7ms / 50, input_shape=[8, 3, 224, 224])
✔️ Relative speed: 1.15 (= 85.0ms / 74.1ms)

OneFlow resnet50 time: 49.4ms (= 2471.7ms / 50, input_shape=[4, 3, 224, 224])
PyTorch resnet50 time: 59.1ms (= 2953.9ms / 50, input_shape=[4, 3, 224, 224])
✔️ Relative speed: 1.20 (= 59.1ms / 49.4ms)

OneFlow resnet50 time: 43.7ms (= 2187.4ms / 50, input_shape=[2, 3, 224, 224])
PyTorch resnet50 time: 49.4ms (= 2467.7ms / 50, input_shape=[2, 3, 224, 224])
✔️ Relative speed: 1.13 (= 49.4ms / 43.7ms)

OneFlow resnet50 time: 44.3ms (= 2212.8ms / 50, input_shape=[1, 3, 224, 224])
PyTorch resnet50 time: 43.8ms (= 2189.8ms / 50, input_shape=[1, 3, 224, 224])
✔️ Relative speed: 0.99 (= 43.8ms / 44.3ms)

OneFlow resnet50 time: 160.5ms (= 8023.1ms / 50, input_shape=[16, 3, 224, 224], ddp, world size=2)
PyTorch resnet50 time: 158.3ms (= 7915.7ms / 50, input_shape=[16, 3, 224, 224], ddp, world size=2)
✔️ Relative speed: 0.99 (= 158.3ms / 160.5ms)

OneFlow resnet50 time: 103.0ms (= 5149.3ms / 50, input_shape=[8, 3, 224, 224], ddp, world size=2)
PyTorch resnet50 time: 102.3ms (= 5117.1ms / 50, input_shape=[8, 3, 224, 224], ddp, world size=2)
✔️ Relative speed: 0.99 (= 102.3ms / 103.0ms)

OneFlow resnet50 time: 80.3ms (= 4016.2ms / 50, input_shape=[4, 3, 224, 224], ddp, world size=2)
PyTorch resnet50 time: 74.6ms (= 3729.2ms / 50, input_shape=[4, 3, 224, 224], ddp, world size=2)
✔️ Relative speed: 0.93 (= 74.6ms / 80.3ms)

OneFlow resnet50 time: 77.6ms (= 3882.3ms / 50, input_shape=[2, 3, 224, 224], ddp, world size=2)
PyTorch resnet50 time: 63.8ms (= 3187.8ms / 50, input_shape=[2, 3, 224, 224], ddp, world size=2)
✔️ Relative speed: 0.82 (= 63.8ms / 77.6ms)

OneFlow resnet50 time: 69.4ms (= 3469.8ms / 50, input_shape=[1, 3, 224, 224], ddp, world size=2)
PyTorch resnet50 time: 60.3ms (= 3013.5ms / 50, input_shape=[1, 3, 224, 224], ddp, world size=2)
✔️ Relative speed: 0.87 (= 60.3ms / 69.4ms)

@oneflow-ci-bot oneflow-ci-bot removed their request for review September 13, 2021 18:29
@oneflow-ci-bot oneflow-ci-bot merged commit 76e78fd into master Sep 13, 2021
@oneflow-ci-bot oneflow-ci-bot deleted the fea/graph/lrs branch September 13, 2021 18:31
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants