Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

add clang tidy target #5957

Merged
merged 10 commits into from
Aug 19, 2021
Merged

add clang tidy target #5957

merged 10 commits into from
Aug 19, 2021

Conversation

mosout
Copy link
Contributor

@mosout mosout commented Aug 19, 2021

No description provided.

PragmaTwice
PragmaTwice reviewed Aug 19, 2021
View reviewed changes
ci/check/clang_tidy_diff.py Outdated
@@ -0,0 +1,318 @@
#!/usr/bin/env python
Copy link
Contributor

@PragmaTwice PragmaTwice Aug 19, 2021 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think it would be better if we could download both files from the Oneflow-Inc/llvm-project repo before we do the check, so that we do not need to maintain two copies at the same time

PragmaTwice
PragmaTwice reviewed Aug 19, 2021
View reviewed changes
cmake/oneflow.cmake Outdated

# clang tidy
add_custom_target(of_tidy
COMMAND ${Python_EXECUTABLE} ${CMAKE_SOURCE_DIR}/ci/check/run_clang_tidy.py --source_dir ${CMAKE_SOURCE_DIR} --build_dir ${CMAKE_BINARY_DIR}
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe you need to refer here: https://github.com/PragmaTwice/oneflow/blob/ci-clang-tidy-add-more-checkers/.github/workflows/simple.yml#L65

@PragmaTwice
Copy link
Contributor

You can follow here to download our custom clang-tidy and run-clang-tidy.py (maybe a proxy is needed in mainland China, i.e. ghproxy.com): https://github.com/Oneflow-Inc/oneflow/blob/master/.github/workflows/simple.yml#L42

@daquexian
Copy link
Contributor

You can follow here to download our custom clang-tidy and run-clang-tidy.py (maybe a proxy is needed in mainland China, i.e. ghproxy.com): https://github.com/Oneflow-Inc/oneflow/blob/master/.github/workflows/simple.yml#L42

话说 clang-tidy-diff.py 是不是放在 oneflow 仓库里维护比较好，这个脚本需要下载的话有些奇怪

@PragmaTwice
Copy link
Contributor

PragmaTwice commented Aug 19, 2021 •
edited
Loading

话说 clang-tidy-diff.py 是不是放在 oneflow 仓库里维护比较好，这个脚本需要下载的话有些奇怪

emmm 确实，那就默认 llvm-project 里面的那两个脚本不再维护了吗？主要是 clang-tidy 本身是需要下载的，可以顺带着把脚本下载下来（clang 的完整安装本身就是带着 run-clang-tidy.py 的，一般在 /usr/share/clang/ 里，既然 clang 是定制化的，脚本也是定制化地也不是很难说得通）
目前我看貌似没用到 clang-tidy-diff.py ？

@daquexian
Copy link
Contributor

主要是 clang-tidy 本身是需要下载的，可以顺带着把脚本下载下来（clang 的完整安装本身就是带着 run-clang-tidy.py 的，一般在 /usr/share/clang/ 里，既然 clang 是定制化的，脚本也是定制化地也不是很难说得通）

感觉有道理，我也觉得放在 oneflow 仓库维护的话 llvm-project 仓库的那份不好处理。那就还把脚本放在 llvm-project 维护吧，在这里下载 @mosout

daquexian
daquexian reviewed Aug 19, 2021
View reviewed changes
cmake/oneflow.cmake
@@ -209,7 +209,10 @@ add_custom_target(of_format
COMMAND ${Python_EXECUTABLE} ${CMAKE_CURRENT_SOURCE_DIR}/ci/check/run_clang_format.py --source_dir ${CMAKE_CURRENT_SOURCE_DIR}/oneflow --fix --quiet
COMMAND ${Python_EXECUTABLE} ${CMAKE_CURRENT_SOURCE_DIR}/ci/check/run_py_format.py --source_dir ${CMAKE_CURRENT_SOURCE_DIR} --fix
)

# clang tidy
add_custom_target(of_tidy
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

给 of_tidy 增加几个 dependencies，https://github.com/PragmaTwice/oneflow/blob/ci-clang-tidy-add-more-checkers/.github/workflows/simple.yml#L52 这里的这些

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

好的

@mosout
Copy link
Contributor Author

mosout commented Aug 19, 2021

主要是 clang-tidy 本身是需要下载的，可以顺带着把脚本下载下来（clang 的完整安装本身就是带着 run-clang-tidy.py 的，一般在 /usr/share/clang/ 里，既然 clang 是定制化的，脚本也是定制化地也不是很难说得通）

感觉有道理，我也觉得放在 oneflow 仓库维护的话 llvm-project 仓库的那份不好处理。那就还把脚本放在 llvm-project 维护吧，在这里下载 @mosout

好的

daquexian
daquexian reviewed Aug 19, 2021
View reviewed changes
Copy link
Contributor

@daquexian daquexian left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

还需要在 cmake 里设置 CMAKE_EXPORT_COMPILE_COMMANDS 为 ON

ci/check/run_clang_tidy.py Outdated
else "https://oneflow-static.oss-cn-beijing.aliyuncs.com/bin/clang-tidy/linux-x86_64/clang-tidy.AppImage",
"https://raw.githubusercontent.com/oneflow-inc/llvm-project/maybe/clang-tools-extra/clang-tidy/tool/clang-tidy-diff.py",
]
dst_dir = f"{build_dir}/.cache/bin"
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

既然已经在 build 目录里，可以不用加 . 前缀

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

好的

ci/check/run_clang_tidy.py Outdated
cmd, stdout=asyncio.subprocess.PIPE, stderr=asyncio.subprocess.PIPE,
)
l = lambda x: split_and_print(f"[{name}]" if name else "", x)
# l = lambda x: x
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

这个是不是需要删掉

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

好

@oneflow-ci-bot oneflow-ci-bot requested review from oneflow-ci-bot and removed request for oneflow-ci-bot August 19, 2021 10:46
@oneflow-ci-bot oneflow-ci-bot self-requested a review August 19, 2021 11:15
@oneflow-ci-bot oneflow-ci-bot requested review from oneflow-ci-bot and removed request for oneflow-ci-bot August 19, 2021 13:45
@oneflow-ci-bot oneflow-ci-bot requested review from oneflow-ci-bot and removed request for oneflow-ci-bot August 19, 2021 14:55
@oneflow-ci-bot oneflow-ci-bot merged commit 494e671 into Oneflow-Inc:master Aug 19, 2021
@mosout mosout deleted the add_clang_tidy branch August 20, 2021 02:16
VertexC added a commit that referenced this pull request Aug 20, 2021
@VertexC @lixinqi
@clackhan @oneflow-ci-bot @daquexian @mosout @PragmaTwice @zhangshen12356 @Flowingsun007 @wyg1997 @leaves-zwx @jackalcooper @liufengwei0103
Dev randint refine (#5981)
671f4f5 
* disable backward pass consistent tensor meta check. (#5871)

* disable backward pass consistent tensor meta check.

* auto format by CI

Co-authored-by: binbinHan <han_binbin@163.com>
Co-authored-by: oneflow-ci-bot <69100618+oneflow-ci-bot@users.noreply.github.com>
Co-authored-by: oneflow-ci-bot <ci-bot@oneflow.org>

* ddp broadcast params and buffers (#5913)

* ddp broadcast params and buffers

Signed-off-by: daquexian <daquexian566@gmail.com>

* auto format by CI

Co-authored-by: oneflow-ci-bot <69100618+oneflow-ci-bot@users.noreply.github.com>
Co-authored-by: oneflow-ci-bot <ci-bot@oneflow.org>

* add clang tidy target (#5957)

* add clang tidy target

* fix a bug

* refine

* refine

* reformat

Co-authored-by: oneflow-ci-bot <69100618+oneflow-ci-bot@users.noreply.github.com>

* cfg: add move assignment operator for performance (#5962)

Co-authored-by: oneflow-ci-bot <69100618+oneflow-ci-bot@users.noreply.github.com>

* add zhangshen op-test (#5600)

* add some op-test

* fix dims_error in my branch

* Fix the bad backward kernel function by using 'cuda::atomic::Add' (#5614)

* Test `nn.AdaptiveAvgPoolXd` (#5615)

* Fix the bad backward kernel function by using 'cuda::atomic::Add'

* Support the 'NoneType' annotation

* Support objects of 'collections.abc.Iterable' as 'output_size'

* Test with all cases of 'output_size'

* Update adaptive_pool_gpu_kernel.cu

* Skip testing `nn.AdaptiveAvgPool3d` for the current PyTorch

* remove some useless test

* Format TODO

* Add the assertion messages for 'output_size'

* Reformat codes

* Remove raw tests for `flow.negative`

* Remove unnecessary codes and add the assertion messages

* Merge updates for 'generators.py' from master

* Remove unnecessary 'random()'

* Delete the separate test for `AvgPool2d`

* Fix import paths

* Fix import problems

* Remove the PyTorch import

* Denote the annotations for `tile` and `repeat` ops

* Add the test for `nn.AvgPool1d`

* Choose better generators for `nn.MaxPoolXd`

* Randomly choose `dilation` and default values

* auto format by CI

* Test more kwargs for `nn.AvgPoolXd`

* Add tests for `return_indices`

* auto format by CI

Co-authored-by: Tianyu Zhao <guikarist@gmail.com>
Co-authored-by: oneflow-ci-bot <69100618+oneflow-ci-bot@users.noreply.github.com>
Co-authored-by: oneflow-ci-bot <ci-bot@oneflow.org>

* fix wrong names (#5951)

* fix wrong names

* auto format by CI

* refine

* auto format by CI

Co-authored-by: oneflow-ci-bot <ci-bot@oneflow.org>
Co-authored-by: oneflow-ci-bot <69100618+oneflow-ci-bot@users.noreply.github.com>

* Enable more checkers for clang-tidy in CI (#5738)

* CI: enable more checkers for clang-tidy

* .clang-tidy: remove cppcoreguidelines-pro-type-vararg

* CI: remove duplicate checkers

* CI: remove clang-analyzer-alpha.deadcode.*

* .clang-tidy: add performance-*

* oneflow/core/eager: remove unnecessary malloc & free

* .clang-tidy: add clang-analyzer-cplusplus.* to werror

* user_kernel: remove useless move

* quantization_aware_training: fix move return

* .clang-tidy: add google-*

* CI: fix clang tidy command

* CI: fix test

Co-authored-by: oneflow-ci-bot <69100618+oneflow-ci-bot@users.noreply.github.com>

* Feat grad mode classes (#5956)

* feat(no_grad): support no_grad decorator

* feat(AutogradMode): export flow.autograd_mode

* feat(GradMode): export some grad_mode class

* docs(GradMode): export documents

* refine

* docs(GradMode): export document for is_grad_enabled

* auto format by CI

* fix(GradMode): fix single client bug

* fix bug

Co-authored-by: oneflow-ci-bot <69100618+oneflow-ci-bot@users.noreply.github.com>
Co-authored-by: oneflow-ci-bot <ci-bot@oneflow.org>

* extract_consistent_to_consistent_op_expr (#5870)

* abstract_consistent_to_consistent_op_expr

* fix compiler complaint

* refactor consistent-to-consistent eager consisitent op interpreter

* fix compiler complaint

* refactor ConsistentToConsistentOpExpr

* lazy interpreter (#5903)

* fix bugs about consistent_id

* refactor functional::ToConsistent

* refactor GetNdSbp

* Update eager_consistent_op_interpreter.cpp

* Update eager_mirrored_op_interpreter.cpp

* fix error

* fix error

* auto format by CI

* Update nd_sbp.h

* refine identity boxing

* fix sync checkmeta error

* avoid consistent id check in lazy

Co-authored-by: Xinqi Li <lixinqi0703106@163.com>
Co-authored-by: leaves-zwx <kunta0932@gmail.com>
Co-authored-by: Li Xinqi <lixinqi2010@gmail.com>
Co-authored-by: oneflow-ci-bot <69100618+oneflow-ci-bot@users.noreply.github.com>
Co-authored-by: oneflow-ci-bot <ci-bot@oneflow.org>

* add CMAKE_INTERPROCEDURAL_OPTIMIZATION in fast cmake cache (#5970)

* add CMAKE_INTERPROCEDURAL_OPTIMIZATION in fast cmake cache

* skip test targets of re2

Co-authored-by: oneflow-ci-bot <69100618+oneflow-ci-bot@users.noreply.github.com>

* check: fix clang-tidy-diff commands (#5972)

* check: fix clang-tidy-diff commands

* CI: fix step names

Co-authored-by: oneflow-ci-bot <69100618+oneflow-ci-bot@users.noreply.github.com>

* Cpu mpi (#5865)

* cuda base cpu mpi boxing

* cpu_mpi

* fix conflicts

* add cpu mpi unittests

* more checks and unittests

* abstract_consistent_to_consistent_op_expr

* fix compiler complaint

* refactor consistent-to-consistent eager consisitent op interpreter

* fix compiler complaint

* refactor ConsistentToConsistentOpExpr

* lazy interpreter (#5903)

* fix bugs about consistent_id

* more test_consistent_cast unittests

* refactor functional::ToConsistent

* refactor GetNdSbp

* fix compiler complaints

* refactor GetDevice4CurrentProcessCtx

* fix error

Co-authored-by: clackhan <han_binbin@163.com>
Co-authored-by: leaves-zwx <kunta0932@gmail.com>
Co-authored-by: oneflow-ci-bot <69100618+oneflow-ci-bot@users.noreply.github.com>

* fix_bug_test_tensor_str (#5958)

* fix bug int test_tensor_str

* format

* fix comment

* fix bug to(cuda) is unavailable in cpu env

Co-authored-by: oneflow-ci-bot <69100618+oneflow-ci-bot@users.noreply.github.com>

* common/error: fix build error in mac (#5971)

Co-authored-by: oneflow-ci-bot <69100618+oneflow-ci-bot@users.noreply.github.com>

* Prevent running oneflow in forked subprocess (#5976)

* prevent_running_oneflow_in_forked_subprocess

* add line change

* IsFork => IsForkedSubProcess

* auto format by CI

Co-authored-by: oneflow-ci-bot <69100618+oneflow-ci-bot@users.noreply.github.com>
Co-authored-by: oneflow-ci-bot <ci-bot@oneflow.org>

* refine randint

Co-authored-by: Li Xinqi <lixinqi2010@gmail.com>
Co-authored-by: binbinHan <han_binbin@163.com>
Co-authored-by: oneflow-ci-bot <69100618+oneflow-ci-bot@users.noreply.github.com>
Co-authored-by: oneflow-ci-bot <ci-bot@oneflow.org>
Co-authored-by: daquexian <daquexian566@gmail.com>
Co-authored-by: Peihong Liu <mosout@qq.com>
Co-authored-by: Twice <i@twice.moe>
Co-authored-by: ZhangShen <55383772+zhangshen12356@users.noreply.github.com>
Co-authored-by: Tianyu Zhao <guikarist@gmail.com>
Co-authored-by: Luyang <flowingsun007@163.com>
Co-authored-by: Yinggang Wang <wyg19970408@gmail.com>
Co-authored-by: Xinqi Li <lixinqi0703106@163.com>
Co-authored-by: leaves-zwx <kunta0932@gmail.com>
Co-authored-by: Shenghang Tsai <jackalcooper@gmail.com>
Co-authored-by: liufengwei0103 <2472937968@qq.com>
Flowingsun007 added a commit that referenced this pull request Aug 27, 2021
@Kevin-XiongC @VertexC
@lixinqi @clackhan @oneflow-ci-bot @daquexian @mosout @PragmaTwice @zhangshen12356 @Flowingsun007 @wyg1997 @leaves-zwx @jackalcooper @liufengwei0103
add randint (#5718)
aadbb46 
* add randint

* add

* add doc test

* Update randint_kernel.cu

* Update randint_kernel.cpp

* Update randint_op.cpp

* reconstrcut

* refine the code

* add test

* add test

* add test

* format

* Dev randint refine (#5981)

* disable backward pass consistent tensor meta check. (#5871)

* disable backward pass consistent tensor meta check.

* auto format by CI

Co-authored-by: binbinHan <han_binbin@163.com>
Co-authored-by: oneflow-ci-bot <69100618+oneflow-ci-bot@users.noreply.github.com>
Co-authored-by: oneflow-ci-bot <ci-bot@oneflow.org>

* ddp broadcast params and buffers (#5913)

* ddp broadcast params and buffers

Signed-off-by: daquexian <daquexian566@gmail.com>

* auto format by CI

Co-authored-by: oneflow-ci-bot <69100618+oneflow-ci-bot@users.noreply.github.com>
Co-authored-by: oneflow-ci-bot <ci-bot@oneflow.org>

* add clang tidy target (#5957)

* add clang tidy target

* fix a bug

* refine

* refine

* reformat

Co-authored-by: oneflow-ci-bot <69100618+oneflow-ci-bot@users.noreply.github.com>

* cfg: add move assignment operator for performance (#5962)

Co-authored-by: oneflow-ci-bot <69100618+oneflow-ci-bot@users.noreply.github.com>

* add zhangshen op-test (#5600)

* add some op-test

* fix dims_error in my branch

* Fix the bad backward kernel function by using 'cuda::atomic::Add' (#5614)

* Test `nn.AdaptiveAvgPoolXd` (#5615)

* Fix the bad backward kernel function by using 'cuda::atomic::Add'

* Support the 'NoneType' annotation

* Support objects of 'collections.abc.Iterable' as 'output_size'

* Test with all cases of 'output_size'

* Update adaptive_pool_gpu_kernel.cu

* Skip testing `nn.AdaptiveAvgPool3d` for the current PyTorch

* remove some useless test

* Format TODO

* Add the assertion messages for 'output_size'

* Reformat codes

* Remove raw tests for `flow.negative`

* Remove unnecessary codes and add the assertion messages

* Merge updates for 'generators.py' from master

* Remove unnecessary 'random()'

* Delete the separate test for `AvgPool2d`

* Fix import paths

* Fix import problems

* Remove the PyTorch import

* Denote the annotations for `tile` and `repeat` ops

* Add the test for `nn.AvgPool1d`

* Choose better generators for `nn.MaxPoolXd`

* Randomly choose `dilation` and default values

* auto format by CI

* Test more kwargs for `nn.AvgPoolXd`

* Add tests for `return_indices`

* auto format by CI

Co-authored-by: Tianyu Zhao <guikarist@gmail.com>
Co-authored-by: oneflow-ci-bot <69100618+oneflow-ci-bot@users.noreply.github.com>
Co-authored-by: oneflow-ci-bot <ci-bot@oneflow.org>

* fix wrong names (#5951)

* fix wrong names

* auto format by CI

* refine

* auto format by CI

Co-authored-by: oneflow-ci-bot <ci-bot@oneflow.org>
Co-authored-by: oneflow-ci-bot <69100618+oneflow-ci-bot@users.noreply.github.com>

* Enable more checkers for clang-tidy in CI (#5738)

* CI: enable more checkers for clang-tidy

* .clang-tidy: remove cppcoreguidelines-pro-type-vararg

* CI: remove duplicate checkers

* CI: remove clang-analyzer-alpha.deadcode.*

* .clang-tidy: add performance-*

* oneflow/core/eager: remove unnecessary malloc & free

* .clang-tidy: add clang-analyzer-cplusplus.* to werror

* user_kernel: remove useless move

* quantization_aware_training: fix move return

* .clang-tidy: add google-*

* CI: fix clang tidy command

* CI: fix test

Co-authored-by: oneflow-ci-bot <69100618+oneflow-ci-bot@users.noreply.github.com>

* Feat grad mode classes (#5956)

* feat(no_grad): support no_grad decorator

* feat(AutogradMode): export flow.autograd_mode

* feat(GradMode): export some grad_mode class

* docs(GradMode): export documents

* refine

* docs(GradMode): export document for is_grad_enabled

* auto format by CI

* fix(GradMode): fix single client bug

* fix bug

Co-authored-by: oneflow-ci-bot <69100618+oneflow-ci-bot@users.noreply.github.com>
Co-authored-by: oneflow-ci-bot <ci-bot@oneflow.org>

* extract_consistent_to_consistent_op_expr (#5870)

* abstract_consistent_to_consistent_op_expr

* fix compiler complaint

* refactor consistent-to-consistent eager consisitent op interpreter

* fix compiler complaint

* refactor ConsistentToConsistentOpExpr

* lazy interpreter (#5903)

* fix bugs about consistent_id

* refactor functional::ToConsistent

* refactor GetNdSbp

* Update eager_consistent_op_interpreter.cpp

* Update eager_mirrored_op_interpreter.cpp

* fix error

* fix error

* auto format by CI

* Update nd_sbp.h

* refine identity boxing

* fix sync checkmeta error

* avoid consistent id check in lazy

Co-authored-by: Xinqi Li <lixinqi0703106@163.com>
Co-authored-by: leaves-zwx <kunta0932@gmail.com>
Co-authored-by: Li Xinqi <lixinqi2010@gmail.com>
Co-authored-by: oneflow-ci-bot <69100618+oneflow-ci-bot@users.noreply.github.com>
Co-authored-by: oneflow-ci-bot <ci-bot@oneflow.org>

* add CMAKE_INTERPROCEDURAL_OPTIMIZATION in fast cmake cache (#5970)

* add CMAKE_INTERPROCEDURAL_OPTIMIZATION in fast cmake cache

* skip test targets of re2

Co-authored-by: oneflow-ci-bot <69100618+oneflow-ci-bot@users.noreply.github.com>

* check: fix clang-tidy-diff commands (#5972)

* check: fix clang-tidy-diff commands

* CI: fix step names

Co-authored-by: oneflow-ci-bot <69100618+oneflow-ci-bot@users.noreply.github.com>

* Cpu mpi (#5865)

* cuda base cpu mpi boxing

* cpu_mpi

* fix conflicts

* add cpu mpi unittests

* more checks and unittests

* abstract_consistent_to_consistent_op_expr

* fix compiler complaint

* refactor consistent-to-consistent eager consisitent op interpreter

* fix compiler complaint

* refactor ConsistentToConsistentOpExpr

* lazy interpreter (#5903)

* fix bugs about consistent_id

* more test_consistent_cast unittests

* refactor functional::ToConsistent

* refactor GetNdSbp

* fix compiler complaints

* refactor GetDevice4CurrentProcessCtx

* fix error

Co-authored-by: clackhan <han_binbin@163.com>
Co-authored-by: leaves-zwx <kunta0932@gmail.com>
Co-authored-by: oneflow-ci-bot <69100618+oneflow-ci-bot@users.noreply.github.com>

* fix_bug_test_tensor_str (#5958)

* fix bug int test_tensor_str

* format

* fix comment

* fix bug to(cuda) is unavailable in cpu env

Co-authored-by: oneflow-ci-bot <69100618+oneflow-ci-bot@users.noreply.github.com>

* common/error: fix build error in mac (#5971)

Co-authored-by: oneflow-ci-bot <69100618+oneflow-ci-bot@users.noreply.github.com>

* Prevent running oneflow in forked subprocess (#5976)

* prevent_running_oneflow_in_forked_subprocess

* add line change

* IsFork => IsForkedSubProcess

* auto format by CI

Co-authored-by: oneflow-ci-bot <69100618+oneflow-ci-bot@users.noreply.github.com>
Co-authored-by: oneflow-ci-bot <ci-bot@oneflow.org>

* refine randint

Co-authored-by: Li Xinqi <lixinqi2010@gmail.com>
Co-authored-by: binbinHan <han_binbin@163.com>
Co-authored-by: oneflow-ci-bot <69100618+oneflow-ci-bot@users.noreply.github.com>
Co-authored-by: oneflow-ci-bot <ci-bot@oneflow.org>
Co-authored-by: daquexian <daquexian566@gmail.com>
Co-authored-by: Peihong Liu <mosout@qq.com>
Co-authored-by: Twice <i@twice.moe>
Co-authored-by: ZhangShen <55383772+zhangshen12356@users.noreply.github.com>
Co-authored-by: Tianyu Zhao <guikarist@gmail.com>
Co-authored-by: Luyang <flowingsun007@163.com>
Co-authored-by: Yinggang Wang <wyg19970408@gmail.com>
Co-authored-by: Xinqi Li <lixinqi0703106@163.com>
Co-authored-by: leaves-zwx <kunta0932@gmail.com>
Co-authored-by: Shenghang Tsai <jackalcooper@gmail.com>
Co-authored-by: liufengwei0103 <2472937968@qq.com>

* refine

* refine

* auto format by CI

* refine

* Update functional_api.yaml

* Update functional_api.yaml

* refine the code

* auto format by CI

* refine

* fix ci error

* fix test

* auto format by CI

* fixtest

* refine code

* auto format by CI

* refine code

* auto format by CI

* fix ci fail

* remove redefination api

* fix ci test

* auto format by CI

* fix consistency with torch

* auto format by CI

* unittest fixed

* fix doctest

Co-authored-by: Bowen Chen <bob2420083992@gmail.com>
Co-authored-by: Li Xinqi <lixinqi2010@gmail.com>
Co-authored-by: binbinHan <han_binbin@163.com>
Co-authored-by: oneflow-ci-bot <69100618+oneflow-ci-bot@users.noreply.github.com>
Co-authored-by: oneflow-ci-bot <ci-bot@oneflow.org>
Co-authored-by: daquexian <daquexian566@gmail.com>
Co-authored-by: Peihong Liu <mosout@qq.com>
Co-authored-by: Twice <i@twice.moe>
Co-authored-by: ZhangShen <55383772+zhangshen12356@users.noreply.github.com>
Co-authored-by: Tianyu Zhao <guikarist@gmail.com>
Co-authored-by: Luyang <flowingsun007@163.com>
Co-authored-by: Yinggang Wang <wyg19970408@gmail.com>
Co-authored-by: Xinqi Li <lixinqi0703106@163.com>
Co-authored-by: leaves-zwx <kunta0932@gmail.com>
Co-authored-by: Shenghang Tsai <jackalcooper@gmail.com>
Co-authored-by: liufengwei0103 <2472937968@qq.com>
oneflow-ci-bot added a commit that referenced this pull request Aug 30, 2021
@Flowingsun007 @Kevin-XiongC
@VertexC @lixinqi @clackhan @oneflow-ci-bot @daquexian @mosout @PragmaTwice @zhangshen12356 @wyg1997 @leaves-zwx @jackalcooper @liufengwei0103
Use flow.randint in dataloader (#6086)
d23f3a1 
* add randint

* add

* add doc test

* Update randint_kernel.cu

* Update randint_kernel.cpp

* Update randint_op.cpp

* reconstrcut

* refine the code

* add test

* add test

* add test

* format

* Dev randint refine (#5981)

* disable backward pass consistent tensor meta check. (#5871)

* disable backward pass consistent tensor meta check.

* auto format by CI

Co-authored-by: binbinHan <han_binbin@163.com>
Co-authored-by: oneflow-ci-bot <69100618+oneflow-ci-bot@users.noreply.github.com>
Co-authored-by: oneflow-ci-bot <ci-bot@oneflow.org>

* ddp broadcast params and buffers (#5913)

* ddp broadcast params and buffers

Signed-off-by: daquexian <daquexian566@gmail.com>

* auto format by CI

Co-authored-by: oneflow-ci-bot <69100618+oneflow-ci-bot@users.noreply.github.com>
Co-authored-by: oneflow-ci-bot <ci-bot@oneflow.org>

* add clang tidy target (#5957)

* add clang tidy target

* fix a bug

* refine

* refine

* reformat

Co-authored-by: oneflow-ci-bot <69100618+oneflow-ci-bot@users.noreply.github.com>

* cfg: add move assignment operator for performance (#5962)

Co-authored-by: oneflow-ci-bot <69100618+oneflow-ci-bot@users.noreply.github.com>

* add zhangshen op-test (#5600)

* add some op-test

* fix dims_error in my branch

* Fix the bad backward kernel function by using 'cuda::atomic::Add' (#5614)

* Test `nn.AdaptiveAvgPoolXd` (#5615)

* Fix the bad backward kernel function by using 'cuda::atomic::Add'

* Support the 'NoneType' annotation

* Support objects of 'collections.abc.Iterable' as 'output_size'

* Test with all cases of 'output_size'

* Update adaptive_pool_gpu_kernel.cu

* Skip testing `nn.AdaptiveAvgPool3d` for the current PyTorch

* remove some useless test

* Format TODO

* Add the assertion messages for 'output_size'

* Reformat codes

* Remove raw tests for `flow.negative`

* Remove unnecessary codes and add the assertion messages

* Merge updates for 'generators.py' from master

* Remove unnecessary 'random()'

* Delete the separate test for `AvgPool2d`

* Fix import paths

* Fix import problems

* Remove the PyTorch import

* Denote the annotations for `tile` and `repeat` ops

* Add the test for `nn.AvgPool1d`

* Choose better generators for `nn.MaxPoolXd`

* Randomly choose `dilation` and default values

* auto format by CI

* Test more kwargs for `nn.AvgPoolXd`

* Add tests for `return_indices`

* auto format by CI

Co-authored-by: Tianyu Zhao <guikarist@gmail.com>
Co-authored-by: oneflow-ci-bot <69100618+oneflow-ci-bot@users.noreply.github.com>
Co-authored-by: oneflow-ci-bot <ci-bot@oneflow.org>

* fix wrong names (#5951)

* fix wrong names

* auto format by CI

* refine

* auto format by CI

Co-authored-by: oneflow-ci-bot <ci-bot@oneflow.org>
Co-authored-by: oneflow-ci-bot <69100618+oneflow-ci-bot@users.noreply.github.com>

* Enable more checkers for clang-tidy in CI (#5738)

* CI: enable more checkers for clang-tidy

* .clang-tidy: remove cppcoreguidelines-pro-type-vararg

* CI: remove duplicate checkers

* CI: remove clang-analyzer-alpha.deadcode.*

* .clang-tidy: add performance-*

* oneflow/core/eager: remove unnecessary malloc & free

* .clang-tidy: add clang-analyzer-cplusplus.* to werror

* user_kernel: remove useless move

* quantization_aware_training: fix move return

* .clang-tidy: add google-*

* CI: fix clang tidy command

* CI: fix test

Co-authored-by: oneflow-ci-bot <69100618+oneflow-ci-bot@users.noreply.github.com>

* Feat grad mode classes (#5956)

* feat(no_grad): support no_grad decorator

* feat(AutogradMode): export flow.autograd_mode

* feat(GradMode): export some grad_mode class

* docs(GradMode): export documents

* refine

* docs(GradMode): export document for is_grad_enabled

* auto format by CI

* fix(GradMode): fix single client bug

* fix bug

Co-authored-by: oneflow-ci-bot <69100618+oneflow-ci-bot@users.noreply.github.com>
Co-authored-by: oneflow-ci-bot <ci-bot@oneflow.org>

* extract_consistent_to_consistent_op_expr (#5870)

* abstract_consistent_to_consistent_op_expr

* fix compiler complaint

* refactor consistent-to-consistent eager consisitent op interpreter

* fix compiler complaint

* refactor ConsistentToConsistentOpExpr

* lazy interpreter (#5903)

* fix bugs about consistent_id

* refactor functional::ToConsistent

* refactor GetNdSbp

* Update eager_consistent_op_interpreter.cpp

* Update eager_mirrored_op_interpreter.cpp

* fix error

* fix error

* auto format by CI

* Update nd_sbp.h

* refine identity boxing

* fix sync checkmeta error

* avoid consistent id check in lazy

Co-authored-by: Xinqi Li <lixinqi0703106@163.com>
Co-authored-by: leaves-zwx <kunta0932@gmail.com>
Co-authored-by: Li Xinqi <lixinqi2010@gmail.com>
Co-authored-by: oneflow-ci-bot <69100618+oneflow-ci-bot@users.noreply.github.com>
Co-authored-by: oneflow-ci-bot <ci-bot@oneflow.org>

* add CMAKE_INTERPROCEDURAL_OPTIMIZATION in fast cmake cache (#5970)

* add CMAKE_INTERPROCEDURAL_OPTIMIZATION in fast cmake cache

* skip test targets of re2

Co-authored-by: oneflow-ci-bot <69100618+oneflow-ci-bot@users.noreply.github.com>

* check: fix clang-tidy-diff commands (#5972)

* check: fix clang-tidy-diff commands

* CI: fix step names

Co-authored-by: oneflow-ci-bot <69100618+oneflow-ci-bot@users.noreply.github.com>

* Cpu mpi (#5865)

* cuda base cpu mpi boxing

* cpu_mpi

* fix conflicts

* add cpu mpi unittests

* more checks and unittests

* abstract_consistent_to_consistent_op_expr

* fix compiler complaint

* refactor consistent-to-consistent eager consisitent op interpreter

* fix compiler complaint

* refactor ConsistentToConsistentOpExpr

* lazy interpreter (#5903)

* fix bugs about consistent_id

* more test_consistent_cast unittests

* refactor functional::ToConsistent

* refactor GetNdSbp

* fix compiler complaints

* refactor GetDevice4CurrentProcessCtx

* fix error

Co-authored-by: clackhan <han_binbin@163.com>
Co-authored-by: leaves-zwx <kunta0932@gmail.com>
Co-authored-by: oneflow-ci-bot <69100618+oneflow-ci-bot@users.noreply.github.com>

* fix_bug_test_tensor_str (#5958)

* fix bug int test_tensor_str

* format

* fix comment

* fix bug to(cuda) is unavailable in cpu env

Co-authored-by: oneflow-ci-bot <69100618+oneflow-ci-bot@users.noreply.github.com>

* common/error: fix build error in mac (#5971)

Co-authored-by: oneflow-ci-bot <69100618+oneflow-ci-bot@users.noreply.github.com>

* Prevent running oneflow in forked subprocess (#5976)

* prevent_running_oneflow_in_forked_subprocess

* add line change

* IsFork => IsForkedSubProcess

* auto format by CI

Co-authored-by: oneflow-ci-bot <69100618+oneflow-ci-bot@users.noreply.github.com>
Co-authored-by: oneflow-ci-bot <ci-bot@oneflow.org>

* refine randint

Co-authored-by: Li Xinqi <lixinqi2010@gmail.com>
Co-authored-by: binbinHan <han_binbin@163.com>
Co-authored-by: oneflow-ci-bot <69100618+oneflow-ci-bot@users.noreply.github.com>
Co-authored-by: oneflow-ci-bot <ci-bot@oneflow.org>
Co-authored-by: daquexian <daquexian566@gmail.com>
Co-authored-by: Peihong Liu <mosout@qq.com>
Co-authored-by: Twice <i@twice.moe>
Co-authored-by: ZhangShen <55383772+zhangshen12356@users.noreply.github.com>
Co-authored-by: Tianyu Zhao <guikarist@gmail.com>
Co-authored-by: Luyang <flowingsun007@163.com>
Co-authored-by: Yinggang Wang <wyg19970408@gmail.com>
Co-authored-by: Xinqi Li <lixinqi0703106@163.com>
Co-authored-by: leaves-zwx <kunta0932@gmail.com>
Co-authored-by: Shenghang Tsai <jackalcooper@gmail.com>
Co-authored-by: liufengwei0103 <2472937968@qq.com>

* refine

* refine

* auto format by CI

* refine

* Update functional_api.yaml

* Update functional_api.yaml

* refine the code

* auto format by CI

* refine

* fix ci error

* replace np randint with flow.randint

* add tensor.item tensor.tolist

* add test case

* refine

* auto format by CI

* fix ci fail

* revert change

* auto format by CI

* refine

* auto format by CI

Co-authored-by: Kevin-XiongC <kevin_xiong1997@outlook.com>
Co-authored-by: Bowen Chen <bob2420083992@gmail.com>
Co-authored-by: Li Xinqi <lixinqi2010@gmail.com>
Co-authored-by: binbinHan <han_binbin@163.com>
Co-authored-by: oneflow-ci-bot <69100618+oneflow-ci-bot@users.noreply.github.com>
Co-authored-by: oneflow-ci-bot <ci-bot@oneflow.org>
Co-authored-by: daquexian <daquexian566@gmail.com>
Co-authored-by: Peihong Liu <mosout@qq.com>
Co-authored-by: Twice <i@twice.moe>
Co-authored-by: ZhangShen <55383772+zhangshen12356@users.noreply.github.com>
Co-authored-by: Tianyu Zhao <guikarist@gmail.com>
Co-authored-by: Yinggang Wang <wyg19970408@gmail.com>
Co-authored-by: Xinqi Li <lixinqi0703106@163.com>
Co-authored-by: leaves-zwx <kunta0932@gmail.com>
Co-authored-by: Shenghang Tsai <jackalcooper@gmail.com>
Co-authored-by: liufengwei0103 <2472937968@qq.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@PragmaTwice PragmaTwice PragmaTwice left review comments

@daquexian daquexian daquexian approved these changes

@oneflow-ci-bot oneflow-ci-bot Awaiting requested review from oneflow-ci-bot

Assignees
No one assigned
Labels
automerge build enhancement
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
4 participants
@mosout @PragmaTwice @daquexian @oneflow-ci-bot