Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Prevent adding subdir in python/test #5514

Merged
merged 20 commits into from
Jul 16, 2021
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
20 commits
Select commit Hold shift + click to select a range
fcab7b3
prevent adding subdir in python/test
jackalcooper Jul 15, 2021
61fab0a
refine
jackalcooper Jul 15, 2021
def1ee9
add ci
jackalcooper Jul 15, 2021
c9fecf4
refactor
jackalcooper Jul 15, 2021
f7016c4
refine
jackalcooper Jul 15, 2021
6323448
refine
jackalcooper Jul 15, 2021
fcf202d
Merge branch 'master' into prevent_adding_subdir_in_test
oneflow-ci-bot Jul 15, 2021
157e7e3
add more timeout-minutes: 45
jackalcooper Jul 15, 2021
3ae1028
Merge branch 'prevent_adding_subdir_in_test' of https://github.com/On…
jackalcooper Jul 15, 2021
f30665c
Merge branch 'master' into prevent_adding_subdir_in_test
oneflow-ci-bot Jul 15, 2021
b6f42d1
Merge branch 'master' into prevent_adding_subdir_in_test
oneflow-ci-bot Jul 15, 2021
7c6f322
Merge branch 'master' into prevent_adding_subdir_in_test
oneflow-ci-bot Jul 15, 2021
caa97e9
Merge branch 'master' into prevent_adding_subdir_in_test
oneflow-ci-bot Jul 15, 2021
2b09291
Merge branch 'master' into prevent_adding_subdir_in_test
oneflow-ci-bot Jul 15, 2021
9f7c470
Merge branch 'master' into prevent_adding_subdir_in_test
oneflow-ci-bot Jul 16, 2021
54d0b76
Merge branch 'master' into prevent_adding_subdir_in_test
oneflow-ci-bot Jul 16, 2021
9d56620
Merge branch 'master' into prevent_adding_subdir_in_test
oneflow-ci-bot Jul 16, 2021
e488890
Merge branch 'master' into prevent_adding_subdir_in_test
oneflow-ci-bot Jul 16, 2021
cae42a0
Merge branch 'master' into prevent_adding_subdir_in_test
oneflow-ci-bot Jul 16, 2021
0fd49cd
Merge branch 'master' into prevent_adding_subdir_in_test
oneflow-ci-bot Jul 16, 2021
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
33 changes: 31 additions & 2 deletions .github/workflows/test.yml
Original file line number Diff line number Diff line change
Expand Up @@ -91,6 +91,9 @@ jobs:
if: ${{ failure() }}
run: |
exit 1
- name: Check source code (prevent creating files at wrong places)
run: |
python3 tools/check_src.py

static_analysis_with_clang:
name: Static analysis with clang
Expand Down Expand Up @@ -133,7 +136,7 @@ jobs:
-DBUILD_TESTING=ON \
-DCMAKE_EXPORT_COMPILE_COMMANDS=ON
cd ..
./run-clang-tidy.py -clang-tidy-binary ./clang-tidy -p build -quiet
./run-clang-tidy.py -clang-tidy-binary ./clang-tidy -p build -quiet

wait_for_gpu_slot:
name: Wait for GPU slots
Expand Down Expand Up @@ -251,16 +254,17 @@ jobs:
ref: ${{ github.event.pull_request.head.sha }}
if: env.is_built != '1'
- name: Build OneFlow
timeout-minutes: 45
if: env.is_built != '1'
uses: ./.github/actions/whl
timeout-minutes: 45
with:
tmp_dir: ${ci_tmp_dir}
extra_flags: ${{ matrix.extra_flags }}
cuda_version: ${{ matrix.cuda_version }}
extra_docker_args: $extra_docker_args
python_version: ${{ matrix.python_version }}
- name: Custom Op test (run by oneflow build docker)
timeout-minutes: 45
if: matrix.test_suite == 'cpu' && env.is_built != '1'
run: |
set -x
Expand Down Expand Up @@ -423,6 +427,7 @@ jobs:
run: |
bash docker/ci/test/build.sh
- name: Exe test
timeout-minutes: 45
if: contains(fromJson('["cuda", "cpu"]'), matrix.test_suite)
run: |
set -x
Expand All @@ -431,25 +436,29 @@ jobs:
${image_name} \
${bin_dir}/oneflow_testexe
- name: Build documentation
timeout-minutes: 45
if: matrix.test_suite == 'cuda'
run: |
set -x
docker run ${{ env.extra_docker_args }} ${{ env.pip_cache_docker_args }} \
${image_name} \
bash -c "python3 -m pip config set global.index-url ${{ env.pip_index_mirror }} && bash ci/test/try_install.sh && bash ci/test/build_docs.sh"
- name: Op test (distributed, 1st try)
timeout-minutes: 45
if: matrix.test_suite == 'cuda'
continue-on-error: true
id: distributed_try_1
run: |
python3 ci/test/distributed_run.py --bash_script=ci/test/2node_op_test.sh --custom_img_tag=${{ env.image_name }} --oneflow_wheel_path=${{ env.wheelhouse_dir }} --oneflow_wheel_python_version=3.6
- name: Op test (distributed, 2nd try)
timeout-minutes: 45
if: matrix.test_suite == 'cuda' && steps.distributed_try_1.outcome=='failure'
continue-on-error: true
id: distributed_try_2
run: |
python3 ci/test/distributed_run.py --bash_script=ci/test/2node_op_test.sh --custom_img_tag=${{ env.image_name }} --oneflow_wheel_path=${{ env.wheelhouse_dir }} --oneflow_wheel_python_version=3.6
- name: Op test (distributed, 3rd try)
timeout-minutes: 45
if: matrix.test_suite == 'cuda' && steps.distributed_try_2.outcome=='failure'
continue-on-error: false
id: distributed_try_3
Expand All @@ -472,6 +481,7 @@ jobs:
${{ env.extra_docker_args }} ${{ env.pip_cache_docker_args }} \
${image_name} bash ci/test/print_stack_from_core.sh python3 distributed-tmp
- name: Doctest
timeout-minutes: 45
if: matrix.test_suite == 'cuda'
run: |
set -x
Expand All @@ -480,6 +490,7 @@ jobs:
${image_name} \
bash -c "python3 -m pip config set global.index-url ${{ env.pip_index_mirror }} && bash ci/test/try_install.sh && bash ci/test/doctest.sh"
- name: Dry run test (run without runtime)
timeout-minutes: 45
if: matrix.test_suite == 'cuda'
run: |
set -x
Expand Down Expand Up @@ -508,14 +519,25 @@ jobs:
wget ${{ env.image_url }}
docker load -i $(basename "${{ env.image_url }}")
- name: Module API test
timeout-minutes: 45
if: matrix.test_suite == 'cuda_new_interface'
run: |
docker run \
${{ env.extra_docker_args }} ${{ env.pip_cache_docker_args }} \
-e ONEFLOW_TEST_DIR=$PWD/oneflow/python/test/modules \
${{ env.image_tag }} \
bash -c "python3 -m pip config set global.index-url ${{ env.pip_index_mirror }} && bash ci/test/try_install.sh && bash ci/test/generic_test.sh"
- name: Dataloader API test
timeout-minutes: 45
if: matrix.test_suite == 'cuda_new_interface'
run: |
docker run \
${{ env.extra_docker_args }} ${{ env.pip_cache_docker_args }} \
-e ONEFLOW_TEST_DIR=$PWD/oneflow/python/test/dataloader \
${{ env.image_tag }} \
bash -c "python3 -m pip config set global.index-url ${{ env.pip_index_mirror }} && bash ci/test/try_install.sh && bash ci/test/generic_test.sh"
- name: Tensor API test
timeout-minutes: 45
if: matrix.test_suite == 'cuda_new_interface'
run: |
docker run \
Expand All @@ -541,6 +563,7 @@ jobs:
${image_name} \
bash -c "python3 -m pip config set global.index-url ${{ env.pip_index_mirror }} && bash ci/test/try_install.sh && bash ci/test/1node_op_test.sh"
- name: Model test
timeout-minutes: 45
if: matrix.test_suite == 'cpu' || matrix.test_suite == 'cuda'
run: |
set -x
Expand All @@ -549,6 +572,7 @@ jobs:
${image_name} \
bash -c "python3 -m pip config set global.index-url ${{ env.pip_index_mirror }} && bash ci/test/try_install.sh && bash ci/test/1node_model_test.sh"
- name: Model serve test
timeout-minutes: 45
id: model_serve_test
if: matrix.test_suite == 'cuda'
run: |
Expand All @@ -564,32 +588,37 @@ jobs:
docker run ${{ env.extra_docker_args }} ${{ env.pip_cache_docker_args }} \
${image_name} bash ci/test/print_stack_from_core.sh python3 serving-tmp
- name: Benchmark (mainly for backward compatibility)
timeout-minutes: 45
if: matrix.test_suite == 'cuda'
run: |
set -x
docker run ${{ env.extra_docker_args }} ${{ env.pip_cache_docker_args }} \
${image_name} \
bash -c "python3 -m pip config set global.index-url ${{ env.pip_index_mirror }} && bash ci/test/try_install.sh && bash ci/test/1node_benchmark_test.sh"
- name: Benchmark FP16 (mainly for backward compatibility)
timeout-minutes: 45
if: matrix.test_suite == 'cuda'
run: |
set -x
docker run ${{ env.extra_docker_args }} ${{ env.pip_cache_docker_args }} \
${image_name} \
bash -c "python3 -m pip config set global.index-url ${{ env.pip_index_mirror }} && bash ci/test/try_install.sh && bash ci/test/1node_benchmark_test_fp16.sh"
- name: XLA Test
timeout-minutes: 45
if: contains(fromJson('["xla", "xla_cpu"]'), matrix.test_suite) && env.is_built != '1'
run: |
set -x
docker run ${{ env.extra_docker_args }} ${{ env.pip_cache_docker_args }} \
${image_name} \
bash -c "python3 -m pip config set global.index-url ${{ env.pip_index_mirror }} && bash ci/test/try_install.sh && bash ci/test/test_xla.sh"
- name: Query system status
timeout-minutes: 45
if: ${{ failure() }}
run: |
nvidia-smi
docker ps
- name: Remove container
timeout-minutes: 45
if: always()
run: |
docker rm -f ${container_name} || true
40 changes: 0 additions & 40 deletions oneflow/python/test/demos/eager_nccl_all_reduce_demo.py

This file was deleted.

26 changes: 0 additions & 26 deletions oneflow/python/test/deprecated/constant_scalar.py

This file was deleted.

78 changes: 0 additions & 78 deletions oneflow/python/test/deprecated/enable_all_reduce_group.py

This file was deleted.

Loading