Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Update the Github Workflow for uploading conda packages #795

Merged
merged 4 commits into from
Sep 18, 2020

Conversation

nudles
Copy link
Member

@nudles nudles commented Sep 15, 2020

The anaconda upload token is stored as a secret on github.

The anaconda upload token is a secret on github (nudles).
If the secret is not available, no uploading.
Note: github ci cannot access the secret directly within the `if` check;
you have to assign it to a env var first and then do if check;
better do the upload on push (not on pull request).
Add the CI to run on MacOS, including building, testing pysinga and
uploading the package to anaconda if the token is available.

Fix the permission issue conda/conda#7267
and the sdk issue.
The sdk is fixed to the latest version (in conda_build_config.yaml).
The python is fixed to 3.6 to be compatible with colab.
@codecov
Copy link

codecov bot commented Sep 15, 2020

Codecov Report

Merging #795 into dev will decrease coverage by 0.01%.
The diff coverage is n/a.

Impacted file tree graph

@@            Coverage Diff             @@
##              dev     #795      +/-   ##
==========================================
- Coverage   70.06%   70.05%   -0.02%     
==========================================
  Files         100      100              
  Lines       11568    11573       +5     
==========================================
+ Hits         8105     8107       +2     
- Misses       3463     3466       +3     
Flag Coverage Δ
#singa-cpp 65.09% <ø> (-0.03%) ⬇️
#singa-python 73.57% <ø> (ø)

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files Coverage Δ
include/singa/core/tensor.h 73.21% <0.00%> (-8.79%) ⬇️
src/core/tensor/tensor.cc 76.54% <0.00%> (+0.34%) ⬆️

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 0d79d52...7b5a0ab. Read the comment docs.

@moazreyad moazreyad mentioned this pull request Sep 15, 2020
@moazreyad
Copy link
Contributor

I think it will be better to create a new separate workflow for publishing conda packages. In the conda package workflow, the build should not use ENABLE_TEST unless it is building a dev package for developers not a normal package for users. This will make the built code faster because it can use compiler optimizations which are not available in test mode and smaller because it does not contain the debug information.

The packaging workflow (for conda, pypi, debian, docker, etc) should not contain the code coverage steps because (1) the code is usually not built with test enabled. (2) there is no need to upload a new coverage report for the same python test cases unless there are special test cases for the the package process itself.

The packaging workflow may not run with each pull request or merge on the dev branch, but they can run periodically every week for example or only when merging with the master branch.

@nudles
Copy link
Member Author

nudles commented Sep 17, 2020

I think it will be better to create a new separate workflow for publishing conda packages. In the conda package workflow, the build should not use ENABLE_TEST unless it is building a dev package for developers not a normal package for users. This will make the built code faster because it can use compiler optimizations which are not available in test mode and smaller because it does not contain the debug information.

The packaging workflow (for conda, pypi, debian, docker, etc) should not contain the code coverage steps because (1) the code is usually not built with test enabled. (2) there is no need to upload a new coverage report for the same python test cases unless there are special test cases for the the package process itself.

The packaging workflow may not run with each pull request or merge on the dev branch, but they can run periodically every week for example or only when merging with the master branch.

Currently,

  1. CPP test is done by native-macos and native-ubuntu for every PR, for which ENABLE_TEST is on.
  2. Python test is done by conda for every PR, for which ENABLE_TEST is off.
  3. conda package uploading is disabled on apache/singa because we cannot set the secret key for the uploading token.

I am fine to disable code coverage for 2 and 3, i.e., conda.yaml.
How should we disable it?

@chrishkchris chrishkchris merged commit 814e2f7 into apache:dev Sep 18, 2020
@moazreyad
Copy link
Contributor

conda package uploading is disabled on apache/singa because we cannot set the secret key for the uploading token

Why we cannot set the secret key? Can we ask INFRA to add the key as discussed in #790?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants