You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
We are now managing our own builds of TensorFlow and Alpine Linux version of precompiled Python pacakges.
Let's host them on a private PyPI repository so that we could simplify docker-build steps here.
For instance, we could run a separate script to build those packages using containers and automatically upload them to the private PyPI repository. Then downloading/installing those packages inside the Dockerfiles will become much simpler than using multi-staged builds. (The big shortcoming of multi-staged builds is that it is difficult to control caches and intermediate image name/tags...)
Previously it was cumbersome and resource-consuming to host a private PyPI repository because we had to run a dedicated server. But now, there is a tool called s3pypi which allows us to build and host a PyPI repository with only S3 static website hosting + CloudFront.
Suggested repository URLs:
https://pypi.backend.ai/kernels/alpine/3.8/
https://pypi.backend.ai/kernels/ubuntu/16.04/
https://pypi.backend.ai/kernels/ubuntu/18.04/
This could eliminate use of our "build.py" script and complicated build chains.
https://pypi.backend.ai/dist/ubuntu/16.04/ or something similar.
In this case the repository is for distributing Backend.AI itself.
Specifically, we could also make a secret repository for enterprise customers.
The text was updated successfully, but these errors were encountered:
We are now managing our own builds of TensorFlow and Alpine Linux version of precompiled Python pacakges.
Let's host them on a private PyPI repository so that we could simplify docker-build steps here.
For instance, we could run a separate script to build those packages using containers and automatically upload them to the private PyPI repository. Then downloading/installing those packages inside the Dockerfiles will become much simpler than using multi-staged builds. (The big shortcoming of multi-staged builds is that it is difficult to control caches and intermediate image name/tags...)
Previously it was cumbersome and resource-consuming to host a private PyPI repository because we had to run a dedicated server. But now, there is a tool called s3pypi which allows us to build and host a PyPI repository with only S3 static website hosting + CloudFront.
Suggested repository URLs:
https://pypi.backend.ai/kernels/alpine/3.8/
https://pypi.backend.ai/kernels/ubuntu/16.04/
https://pypi.backend.ai/kernels/ubuntu/18.04/
This could eliminate use of our "build.py" script and complicated build chains.
Also, @tlqaksqhr's https://github.com/lablup/backend.ai-packages could be hosted in this way as well, probably via:
https://pypi.backend.ai/dist/ubuntu/16.04/
or something similar.In this case the repository is for distributing Backend.AI itself.
Specifically, we could also make a secret repository for enterprise customers.
The text was updated successfully, but these errors were encountered: