Cog base images #1605
Merged
Cog base images #1605
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
andreasjansson
force-pushed
the
cog-base-images
branch
2 times, most recently
from
49a750f
to
9308122
Compare
tempusfrangit
force-pushed
the
cog-base-images
branch
2 times, most recently
from
c9b5661
to
97e62ff
Compare
|
Wahey. A couple of things:
|
Signed-off-by: andreasjansson <andreas@replicate.ai>
Signed-off-by: andreasjansson <andreas@replicate.ai>
Signed-off-by: andreasjansson <andreas@replicate.ai>
And make config_test.go use smaller yaml indentation. My editor replaces spaces with tabs in go files... Signed-off-by: andreasjansson <andreas@replicate.ai>
Update and add the Build Action to build bases. This implmenets the following * Build Action * Docker Layer Cache * Argument to specify docker layer cache within GH action cache * Matrix output to support matrix strategy for building the image * DockerFile cache Key generator (create a cache key based upon the docker file SHA256) * Remove BuildKit Version validation as the output is highly variable the push should only be done from a highly controlled environment (CI) This is all in support of consistent and reproducible builds. We may need to think about the docker file content cache as it may make security updates more challenging. Image push is currently a stub and does not actually push images Signed-off-by: Morgan Fainberg <code@tempusfrang.it> Signed-off-by: andreasjansson <andreas@replicate.ai>
Add improvements to the GH action and use the actual build actions. This aleviates the potential issues with pushing later on. Additionally, setup the scaffolding for reproducible builds (commented out for now). Signed-off-by: andreasjansson <andreas@replicate.ai>
* Separate Tags and Image name in the Matrix * Use APT Cache for reproducible build * Consider adding further caches * Restrict usage of the workflow * Upload to GHCR (an additional workflow will migrate the content to r8.im) * Add scoped cache that includes image-name and tag. * Use proper metadata extraction Signed-off-by: Morgan Fainberg <code@tempusfrang.it> Signed-off-by: andreasjansson <andreas@replicate.ai>
Signed-off-by: andreasjansson <andreas@replicate.ai>
Signed-off-by: andreasjansson <andreas@replicate.ai>
Signed-off-by: andreasjansson <andreas@replicate.ai>
Signed-off-by: andreasjansson <andreas@replicate.ai>
Remove the unsupported base combination Signed-off-by: Morgan Fainberg <code@tempusfrang.it> Signed-off-by: andreasjansson <andreas@replicate.ai>
Signed-off-by: andreasjansson <andreas@replicate.ai>
Signed-off-by: andreasjansson <andreas@replicate.ai>
Signed-off-by: andreasjansson <andreas@replicate.ai>
Signed-off-by: andreasjansson <andreas@replicate.ai>
Signed-off-by: andreasjansson <andreas@replicate.ai>
Signed-off-by: andreasjansson <andreas@replicate.ai>
Signed-off-by: andreasjansson <andreas@replicate.ai>
Signed-off-by: andreasjansson <andreas@replicate.ai>
Signed-off-by: andreasjansson <andreas@replicate.ai>
andreasjansson
force-pushed
the
cog-base-images
branch
from
97e62ff
to
37a1e1f
Compare
I want to give this to the models team for a week to test, since it's quite a large change and we might need to support more configurations of base images. But yes, once we've validated it internally we should make it default.
Good shout. @tempusfrangit mentioned this as well, we should hide that. |
* Torch 2.0.2 is no longer available * Torch 0.1.5 is not available for modern cuda (e.g. > 10) * Torch 2.20.0 was clearly a typo, 2.20 does not exist Signed-off-by: Morgan Fainberg <code@tempusfrang.it>
Signed-off-by: andreasjansson <andreas@replicate.ai>
andreasjansson
force-pushed
the
cog-base-images
branch
from
0163257
to
93c6ee1
Compare
Signed-off-by: andreasjansson <andreas@replicate.ai>
Signed-off-by: andreasjansson <andreas@replicate.ai>
andreasjansson
force-pushed
the
cog-base-images
branch
from
843a0e2
to
3967298
Compare
Signed-off-by: andreasjansson <andreas@replicate.ai>
mattt
approved these changes
Apr 11, 2024
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This PR adds a
--use-cog-base-imageflag tocog {build,push,run,debug}that builds the Cog image from one of our supported base images.Base images can be built using the new
baseimagecommand, which is being deployed by @tempusfrangit on Cloud Build. Inside each base image is:The set of base images that are currently supported have been assembled by looking at metadata of the popular public models on Replicate.com.
When building from a base image, we only install the user's system packages, python packages, and run commands (excluding the packages that are present in the base image). Python packages are not installed with multi-stage build, because of various pyenv complexities -- we can fix that later if we need to.