Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We鈥檒l occasionally send you account related emails.

Already on GitHub? Sign in to your account

Update and optimize meta tester and inflator images #3958

Merged
merged 1 commit into from Dec 18, 2023

Conversation

HebaruSan
Copy link
Member

@HebaruSan HebaruSan commented Dec 16, 2023

Motivations

  • After Update to Python 3.11聽NetKAN-Infra#322, our Docker images are updated to Python 3.11, except for the meta tester. We don't want to fall too far behind the current versions of Python.
  • The meta tester contains a lot of stuff it doesn't need, weighing in at a hefty 1.23GB:
    • C/C++ compilers and libraries
    • Mono compilers
    • Mono libraries we don't need
    • Python build tools
    • Flask and gunicorn
    • APT and pip download caches
    • Old logs
    • Etc.
REPOSITORY         TAG       IMAGE ID       CREATED        SIZE
kspckan/metadata   latest    df95b1b2804f   20 hours ago   1.23GB
kspckan/inflator   latest    165d396574f3   20 hours ago   792MB

Changes

Now instead of using mono:latest, which is an old OS that installs Python 3.7, we start with the much more current ubuntu:latest and install the latest Mono and Python. This gives the following software versions as of this writing:

Ubuntu 22.04.3 LTS
git 2.40.1
Python 3.10.12
Mono 6.12.0.200

These will be updated to the current versions for ubuntu:latest each time we build the image. Using a newer base OS also allows us to retire the certificate hackery from #3457, since the expired certificate has been removed from the distro. The image now includes only the files it needs at runtime (I built it and confirmed it can inflate netkans from GitHub and SpaceDock, create fake instances, and install mods) and is much smaller:

REPOSITORY            TAG       IMAGE ID       CREATED          SIZE
meta_tester_testing   latest    cecc228c5517   41 minutes ago   364MB
inflator_testing      latest    7f20aaa5f3cc   27 seconds ago   200MB

This might allow it to load faster when we use it on GitHub. 馃

The inflator also now excludes the Mono build tools and is a fraction of its former size.

To make the inflator's smoke test work without installing curl in the image, netkan.exe can now accept a URL to a netkan as its command line parameter.

@HebaruSan HebaruSan added Enhancement Netkan Issues affecting the netkan data Infrastructure Issues affecting everything around CKAN (the GitHub repos, build process, CI, ...) Metadata Issues specific for our metadata labels Dec 16, 2023
@HebaruSan HebaruSan force-pushed the fix/metatester-updates branch 2 times, most recently from 482bed3 to 7fd8310 Compare December 16, 2023 21:45
@HebaruSan HebaruSan changed the title Update and optimize meta tester image Update and optimize meta tester and inflator images Dec 16, 2023
@HebaruSan HebaruSan force-pushed the fix/metatester-updates branch 3 times, most recently from dc28347 to 17ba867 Compare December 16, 2023 22:19
Copy link
Member

@techman83 techman83 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks awesome, thanks for spending some time on this! There is probably a few more opportunities to shave more off. It might be worth combining all the apt pieces into one run action. It might not be significant, docker is a lot more efficient by default than the earlier days.

@HebaruSan HebaruSan merged commit 9b7dd22 into KSP-CKAN:master Dec 18, 2023
8 checks passed
@HebaruSan HebaruSan deleted the fix/metatester-updates branch December 18, 2023 03:09
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Enhancement Infrastructure Issues affecting everything around CKAN (the GitHub repos, build process, CI, ...) Metadata Issues specific for our metadata Netkan Issues affecting the netkan data
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

2 participants