Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Introduce packaging tests for Docker #46599

Open
wants to merge 3 commits into
base: master
from

Conversation

@pugnascotia
Copy link
Contributor

commented Sep 11, 2019

Add packaging tests for our Docker images, similar to what we have for RPMs or Debian packages. This works by running a container and probing it e.g. via docker exec. Test can also be run in Vagrant, by exporting the Docker images to disk and loading them again in VMs.

Notes:

  • Docker is installed via Vagrantfile in a selection of boxes. We could change the boxes built by Infra if we felt it was worthwhile, I wasn't sure.
  • The Docker images are only tested with the bundled JDK. It seemed counter to the spirit of using containers to run one using a bundled JDK, though there's nothing stopped a user doing that.
Introduce packaging tests for Docker
Add packaging tests for our Docker images, similar to what we have for
RPMs or Debian packages. This works by running a container and probing
it e.g. via `docker exec`. Test can also be run in Vagrant, by exporting
the Docker images to disk and loading them again in VMs. Docker is
installed via `Vagrantfile` in a selection of boxes.
@elasticmachine

This comment has been minimized.

Copy link
Collaborator

commented Sep 11, 2019

@dliappis
Copy link
Contributor

left a comment

Great work this is really needed after integrating the Docker parts from the old separate repo!

I took an initial pass and left some comments.

Vagrantfile Outdated
@@ -183,6 +183,35 @@ def deb_common(config, name, extra: '')
install_command: 'apt-get install -y',
extra: extra_with_lintian
)
deb_docker(config)

This comment has been minimized.

Copy link
@dliappis

dliappis Sep 12, 2019

Contributor

As discussed privately, there's not much value running this on all distros (and not all support Docker like centos-6) so better be specific about where we call this.

This comment has been minimized.

Copy link
@atorok

atorok Sep 12, 2019

Contributor

I think it would be better to bake this into the image rather than install on-demand.
The CI packaging images already have docker ( at least the ones that support it ),
so we would need to add it to vagrant images only.

This comment has been minimized.

Copy link
@pugnascotia

pugnascotia Sep 12, 2019

Author Contributor

I'll sync with Infra about this, and remove it from Vagrantfile once the boxes include Docker.

Vagrantfile Outdated
@@ -193,6 +222,26 @@ def rpm_common(config, name)
update_tracking_file: '/var/cache/yum/last_update',
install_command: 'yum install -y'
)
rpm_docker(config)

This comment has been minimized.

Copy link
@dliappis

dliappis Sep 12, 2019

Contributor

Same comment as above, especially since centos-7/oel-6 won't support this.

@atorok
Copy link
Contributor

left a comment

Thanks for taking this on!
Unfortunately we don't have Pr checks for running these directly on jenkins workers, so it would be nice to test this before merging to make sure the CI jobs don't break.

Vagrantfile Outdated
@@ -183,6 +183,35 @@ def deb_common(config, name, extra: '')
install_command: 'apt-get install -y',
extra: extra_with_lintian
)
deb_docker(config)

This comment has been minimized.

Copy link
@atorok

atorok Sep 12, 2019

Contributor

I think it would be better to bake this into the image rather than install on-demand.
The CI packaging images already have docker ( at least the ones that support it ),
so we would need to add it to vagrant images only.

distribution/docker/build.gradle Outdated Show resolved Hide resolved
@@ -0,0 +1,2 @@
// This file is intentionally blank. All configuration of the
// export is done in the parent project.

This comment has been minimized.

Copy link
@atorok

atorok Sep 12, 2019

Contributor

Why do we need a project for the extraction ? Could it just be a task on parent ?

This comment has been minimized.

Copy link
@pugnascotia

pugnascotia Sep 12, 2019

Author Contributor

It's to do with DistributionDownloadPlugin - as far as I can tell, it's written to create depdencies between ES distributions and the default config on a project. I'm new to Gradle, but it looks like it doesn't depend on a task because the plugin needs to be able to locate the built archives. I'd be very happy to be pointed at better ways of doing this. I had to spend a while deciphering what was going on between DistroTestPlugin and DistributionDownloadPlugin.

This comment has been minimized.

Copy link
@pugnascotia

pugnascotia Sep 12, 2019

Author Contributor

Also, the same pattern is used in :distribution:packages.

This comment has been minimized.

Copy link
@atorok

atorok Sep 12, 2019

Contributor

I see, it's a bit strange that this we have projects for export only here, but I'll defer to @rjernst

@pugnascotia

This comment has been minimized.

Copy link
Contributor Author

commented Sep 18, 2019

@elasticmachine run elasticsearch-ci/1

@pugnascotia

This comment has been minimized.

Copy link
Contributor Author

commented Sep 18, 2019

@elasticmachine run elasticsearch-ci/packaging

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
4 participants
You can’t perform that action at this time.