Skip to content

Add docs to show how to use custom images #1323

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
merged 6 commits into from
Jun 28, 2023
Merged

Add docs to show how to use custom images #1323

merged 6 commits into from
Jun 28, 2023

Conversation

@mrodm
Copy link
Contributor

@mrodm mrodm commented Jun 27, 2023

This PR includes documentation to set overrides for some docker images. Currently, the docker images that can be overwritten are:

  • Elasticsearch
  • Kibana
  • Elastic Agent

This overwrites can be achieved in two different ways:

  • Using Environment Variables
  • Using the configuration file (~/.elastic-package/config.yml)

Link to the new doc has been added to the description of elastic-package stack up.

How to test locally

As an example, it is going to be set an overwrite for Elastic Agent (docs link):

# Go to the elastic-agent repository
cd /path/to/elastic-agent
git checkout 8.8
DEV=true SNAPSHOT=true EXTERNAL=true PLATFORMS=linux/amd64 PACKAGES=docker  mage -v package
docker tag docker.elastic.co/beats/elastic-agent-complete:8.8.2-SNAPSHOT docker.elastic.co/beats/elastic-agent-complete:8.8.2-SNAPSHOT-test

git checkout v8.8.1
DEV=true SNAPSHOT=true EXTERNAL=true PLATFORMS=linux/amd64 PACKAGES=docker  mage -v package
docker tag docker.elastic.co/beats/elastic-agent-complete:8.8.1-SNAPSHOT docker.elastic.co/beats/elastic-agent-complete:8.8.1-SNAPSHOT-test

As this image is used by both Elastic Agent and Fleet Server services, docker ps will show two containers running the custom image.

  • Using environment variables:

    export ELASTIC_AGENT_IMAGE_REF_OVERRIDE=docker.elastic.co/beats/elastic-agent-complete:8.8.2-SNAPSHOT-test
elastic-package stack up -v --version 8.8.2-SNAPSHOT
docker ps --format "{{.Image}}"

  • Using configuration file

    • Configuration file should be like: 
      stack:
    image_ref_overrides:
      8.8.2-SNAPSHOT:
        elastic-agent: "docker.elastic.co/beats/elastic-agent-complete:8.8.2-SNAPSHOT-test"
profile:
    current: default
    • steps to follow 
      unset ELASTIC_AGENT_IMAGE_REF_OVERRIDE
elastic-package stack up -v -d --version 8.8.2-SNAPSHOT
docker ps --format "{{.Image}}"

@mrodm mrodm requested a review from a team June 27, 2023 13:59
@mrodm mrodm self-assigned this Jun 27, 2023
@pierrehilbert pierrehilbert added the Team:Elastic-Agent Label for the Agent team label Jun 27, 2023
@elasticmachine
Copy link
Collaborator

Pinging @elastic/elastic-agent (Team:Elastic-Agent)

mrodm
mrodm commented Jun 27, 2023
View reviewed changes
cmd/stack.go Outdated
Be aware that a common issue while trying to boot up the stack is that your Docker environments settings are too low in terms of memory threshold.
To expose local packages in the Package Registry, build them first and boot up the stack from inside of the Git repository containing the package (e.g. elastic/integrations). They will be copied to the development stack (~/.elastic-package/stack/development) and used to build a custom Docker image of the Package Registry.
For Elastic stack versions prior to 8.7, to expose local packages in the Package Registry, build them first and boot up the stack from inside of the Git repository containing the package (e.g. elastic/integrations). They will be copied to the development stack (~/.elastic-package/stack/development) and used to build a custom Docker image of the Package Registry.
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What about adding here this note to expose local packages?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

From 8.7 the same instructions apply to expose local packages in the Package Registry, isn't it?

Would you like to mean here that from 8.7 this is not needed to test packages?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Right, same instructions apply to expose local packages in the Elastic Package Registry.

I've rephrased this paragraph to indicate that this is not needed to run the tests, starting with Elastic stack >= 8.7.0

@mrodm
Copy link
Contributor Author

mrodm commented Jun 27, 2023

/test

jsoriano
jsoriano reviewed Jun 27, 2023
View reviewed changes
Copy link
Member

@jsoriano jsoriano left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice, it is great to have documentation about this feature.

cmd/stack.go Outdated
Be aware that a common issue while trying to boot up the stack is that your Docker environments settings are too low in terms of memory threshold.
To expose local packages in the Package Registry, build them first and boot up the stack from inside of the Git repository containing the package (e.g. elastic/integrations). They will be copied to the development stack (~/.elastic-package/stack/development) and used to build a custom Docker image of the Package Registry.
For Elastic stack versions prior to 8.7, to expose local packages in the Package Registry, build them first and boot up the stack from inside of the Git repository containing the package (e.g. elastic/integrations). They will be copied to the development stack (~/.elastic-package/stack/development) and used to build a custom Docker image of the Package Registry.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

From 8.7 the same instructions apply to expose local packages in the Package Registry, isn't it?

Would you like to mean here that from 8.7 this is not needed to test packages?

mrodm and others added 3 commits June 27, 2023 22:33
@mrodm @jsoriano
Apply suggestions from code review
7da737a 
Co-authored-by: Jaime Soriano Pastor <jaime.soriano@elastic.co>
@mrodm
Update readme
5896673
@mrodm
Rephrase README
2a58163
jsoriano
jsoriano approved these changes Jun 28, 2023
View reviewed changes
Copy link
Member

@jsoriano jsoriano left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🚀

@elasticmachine
Copy link
Collaborator

💚 Build Succeeded

History

cc @mrodm

@mrodm mrodm merged commit 57d0c6f into elastic:main Jun 28, 2023
@mrodm mrodm deleted the add_docs_custom_images branch June 28, 2023 08:42
@mrodm mrodm mentioned this pull request Aug 30, 2023
Open
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@jsoriano jsoriano jsoriano approved these changes

Assignees

@mrodm mrodm

Labels

Team:Elastic-Agent Label for the Agent team

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

4 participants

@mrodm @elasticmachine @jsoriano @pierrehilbert