Enable package compatibility checks with older Elastic stacks #740

mtojek · 2021-02-23T12:11:08Z

What does this PR do?

This PR enables package compatibility checks with older Elastic stacks. A package will be installed (not tested) in Kibana.

Checklist

~~- [ ] I have reviewed [tips for building integrations]~~(https://github.com/elastic/integrations/blob/master/docs/tips_for_building_integrations.md) and this pull request is aligned with them.
~~- [ ] I have verified that all data streams collect metrics or logs.~~
~~- [ ] I have added an entry to my package's changelog.yml file.~~

How to test this PR locally

CI will verify it.

Related issues

Fixes Enable backward compatibility tests #537

mtojek · 2021-02-23T12:11:21Z

Blocked by: elastic/beats#24163

elasticmachine · 2021-02-23T12:25:23Z

💚 Build Succeeded

the below badges are clickable and redirect to their specific view in the CI or DOCS

Expand to view the summary

Build stats

Build Cause: Pull request #740 updated
Start Time: 2021-03-01T14:01:24.218+0000
Duration: 38 min 43 sec
Commit: ed26c02

Test stats 🧪

Test	Results
Failed	0
Passed	1723
Skipped	3
Total	1726

Trends 🧪

mtojek · 2021-02-25T13:08:16Z

I found a weird behavior when used -SNAPSHOT for constraint checks:

../../build/elastic-package install -c kibana.version=7.11.0-SNAPSHOT -v
— Check compatibility with 7.11.0-SNAPSHOT
<1s
[2021-02-25T12:40:26.436Z] + ../../build/elastic-package install -c kibana.version=7.11.0-SNAPSHOT -v
[2021-02-25T12:40:26.436Z] 2021/02/25 12:40:26 DEBUG Enable verbose logging
[2021-02-25T12:40:26.436Z] Check conditions for package
[2021-02-25T12:40:26.436Z] 2021/02/25 12:40:26 DEBUG Verify Kibana version (constraint: ^7.9.0, requirement: 7.11.0-SNAPSHOT)
[2021-02-25T12:40:26.436Z] Error: checking conditions failed: Kibana constraint unsatisfied: [0] 7.11.0-SNAPSHOT is a prerelease version and the constraint is only looking for release versions

I will try with releases, e.g. 7.11.0, 7.10.0

mtojek · 2021-02-25T13:24:45Z

Ok, without SNAPSHOT tags, the elastic-package properly considers the version as valid/compatible. It fails because of #745 (need to merge this one first).

mtojek · 2021-02-25T15:24:29Z

I think I have to improve the elastic-package, as it supports only released version:

docker pull docker.elastic.co/elasticsearch/elasticsearch:7.11.0 - works

docker pull docker.elastic.co/elasticsearch/elasticsearch:7.12.0 - doesn't work, as 7.12.0 is not released

mtojek · 2021-02-25T15:28:31Z

@ycombinator it's not ready for merge yet (-SNAPSHOT case), but feel free to comment if you want to.

EDIT:

I think I will fix the condition in a follow up PR. This one already polishes a couple packages in terms of compatibility.

ycombinator · 2021-02-25T20:32:26Z

.ci/Jenkinsfile

+  try {
+    dir("${BASE_DIR}/packages/${integrationName}") {
+      sh(label: "Boot up the Elastic stack (${stackVersion})", script: "../../build/elastic-package stack up -d -v --version ${stackVersion}")
+      sh(label: "Install integration: ${integrationName}", script: '''


So it seems we are calling a package as compatible if we are able to install it. Is this sufficient or do we also want to run tests for the package against the previous stack versions?

No, I prefer to stick to installation as testing will elongate test execution much (+20minutes).

Understood. Does installing a package result in any testing? In other words, what are the reasons this step would fail in CI?

e.g. installation can check if all Kibana objects can be installed (correct migrationVersion field).

ycombinator · 2021-02-25T20:33:40Z

packages/microsoft/manifest.yml

@@ -11,7 +11,7 @@ release: experimental
 license: basic
 type: integration
 conditions:
-  kibana.version: '^7.9.0'
+  kibana.version: '^7.11.0'


I didn't understand why we need these kibana.version bumps here and in other packages in this PR?

It's because of the fact that these packages are not compatible with older Kibana stacks.

I see. Is there a reason these changes are part of this PR? In other words, do we need these changes anyway even without this PR? Did the compatibility checks introduced in this PR catch that these packages needed kibana version bumps?

Did the compatibility checks introduced in this PR catch that these packages needed kibana version bumps?

Yes, exactly this. Otherwise the CI for this PR will be red.

ycombinator

LGTM. This is a nice addition!

…c#740) * Start stack two more times * Check package compatibility with older stacks * Correct versions * nit * Fix * Fix: ignore prerelease * Adjust versions * Update release numbers * Fix: shellinit * Fix * Fix * Fix * Fix compatibility with Kibana

mtojek added 2 commits February 23, 2021 12:29

Start stack two more times

948cde2

Check package compatibility with older stacks

e4c2fff

mtojek self-assigned this Feb 23, 2021

mtojek added 2 commits February 23, 2021 13:11

Correct versions

74d6c14

nit

4d39032

This was referenced Feb 23, 2021

Enable backward compatibility tests #537

Closed

Elastic Agent docker: new revisions are not getting released elastic/beats#24198

Closed

mtojek added 2 commits February 25, 2021 13:31

Fix

dec50ac

Fix: ignore prerelease

af7f455

mtojek added 2 commits February 25, 2021 16:00

Merge branch 'master' into 537-enable-support-matrix

bc7167f

Adjust versions

0020bb9

Update release numbers

0bc4a9e

mtojek requested a review from ycombinator February 25, 2021 15:27

mtojek added 5 commits February 25, 2021 17:06

Fix: shellinit

ae739c4

Fix

44c40d3

Fix

c45ee22

Fix

074b26e

Fix compatibility with Kibana

681fcf1

mtojek marked this pull request as ready for review February 25, 2021 18:02

ycombinator reviewed Feb 25, 2021

View reviewed changes

mtojek added 2 commits March 1, 2021 11:17

Merge branch 'master' into 537-enable-support-matrix

f813d12

Merge branch 'master' into 537-enable-support-matrix

df5654e

mtojek requested a review from ycombinator March 1, 2021 13:56

Merge branch 'master' into 537-enable-support-matrix

ed26c02

ycombinator approved these changes Mar 1, 2021

View reviewed changes

mtojek merged commit 06284f6 into elastic:master Mar 1, 2021

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Enable package compatibility checks with older Elastic stacks #740

Enable package compatibility checks with older Elastic stacks #740

mtojek commented Feb 23, 2021

mtojek commented Feb 23, 2021

elasticmachine commented Feb 23, 2021 •

edited

Build stats

Test stats 🧪

Trends 🧪

mtojek commented Feb 25, 2021

mtojek commented Feb 25, 2021

mtojek commented Feb 25, 2021 •

edited

mtojek commented Feb 25, 2021 •

edited

ycombinator Feb 25, 2021

mtojek Mar 1, 2021

ycombinator Mar 1, 2021

mtojek Mar 1, 2021

ycombinator Feb 25, 2021

mtojek Mar 1, 2021

ycombinator Mar 1, 2021

mtojek Mar 1, 2021 •

edited

ycombinator left a comment

Enable package compatibility checks with older Elastic stacks #740

Enable package compatibility checks with older Elastic stacks #740

Conversation

mtojek commented Feb 23, 2021

What does this PR do?

Checklist

How to test this PR locally

Related issues

mtojek commented Feb 23, 2021

elasticmachine commented Feb 23, 2021 • edited

💚 Build Succeeded

Build stats

Test stats 🧪

Trends 🧪

mtojek commented Feb 25, 2021

mtojek commented Feb 25, 2021

mtojek commented Feb 25, 2021 • edited

mtojek commented Feb 25, 2021 • edited

ycombinator Feb 25, 2021

Choose a reason for hiding this comment

mtojek Mar 1, 2021

Choose a reason for hiding this comment

ycombinator Mar 1, 2021

Choose a reason for hiding this comment

mtojek Mar 1, 2021

Choose a reason for hiding this comment

ycombinator Feb 25, 2021

Choose a reason for hiding this comment

mtojek Mar 1, 2021

Choose a reason for hiding this comment

ycombinator Mar 1, 2021

Choose a reason for hiding this comment

mtojek Mar 1, 2021 • edited

Choose a reason for hiding this comment

ycombinator left a comment

Choose a reason for hiding this comment

elasticmachine commented Feb 23, 2021 •

edited

mtojek commented Feb 25, 2021 •

edited

mtojek commented Feb 25, 2021 •

edited

mtojek Mar 1, 2021 •

edited