Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Addressing multiple dashboard issues: deps loading once, field conversion, etc. #27669

Merged
merged 3 commits into from
Sep 1, 2021

Conversation

kvch
Copy link
Contributor

@kvch kvch commented Aug 31, 2021

What does this PR do?

This PR addresses 3 minor issues in dashboard loading:

  1. Everything should be loaded once
  2. Some fields have to be strings that were objects before
  3. Replacing index names in dashboards is no longer an NDJSON

Why is it important?

This prevented setup from working properly.

Checklist

  • My code follows the style guidelines of this project
    - [ ] I have commented my code, particularly in hard-to-understand areas
    - [ ] I have made corresponding changes to the documentation
    - [ ] I have made corresponding change to the default configuration files
    - [ ] I have added tests that prove my fix is effective or that my feature works
    - [ ] I have added an entry in CHANGELOG.next.asciidoc or CHANGELOG-developer.next.asciidoc.

@botelastic botelastic bot added the needs_team Indicates that the issue/PR needs a Team:* label label Aug 31, 2021
@kvch kvch added backport-v7.15.0 Automated backport with mergify backport-v7.16.0 Automated backport with mergify Team:Elastic-Agent Label for the Agent team labels Aug 31, 2021
@elasticmachine
Copy link
Collaborator

Pinging @elastic/agent (Team:Agent)

@botelastic botelastic bot removed the needs_team Indicates that the issue/PR needs a Team:* label label Aug 31, 2021
@kvch kvch requested a review from jsoriano August 31, 2021 16:07
Copy link
Member

@jsoriano jsoriano left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Tested locally and works for me. 👍

libbeat/dashboards/modify_json.go Outdated Show resolved Hide resolved
libbeat/dashboards/modify_json.go Outdated Show resolved Hide resolved
@elasticmachine
Copy link
Collaborator

elasticmachine commented Aug 31, 2021

💔 Build Failed

the below badges are clickable and redirect to their specific view in the CI or DOCS
Pipeline View Test View Changes Artifacts preview preview

Expand to view the summary

Build stats

  • Start Time: 2021-08-31T17:59:18.701+0000

  • Duration: 146 min 15 sec

  • Commit: 283a353

Test stats 🧪

Test Results
Failed 0
Passed 53771
Skipped 5325
Total 59096

Trends 🧪

Image of Build Times

Image of Tests

Steps errors 2

Expand to view the steps failures

heartbeat-unitTest - mage build unitTest
  • Took 12 min 44 sec . View more details on here
  • Description: mage build unitTest
x-pack/metricbeat-unitTest - mage build unitTest
  • Took 4 min 45 sec . View more details on here
  • Description: mage build unitTest

Log output

Expand to view the last 100 lines of log output

[2021-08-31T20:24:00.083Z] fe4fa4ea17ef: Preparing
[2021-08-31T20:24:00.083Z] f525f363770a: Preparing
[2021-08-31T20:24:00.083Z] d91f980843e8: Preparing
[2021-08-31T20:24:00.083Z] ef787680e227: Preparing
[2021-08-31T20:24:00.083Z] b98ed0b3bc94: Preparing
[2021-08-31T20:24:00.083Z] 8ba798ca9dd2: Preparing
[2021-08-31T20:24:00.083Z] b89e91e1a428: Preparing
[2021-08-31T20:24:00.083Z] 6be89e890f99: Preparing
[2021-08-31T20:24:00.083Z] f78583c4380f: Preparing
[2021-08-31T20:24:00.083Z] 174f56854903: Preparing
[2021-08-31T20:24:00.083Z] 7f71968df527: Waiting
[2021-08-31T20:24:00.083Z] fe4fa4ea17ef: Waiting
[2021-08-31T20:24:00.083Z] f525f363770a: Waiting
[2021-08-31T20:24:00.083Z] d91f980843e8: Waiting
[2021-08-31T20:24:00.083Z] ef787680e227: Waiting
[2021-08-31T20:24:00.083Z] b98ed0b3bc94: Waiting
[2021-08-31T20:24:00.083Z] 8ba798ca9dd2: Waiting
[2021-08-31T20:24:00.083Z] 174f56854903: Waiting
[2021-08-31T20:24:00.083Z] 6be89e890f99: Waiting
[2021-08-31T20:24:00.083Z] b89e91e1a428: Waiting
[2021-08-31T20:24:00.083Z] f78583c4380f: Waiting
[2021-08-31T20:24:00.083Z] aa7cea5e1ebc: Layer already exists
[2021-08-31T20:24:00.083Z] 4ba00a2f40d3: Layer already exists
[2021-08-31T20:24:00.342Z] fe4fa4ea17ef: Layer already exists
[2021-08-31T20:24:00.342Z] eade5a33b239: Layer already exists
[2021-08-31T20:24:00.342Z] 1bd9ee165810: Layer already exists
[2021-08-31T20:24:00.342Z] e9ebed6e4151: Layer already exists
[2021-08-31T20:24:00.342Z] 7f71968df527: Layer already exists
[2021-08-31T20:24:00.342Z] ef787680e227: Layer already exists
[2021-08-31T20:24:00.342Z] f525f363770a: Layer already exists
[2021-08-31T20:24:00.342Z] d91f980843e8: Layer already exists
[2021-08-31T20:24:00.602Z] 8ba798ca9dd2: Layer already exists
[2021-08-31T20:24:00.602Z] f78583c4380f: Layer already exists
[2021-08-31T20:24:00.602Z] b98ed0b3bc94: Layer already exists
[2021-08-31T20:24:00.602Z] 174f56854903: Layer already exists
[2021-08-31T20:24:00.602Z] b89e91e1a428: Layer already exists
[2021-08-31T20:24:00.602Z] 6be89e890f99: Layer already exists
[2021-08-31T20:24:01.985Z] 283a353736a21cd4d99ae6b8d6f96c6b82d12670-amd64: digest: sha256:8f39e2b59723a1cf93d076348508848383ebdffbf2527ca69655b115e58ee36d size: 3665
[2021-08-31T20:24:02.109Z] Timeout set to expire in 5 min 0 sec
[2021-08-31T20:24:02.412Z] Cleaning up /var/lib/jenkins/workspace/PR-27669-3-a7ef95fe-6a82-48a5-903b-2651cc3225bf
[2021-08-31T20:24:02.412Z] Change ownership of all files inside the specific folder from root/root to current user/group
[2021-08-31T20:24:02.412Z] ++ id -u
[2021-08-31T20:24:02.412Z] ++ id -g
[2021-08-31T20:24:02.412Z] + docker run -v /var/lib/jenkins/workspace/PR-27669-3-a7ef95fe-6a82-48a5-903b-2651cc3225bf:/beat alpine:3.4 sh -c 'find /beat -user 0 -exec chown -h 1170:1171 {} \;'
[2021-08-31T20:24:02.412Z] Unable to find image 'alpine:3.4' locally
[2021-08-31T20:24:02.985Z] 3.4: Pulling from library/alpine
[2021-08-31T20:24:03.245Z] c1e54eec4b57: Pulling fs layer
[2021-08-31T20:24:03.505Z] c1e54eec4b57: Verifying Checksum
[2021-08-31T20:24:03.505Z] c1e54eec4b57: Download complete
[2021-08-31T20:24:03.767Z] c1e54eec4b57: Pull complete
[2021-08-31T20:24:03.767Z] Digest: sha256:b733d4a32c4da6a00a84df2ca32791bb03df95400243648d8c539e7b4cce329c
[2021-08-31T20:24:03.767Z] Status: Downloaded newer image for alpine:3.4
[2021-08-31T20:24:05.678Z] + set -e
[2021-08-31T20:24:05.678Z] + echo 'Change permissions with write access of all files inside the specific folder'
[2021-08-31T20:24:05.678Z] Change permissions with write access of all files inside the specific folder
[2021-08-31T20:24:05.678Z] + chmod -R +w /var/lib/jenkins/workspace/PR-27669-3-a7ef95fe-6a82-48a5-903b-2651cc3225bf
[2021-08-31T20:24:06.302Z] Running in /var/lib/jenkins/workspace/PR-27669-3-a7ef95fe-6a82-48a5-903b-2651cc3225bf/src/github.com/elastic/beats/build
[2021-08-31T20:24:06.606Z] + rm -rf ve
[2021-08-31T20:24:06.606Z] + find . -type d -name vendor -exec rm -r {} ;
[2021-08-31T20:24:06.934Z] + python .ci/scripts/pre_archive_test.py
[2021-08-31T20:24:16.928Z] Copy ./x-pack/heartbeat/build into build/x-pack/heartbeat/build
[2021-08-31T20:24:16.928Z] Copy ./x-pack/heartbeat/build/golang-crossbuild into build/x-pack/heartbeat/build/golang-crossbuild
[2021-08-31T20:24:16.928Z] Copy ./x-pack/filebeat/build into build/x-pack/filebeat/build
[2021-08-31T20:24:16.928Z] Copy ./x-pack/filebeat/build/golang-crossbuild into build/x-pack/filebeat/build/golang-crossbuild
[2021-08-31T20:24:16.928Z] Copy ./x-pack/elastic-agent/build into build/x-pack/elastic-agent/build
[2021-08-31T20:24:16.928Z] Copy ./x-pack/elastic-agent/build/package/elastic-agent-ubi8/elastic-agent-linux-amd64.docker/docker-build into build/x-pack/elastic-agent/build/package/elastic-agent-ubi8/elastic-agent-linux-amd64.docker/docker-build
[2021-08-31T20:24:16.928Z] Copy ./x-pack/elastic-agent/build/package/elastic-agent/elastic-agent-linux-amd64.docker/docker-build into build/x-pack/elastic-agent/build/package/elastic-agent/elastic-agent-linux-amd64.docker/docker-build
[2021-08-31T20:24:16.929Z] Copy ./x-pack/elastic-agent/build/golang-crossbuild into build/x-pack/elastic-agent/build/golang-crossbuild
[2021-08-31T20:24:16.929Z] Copy ./x-pack/metricbeat/build into build/x-pack/metricbeat/build
[2021-08-31T20:24:16.929Z] Copy ./x-pack/metricbeat/build/golang-crossbuild into build/x-pack/metricbeat/build/golang-crossbuild
[2021-08-31T20:24:16.953Z] Running in /var/lib/jenkins/workspace/PR-27669-3-a7ef95fe-6a82-48a5-903b-2651cc3225bf/src/github.com/elastic/beats/build
[2021-08-31T20:24:16.980Z] Recording test results
[2021-08-31T20:24:20.285Z] None of the test reports contained any result
[2021-08-31T20:24:20.294Z] [Checks API] No suitable checks publisher found.
[2021-08-31T20:24:20.655Z] + go clean -modcache
[2021-08-31T20:24:23.990Z] Timeout set to expire in 5 min 0 sec
[2021-08-31T20:24:24.291Z] Cleaning up /var/lib/jenkins/workspace/PR-27669-3-a7ef95fe-6a82-48a5-903b-2651cc3225bf
[2021-08-31T20:24:24.291Z] Change ownership of all files inside the specific folder from root/root to current user/group
[2021-08-31T20:24:24.291Z] ++ id -u
[2021-08-31T20:24:24.291Z] ++ id -g
[2021-08-31T20:24:24.291Z] + docker run -v /var/lib/jenkins/workspace/PR-27669-3-a7ef95fe-6a82-48a5-903b-2651cc3225bf:/beat alpine:3.4 sh -c 'find /beat -user 0 -exec chown -h 1170:1171 {} \;'
[2021-08-31T20:25:22.430Z] + set -e
[2021-08-31T20:25:22.430Z] + echo 'Change permissions with write access of all files inside the specific folder'
[2021-08-31T20:25:22.430Z] Change permissions with write access of all files inside the specific folder
[2021-08-31T20:25:22.430Z] + chmod -R +w /var/lib/jenkins/workspace/PR-27669-3-a7ef95fe-6a82-48a5-903b-2651cc3225bf
[2021-08-31T20:25:22.512Z] Running in /var/lib/jenkins/workspace/PR-27669-3-a7ef95fe-6a82-48a5-903b-2651cc3225bf
[2021-08-31T20:25:27.144Z] + gsutil --version
[2021-08-31T20:25:28.592Z] Masking supported pattern matches of $FILE_CREDENTIAL
[2021-08-31T20:25:28.913Z] + gcloud auth activate-service-account --key-file ****
[2021-08-31T20:25:29.485Z] Activated service account credentials for: [beats-ci-gcs-plugin@elastic-ci-prod.iam.gserviceaccount.com]
[2021-08-31T20:25:30.075Z] + gsutil -m -q cp eC1wYWNrL2VsYXN0aWMtYWdlbnQtcGFja2FnaW5nLWxpbnV4MjgzYTM1MzczNmEyMWNkNGQ5OWFlNmI4ZDZmOTZjNmI4MmQxMjY3MA gs://beats-ci-temp/ci/cache/
[2021-08-31T20:25:32.177Z] Stage "Packaging-Pipeline" skipped due to earlier failure(s)
[2021-08-31T20:25:32.266Z] Running in /var/lib/jenkins/workspace/Beats_beats_PR-27669/src/github.com/elastic/beats
[2021-08-31T20:25:32.592Z] Running on Jenkins in /var/lib/jenkins/workspace/Beats_beats_PR-27669
[2021-08-31T20:25:32.666Z] [INFO] getVaultSecret: Getting secrets
[2021-08-31T20:25:32.723Z] Masking supported pattern matches of $VAULT_ADDR or $VAULT_ROLE_ID or $VAULT_SECRET_ID
[2021-08-31T20:25:33.555Z] + chmod 755 generate-build-data.sh
[2021-08-31T20:25:33.556Z] + ./generate-build-data.sh https://beats-ci.elastic.co/blue/rest/organizations/jenkins/pipelines/Beats/beats/PR-27669/ https://beats-ci.elastic.co/blue/rest/organizations/jenkins/pipelines/Beats/beats/PR-27669/runs/3 FAILURE 8774590
[2021-08-31T20:25:33.556Z] INFO: curl https://beats-ci.elastic.co/blue/rest/organizations/jenkins/pipelines/Beats/beats/PR-27669/runs/3/steps/?limit=10000 -o steps-info.json
[2021-08-31T20:25:51.079Z] INFO: curl https://beats-ci.elastic.co/blue/rest/organizations/jenkins/pipelines/Beats/beats/PR-27669/runs/3/tests/?status=FAILED -o tests-errors.json

💚 Flaky test report

Tests succeeded.

Expand to view the summary

Test stats 🧪

Test Results
Failed 0
Passed 53771
Skipped 5325
Total 59096

Copy link
Member

@jsoriano jsoriano left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

👍

@kvch kvch merged commit 42ca950 into elastic:master Sep 1, 2021
kvch added a commit that referenced this pull request Sep 1, 2021
…sion, etc. (#27669)

This PR addresses 3 minor issues in dashboard loading:
1. Everything should be loaded once
2. Some fields have to be strings that were objects before
3. Replacing index names in dashboards is no longer an NDJSON

This prevented setup from working properly.

(cherry picked from commit 42ca950)
kvch added a commit that referenced this pull request Sep 1, 2021
…sion, etc. (#27669)

This PR addresses 3 minor issues in dashboard loading:
1. Everything should be loaded once
2. Some fields have to be strings that were objects before
3. Replacing index names in dashboards is no longer an NDJSON

This prevented setup from working properly.

(cherry picked from commit 42ca950)
kvch added a commit that referenced this pull request Sep 1, 2021
…sion, etc. (#27669) (#27682)

This PR addresses 3 minor issues in dashboard loading:
1. Everything should be loaded once
2. Some fields have to be strings that were objects before
3. Replacing index names in dashboards is no longer an NDJSON

This prevented setup from working properly.

(cherry picked from commit 42ca950)

Co-authored-by: Noémi Ványi <kvch@users.noreply.github.com>
kvch added a commit that referenced this pull request Sep 1, 2021
…sion, etc. (#27669) (#27683)

This PR addresses 3 minor issues in dashboard loading:
1. Everything should be loaded once
2. Some fields have to be strings that were objects before
3. Replacing index names in dashboards is no longer an NDJSON

This prevented setup from working properly.

(cherry picked from commit 42ca950)

Co-authored-by: Noémi Ványi <kvch@users.noreply.github.com>
mdelapenya added a commit to mdelapenya/beats that referenced this pull request Sep 1, 2021
* master:
  Forward port 7.14.1 changelog to master  (elastic#27687)
  Addressing multiple dashboard issues: deps loading once, field conversion, etc. (elastic#27669)
  Remove adaptive queue sizes from agent's spec files (elastic#27653)
  Osquerybeat: Improve testability and unit test coverage (elastic#27591)
  Osquerybeat: lockdown flagsfile, prevent global defaults (elastic#27611)
  Import the references of dashboard assets using the Saved Objects API (elastic#27647)
  Fix bug with override path in cgroups (elastic#27620)
  Allow Kibana client to authorize with Elasticsearch API key (elastic#27540)
  Filebeat auditd: Fix Top Exec Commands dashboard visualization (elastic#27638)
  [elastic-agent] Fix docker tar.gz generation for complete image (elastic#27621)
  Follow up changes in dashboards in mage check && fix minor issue (elastic#27553)
  [Heartbeat] Fix bug where `enabled: false` is ignored. (elastic#27615)
  Support kube_state_metrics v2.0.0 (elastic#27552)
Icedroid pushed a commit to Icedroid/beats that referenced this pull request Nov 1, 2021
…sion, etc. (elastic#27669)

## What does this PR do?

This PR addresses 3 minor issues in dashboard loading:
1. Everything should be loaded once
2. Some fields have to be strings that were objects before
3. Replacing index names in dashboards is no longer an NDJSON

## Why is it important?

This prevented setup from working properly.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
backport-v7.15.0 Automated backport with mergify backport-v7.16.0 Automated backport with mergify Team:Elastic-Agent Label for the Agent team
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants