Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: Add jsonnet-bundler support #12720

Merged
merged 29 commits into from
Nov 22, 2021
Merged

Conversation

andrein
Copy link
Contributor

@andrein andrein commented Nov 17, 2021

Changes:

This PR adds support for jsonnet-bundler.

TODO:

Context:

Closes #9800

Documentation (please check one with an [x])

  • I have updated the documentation, or
  • No documentation update is required

How I've tested my work (please tick one)

I have verified these changes via:

  • Code inspection only, or
  • Newly added/modified unit tests, or
  • No unit tests but ran on a real repository, or
  • Both unit tests + ran on a real repository

lib/manager/jsonnet-bundler/artifacts.spec.ts Outdated Show resolved Hide resolved
lib/manager/jsonnet-bundler/artifacts.spec.ts Outdated Show resolved Hide resolved
lib/manager/jsonnet-bundler/artifacts.ts Outdated Show resolved Hide resolved
lib/manager/jsonnet-bundler/artifacts.ts Outdated Show resolved Hide resolved
lib/manager/jsonnet-bundler/extract.spec.ts Outdated Show resolved Hide resolved
lib/manager/jsonnet-bundler/extract.ts Outdated Show resolved Hide resolved
lib/manager/jsonnet-bundler/extract.ts Outdated Show resolved Hide resolved
lib/manager/jsonnet-bundler/extract.ts Outdated Show resolved Hide resolved
lib/manager/jsonnet-bundler/index.ts Outdated Show resolved Hide resolved
lib/manager/jsonnet-bundler/index.ts Outdated Show resolved Hide resolved
@andrein
Copy link
Contributor Author

andrein commented Nov 18, 2021

The only remaining thing on my todo is "add jb to https://github.com/containerbase/buildpack". How do I do that?

@andrein andrein marked this pull request as ready for review November 18, 2021 21:50
@viceice viceice marked this pull request as draft November 19, 2021 08:23
@viceice
Copy link
Member

viceice commented Nov 19, 2021

Marking as draft, as it needs a buildpack update first.

@viceice
Copy link
Member

viceice commented Nov 19, 2021

The only remaining thing on my todo is "add jb to https://github.com/containerbase/buildpack". How do I do that?

Please open an issue there to discuss. Add some references how to install jb (should also be possible without root)

lib/manager/jsonnet-bundler/artifacts.spec.ts Outdated Show resolved Hide resolved
lib/manager/jsonnet-bundler/artifacts.spec.ts Outdated Show resolved Hide resolved
lib/manager/jsonnet-bundler/artifacts.spec.ts Outdated Show resolved Hide resolved
lib/manager/jsonnet-bundler/artifacts.spec.ts Outdated Show resolved Hide resolved
lib/manager/jsonnet-bundler/extract.ts Outdated Show resolved Hide resolved
@andrein andrein marked this pull request as ready for review November 20, 2021 11:54
@andrein andrein requested a review from viceice November 20, 2021 12:15
Copy link
Member

@viceice viceice left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

can you please do a real run with binarySource=docker ?

lib/manager/jsonnet-bundler/artifacts.ts Outdated Show resolved Hide resolved
Co-authored-by: Michael Kriese <michael.kriese@visualon.de>
@andrein
Copy link
Contributor Author

andrein commented Nov 20, 2021

I did a real run with binarySource=docker and it failed because the sidecar image isn't published yet. I've inspected the output and it looks like it's doing the right thing (tm)

       "err": {
         "killed": false,
         "code": 1,
         "signal": null,
         "cmd": "docker run --rm --name=renovate_sidecar --label=renovate_child -v \"/var/folders/_t/f_q95xcd0ws9s_cqvh6tpgfh0000gn/T/renovate/repos/github/andrein/renovate-test\":\"/var/folders/_t/f_q95xcd0ws9s_cqvh6tpgfh0000gn/T/renovate/repos/github/andrein/renovate-test\" -v \"/var/folders/_t/f_q95xcd0ws9s_cqvh6tpgfh0000gn/T/renovate/cache\":\"/var/folders/_t/f_q95xcd0ws9s_cqvh6tpgfh0000gn/T/renovate/cache\" -w \"/var/folders/_t/f_q95xcd0ws9s_cqvh6tpgfh0000gn/T/renovate/repos/github/andrein/renovate-test/jsonnet-bundler\" docker.io/renovate/sidecar bash -l -c \"install-tool jb v0.4.0 && jb update https://github.com/thanos-io/thanos.git/mixin\"",
         "stdout": "",
         "stderr": "No tool defined - skipping: jb\n",
         "message": "Command failed: docker run --rm --name=renovate_sidecar --label=renovate_child -v \"/var/folders/_t/f_q95xcd0ws9s_cqvh6tpgfh0000gn/T/renovate/repos/github/andrein/renovate-test\":\"/var/folders/_t/f_q95xcd0ws9s_cqvh6tpgfh0000gn/T/renovate/repos/github/andrein/renovate-test\" -v \"/var/folders/_t/f_q95xcd0ws9s_cqvh6tpgfh0000gn/T/renovate/cache\":\"/var/folders/_t/f_q95xcd0ws9s_cqvh6tpgfh0000gn/T/renovate/cache\" -w \"/var/folders/_t/f_q95xcd0ws9s_cqvh6tpgfh0000gn/T/renovate/repos/github/andrein/renovate-test/jsonnet-bundler\" docker.io/renovate/sidecar bash -l -c \"install-tool jb v0.4.0 && jb update https://github.com/thanos-io/thanos.git/mixin\"\nNo tool defined - skipping: jb\n",
         "stack": "Error: Command failed: docker run --rm --name=renovate_sidecar --label=renovate_child -v \"/var/folders/_t/f_q95xcd0ws9s_cqvh6tpgfh0000gn/T/renovate/repos/github/andrein/renovate-test\":\"/var/folders/_t/f_q95xcd0ws9s_cqvh6tpgfh0000gn/T/renovate/repos/github/andrein/renovate-test\" -v \"/var/folders/_t/f_q95xcd0ws9s_cqvh6tpgfh0000gn/T/renovate/cache\":\"/var/folders/_t/f_q95xcd0ws9s_cqvh6tpgfh0000gn/T/renovate/cache\" -w \"/var/folders/_t/f_q95xcd0ws9s_cqvh6tpgfh0000gn/T/renovate/repos/github/andrein/renovate-test/jsonnet-bundler\" docker.io/renovate/sidecar bash -l -c \"install-tool jb v0.4.0 && jb update https://github.com/thanos-io/thanos.git/mixin\"\nNo tool defined - skipping: jb\n\n    at ChildProcess.exithandler (node:child_process:397:12)\n    at ChildProcess.emit (node:events:390:28)\n    at ChildProcess.emit (node:domain:475:12)\n    at maybeClose (node:internal/child_process:1062:16)\n    at Socket.<anonymous> (node:internal/child_process:448:11)\n    at Socket.emit (node:events:390:28)\n    at Socket.emit (node:domain:475:12)\n    at Pipe.<anonymous> (node:net:687:12)"
       }```

@andrein andrein requested a review from viceice November 20, 2021 13:03
@viceice
Copy link
Member

viceice commented Nov 20, 2021

I did a real run with binarySource=docker and it failed because the sidecar image isn't published yet. I've inspected the output and it looks like it's doing the right thing (tm)

       "err": {
         "killed": false,
         "code": 1,
         "signal": null,
         "cmd": "docker run --rm --name=renovate_sidecar --label=renovate_child -v \"/var/folders/_t/f_q95xcd0ws9s_cqvh6tpgfh0000gn/T/renovate/repos/github/andrein/renovate-test\":\"/var/folders/_t/f_q95xcd0ws9s_cqvh6tpgfh0000gn/T/renovate/repos/github/andrein/renovate-test\" -v \"/var/folders/_t/f_q95xcd0ws9s_cqvh6tpgfh0000gn/T/renovate/cache\":\"/var/folders/_t/f_q95xcd0ws9s_cqvh6tpgfh0000gn/T/renovate/cache\" -w \"/var/folders/_t/f_q95xcd0ws9s_cqvh6tpgfh0000gn/T/renovate/repos/github/andrein/renovate-test/jsonnet-bundler\" docker.io/renovate/sidecar bash -l -c \"install-tool jb v0.4.0 && jb update https://github.com/thanos-io/thanos.git/mixin\"",
         "stdout": "",
         "stderr": "No tool defined - skipping: jb\n",
         "message": "Command failed: docker run --rm --name=renovate_sidecar --label=renovate_child -v \"/var/folders/_t/f_q95xcd0ws9s_cqvh6tpgfh0000gn/T/renovate/repos/github/andrein/renovate-test\":\"/var/folders/_t/f_q95xcd0ws9s_cqvh6tpgfh0000gn/T/renovate/repos/github/andrein/renovate-test\" -v \"/var/folders/_t/f_q95xcd0ws9s_cqvh6tpgfh0000gn/T/renovate/cache\":\"/var/folders/_t/f_q95xcd0ws9s_cqvh6tpgfh0000gn/T/renovate/cache\" -w \"/var/folders/_t/f_q95xcd0ws9s_cqvh6tpgfh0000gn/T/renovate/repos/github/andrein/renovate-test/jsonnet-bundler\" docker.io/renovate/sidecar bash -l -c \"install-tool jb v0.4.0 && jb update https://github.com/thanos-io/thanos.git/mixin\"\nNo tool defined - skipping: jb\n",
         "stack": "Error: Command failed: docker run --rm --name=renovate_sidecar --label=renovate_child -v \"/var/folders/_t/f_q95xcd0ws9s_cqvh6tpgfh0000gn/T/renovate/repos/github/andrein/renovate-test\":\"/var/folders/_t/f_q95xcd0ws9s_cqvh6tpgfh0000gn/T/renovate/repos/github/andrein/renovate-test\" -v \"/var/folders/_t/f_q95xcd0ws9s_cqvh6tpgfh0000gn/T/renovate/cache\":\"/var/folders/_t/f_q95xcd0ws9s_cqvh6tpgfh0000gn/T/renovate/cache\" -w \"/var/folders/_t/f_q95xcd0ws9s_cqvh6tpgfh0000gn/T/renovate/repos/github/andrein/renovate-test/jsonnet-bundler\" docker.io/renovate/sidecar bash -l -c \"install-tool jb v0.4.0 && jb update https://github.com/thanos-io/thanos.git/mixin\"\nNo tool defined - skipping: jb\n\n    at ChildProcess.exithandler (node:child_process:397:12)\n    at ChildProcess.emit (node:events:390:28)\n    at ChildProcess.emit (node:domain:475:12)\n    at maybeClose (node:internal/child_process:1062:16)\n    at Socket.<anonymous> (node:internal/child_process:448:11)\n    at Socket.emit (node:events:390:28)\n    at Socket.emit (node:domain:475:12)\n    at Pipe.<anonymous> (node:net:687:12)"
       }```

renovate is missing on https://github.com/renovatebot/docker-sidecar, so the image isn't yet updated with latest buildpack v2.3.0

@andrein
Copy link
Contributor Author

andrein commented Nov 20, 2021

Shall I make a PR to bump the version/add renovate?

@viceice
Copy link
Member

viceice commented Nov 20, 2021

Shall I make a PR to bump the version/add renovate?

nope, the app simply wasn't installed on that repo, which is now fixed and the update should be release in a couple of minutes

@viceice
Copy link
Member

viceice commented Nov 20, 2021

ok, released: https://hub.docker.com/r/renovate/sidecar/tags

Please retry docker binary source

@andrein
Copy link
Contributor Author

andrein commented Nov 20, 2021

Retried, and it failed with:

DEBUG: Executing command (repository=andrein/renovate-test, branch=renovate/thanos-0.x)
       "command": [
         "docker run --rm --name=renovate_sidecar --label=renovate_child -v \"/var/folders/_t/f_q95xcd0ws9s_cqvh6tpgfh0000gn/T/renovate/repos/github/andrein/renovate-test\":\"/var/folders/_t/f_q95xcd0ws9s_cqvh6tpgfh0000gn/T/renovate/repos/github/andrein/renovate-test\" -v \"/var/folders/_t/f_q95xcd0ws9s_cqvh6tpgfh0000gn/T/renovate/cache\":\"/var/folders/_t/f_q95xcd0ws9s_cqvh6tpgfh0000gn/T/renovate/cache\" -w \"/var/folders/_t/f_q95xcd0ws9s_cqvh6tpgfh0000gn/T/renovate/repos/github/andrein/renovate-test/jsonnet-bundler\" docker.io/renovate/sidecar bash -l -c \"install-tool jb v0.4.0 && jb update https://github.com/thanos-io/thanos.git/mixin\""
       ]
DEBUG: rawExec err (repository=andrein/renovate-test, branch=renovate/thanos-0.x)
       "err": {
         "killed": false,
         "code": 1,
         "signal": null,
         "cmd": "docker run --rm --name=renovate_sidecar --label=renovate_child -v \"/var/folders/_t/f_q95xcd0ws9s_cqvh6tpgfh0000gn/T/renovate/repos/github/andrein/renovate-test\":\"/var/folders/_t/f_q95xcd0ws9s_cqvh6tpgfh0000gn/T/renovate/repos/github/andrein/renovate-test\" -v \"/var/folders/_t/f_q95xcd0ws9s_cqvh6tpgfh0000gn/T/renovate/cache\":\"/var/folders/_t/f_q95xcd0ws9s_cqvh6tpgfh0000gn/T/renovate/cache\" -w \"/var/folders/_t/f_q95xcd0ws9s_cqvh6tpgfh0000gn/T/renovate/repos/github/andrein/renovate-test/jsonnet-bundler\" docker.io/renovate/sidecar bash -l -c \"install-tool jb v0.4.0 && jb update https://github.com/thanos-io/thanos.git/mixin\"",
         "stdout": "Installing tool jb v0.4.0\n",
         "stderr": "v0.4.0\njb: error: failed to load jsonnetfile: open /var/folders/_t/f_q95xcd0ws9s_cqvh6tpgfh0000gn/T/renovate/repos/github/andrein/renovate-test/jsonnet-bundler/jsonnetfile.json: no such file or directory\n",
         "message": "Command failed: docker run --rm --name=renovate_sidecar --label=renovate_child -v \"/var/folders/_t/f_q95xcd0ws9s_cqvh6tpgfh0000gn/T/renovate/repos/github/andrein/renovate-test\":\"/var/folders/_t/f_q95xcd0ws9s_cqvh6tpgfh0000gn/T/renovate/repos/github/andrein/renovate-test\" -v \"/var/folders/_t/f_q95xcd0ws9s_cqvh6tpgfh0000gn/T/renovate/cache\":\"/var/folders/_t/f_q95xcd0ws9s_cqvh6tpgfh0000gn/T/renovate/cache\" -w \"/var/folders/_t/f_q95xcd0ws9s_cqvh6tpgfh0000gn/T/renovate/repos/github/andrein/renovate-test/jsonnet-bundler\" docker.io/renovate/sidecar bash -l -c \"install-tool jb v0.4.0 && jb update https://github.com/thanos-io/thanos.git/mixin\"\nv0.4.0\njb: error: failed to load jsonnetfile: open /var/folders/_t/f_q95xcd0ws9s_cqvh6tpgfh0000gn/T/renovate/repos/github/andrein/renovate-test/jsonnet-bundler/jsonnetfile.json: no such file or directory\n",
         "stack": "Error: Command failed: docker run --rm --name=renovate_sidecar --label=renovate_child -v \"/var/folders/_t/f_q95xcd0ws9s_cqvh6tpgfh0000gn/T/renovate/repos/github/andrein/renovate-test\":\"/var/folders/_t/f_q95xcd0ws9s_cqvh6tpgfh0000gn/T/renovate/repos/github/andrein/renovate-test\" -v \"/var/folders/_t/f_q95xcd0ws9s_cqvh6tpgfh0000gn/T/renovate/cache\":\"/var/folders/_t/f_q95xcd0ws9s_cqvh6tpgfh0000gn/T/renovate/cache\" -w \"/var/folders/_t/f_q95xcd0ws9s_cqvh6tpgfh0000gn/T/renovate/repos/github/andrein/renovate-test/jsonnet-bundler\" docker.io/renovate/sidecar bash -l -c \"install-tool jb v0.4.0 && jb update https://github.com/thanos-io/thanos.git/mixin\"\nv0.4.0\njb: error: failed to load jsonnetfile: open /var/folders/_t/f_q95xcd0ws9s_cqvh6tpgfh0000gn/T/renovate/repos/github/andrein/renovate-test/jsonnet-bundler/jsonnetfile.json: no such file or directory\n\n    at ChildProcess.exithandler (node:child_process:397:12)\n    at ChildProcess.emit (node:events:390:28)\n    at ChildProcess.emit (node:domain:475:12)\n    at maybeClose (node:internal/child_process:1062:16)\n    at Socket.<anonymous> (node:internal/child_process:448:11)\n    at Socket.emit (node:events:390:28)\n    at Socket.emit (node:domain:475:12)\n    at Pipe.<anonymous> (node:net:687:12)"
       }

Did I miss anything?

@viceice
Copy link
Member

viceice commented Nov 20, 2021

Your local user id is probably not 1000, so the docker sidecar can't read the git checkout.

@andrein
Copy link
Contributor Author

andrein commented Nov 20, 2021

Your local user id is probably not 1000, so the docker sidecar can't read the git checkout.

That wasn't it. I was connecting to a remote docker daemon and volumes weren't working (suprise).

When I ran it with a local docker daemon it worked:

DEBUG: No tag or tagConstraint specified (repository=andrein/renovate-test, branch=renovate/thanos-0.x)
       "image": "docker.io/renovate/sidecar"
DEBUG: Fetching Docker image: docker.io/renovate/sidecar (repository=andrein/renovate-test, branch=renovate/thanos-0.x)
DEBUG: Finished fetching Docker image (repository=andrein/renovate-test, branch=renovate/thanos-0.x)
DEBUG: Executing command (repository=andrein/renovate-test, branch=renovate/thanos-0.x)
       "command": [
         "docker run --rm --name=renovate_sidecar --label=renovate_child -v \"/var/folders/_t/f_q95xcd0ws9s_cqvh6tpgfh0000gn/T/renovate/repos/github/andrein/renovate-test\":\"/var/folders/_t/f_q95xcd0ws9s_cqvh6tpgfh0000gn/T/renovate/repos/github/andrein/renovate-test\" -v \"/var/folders/_t/f_q95xcd0ws9s_cqvh6tpgfh0000gn/T/renovate/cache\":\"/var/folders/_t/f_q95xcd0ws9s_cqvh6tpgfh0000gn/T/renovate/cache\" -w \"/var/folders/_t/f_q95xcd0ws9s_cqvh6tpgfh0000gn/T/renovate/repos/github/andrein/renovate-test/jsonnet-bundler\" docker.io/renovate/sidecar bash -l -c \"install-tool jb v0.4.0 && jb update https://github.com/thanos-io/thanos.git/mixin\""
       ]
DEBUG: exec completed (repository=andrein/renovate-test, branch=renovate/thanos-0.x)
       "cmd": "docker run --rm --name=renovate_sidecar --label=renovate_child -v \"/var/folders/_t/f_q95xcd0ws9s_cqvh6tpgfh0000gn/T/renovate/repos/github/andrein/renovate-test\":\"/var/folders/_t/f_q95xcd0ws9s_cqvh6tpgfh0000gn/T/renovate/repos/github/andrein/renovate-test\" -v \"/var/folders/_t/f_q95xcd0ws9s_cqvh6tpgfh0000gn/T/renovate/cache\":\"/var/folders/_t/f_q95xcd0ws9s_cqvh6tpgfh0000gn/T/renovate/cache\" -w \"/var/folders/_t/f_q95xcd0ws9s_cqvh6tpgfh0000gn/T/renovate/repos/github/andrein/renovate-test/jsonnet-bundler\" docker.io/renovate/sidecar bash -l -c \"install-tool jb v0.4.0 && jb update https://github.com/thanos-io/thanos.git/mixin\"",
       "durationMs": 9776,
       "stdout": "Installing tool jb v0.4.0\n",
       "stderr": "v0.4.0\nGET https://github.com/thanos-io/thanos/archive/68ff28df232fc1f610a04b114290bdb68d1ba61e.tar.gz 200\n"
DEBUG: Updated 1 package files (repository=andrein/renovate-test, branch=renovate/thanos-0.x)
DEBUG: Updated 11 lock files (repository=andrein/renovate-test, branch=renovate/thanos-0.x)
       "updatedArtifacts": [
         "jsonnet-bundler/jsonnetfile.json",
         "jsonnet-bundler/jsonnetfile.lock.json",
         "jsonnet-bundler/vendor/github.com/thanos-io/thanos/mixin/README.md",
         "jsonnet-bundler/vendor/github.com/thanos-io/thanos/mixin/dashboards/bucket_replicate.libsonnet",
         "jsonnet-bundler/vendor/github.com/thanos-io/thanos/mixin/dashboards/compact.libsonnet",
         "jsonnet-bundler/vendor/github.com/thanos-io/thanos/mixin/dashboards/query.libsonnet",
         "jsonnet-bundler/vendor/github.com/thanos-io/thanos/mixin/dashboards/receive.libsonnet",
         "jsonnet-bundler/vendor/github.com/thanos-io/thanos/mixin/dashboards/rule.libsonnet",
         "jsonnet-bundler/vendor/github.com/thanos-io/thanos/mixin/dashboards/sidecar.libsonnet",
         "jsonnet-bundler/vendor/github.com/thanos-io/thanos/mixin/dashboards/store.libsonnet",
         "jsonnet-bundler/vendor/github.com/thanos-io/thanos/mixin/lib/thanos-grafana-builder/README.md"
       ]

docs/usage/configuration-options.md Outdated Show resolved Hide resolved
andrein and others added 2 commits November 22, 2021 11:07
Co-authored-by: HonkingGoose <34918129+HonkingGoose@users.noreply.github.com>
Copy link
Collaborator

@HonkingGoose HonkingGoose left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Documentation looks good to me!

@rarkins rarkins changed the title feat(manager/jsonnet-bundler): Add jsonnet-bundler support feat: Add jsonnet-bundler support Nov 22, 2021
@rarkins rarkins enabled auto-merge (squash) November 22, 2021 15:28
@rarkins rarkins merged commit d2d356c into renovatebot:main Nov 22, 2021
@renovate-release
Copy link
Collaborator

🎉 This PR is included in version 29.17.0 🎉

The release is available on:

Your semantic-release bot 📦🚀

@andrein andrein deleted the feat/jsonnet-bundler branch November 23, 2021 08:50
@github-actions github-actions bot locked as resolved and limited conversation to collaborators Dec 24, 2021
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Support for a new package manager : jsonnet-bundler
5 participants