Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Pixie doesn't work on Rancher Desktop #447

Open
ismaeIfm opened this issue May 31, 2022 · 12 comments
Open

Pixie doesn't work on Rancher Desktop #447

ismaeIfm opened this issue May 31, 2022 · 12 comments
Assignees
Labels
area/datacollector Issues related to Stirling (datacollector) kind/compatibility Compatibility related to either K8s, architecture, library versions priority/important-longterm Important over the long term, but may not be staffed and/or may need multiple releases to complete. triage/needs-information Indicates an issue needs more information in order to work on it.

Comments

@ismaeIfm
Copy link

Describe the bug
Pixie doesn't work on Rancher Desktop.

To Reproduce
Steps to reproduce the behavior:

  1. Install pixie on Rancher Desktop (k8s)
  2. Select dockerd runtime
  3. Install pixie
  4. See error

Expected behavior
I expect pixie to work

Screenshots
Screenshot 2022-05-31 at 22 27 28

Logs
Please attach the logs by running the following command:

./px collect-logs

pixie_logs_20220531221823.zip

App information (please complete the following information):

  • Pixie version: 0.7.9+Distribution.a47d77a.20220510221202.1
  • K8s cluster version Server Version: version.Info{Major:"1", Minor:"19", GitVersion:"v1.19.16+k3s1", GitCommit:"da16869555775cf17d4d97ffaf8a13b70bc738c2", GitTreeState:"clean", BuildDate:"2021-11-04T00:54:15Z", GoVersion:"go1.15.14", Compiler:"gc", Platform:"linux/arm64"}
  • Node Kernel version
  • Browser version

Additional context
Add any other context about the problem here.

@aimichelle aimichelle assigned aimichelle and yzhao1012 and unassigned aimichelle May 31, 2022
@yzhao1012
Copy link
Contributor

Turns out that the VM started by rancher-desktop does not have Kernel config file, that’s a list of config entries originally used in building the kernel. Pixie uses it to patch the packaged headers to make them as close to the actual kernel as possible.

lima-vm/alpine-lima#113
I have created this issue on lima-vm, which is the VM tool used by rancher-desktop.

@jandubois
Copy link

The file is available in the boot image:

$ rdctl shell head /media/sr0/boot/config-virt
#
# Automatically generated file; DO NOT EDIT.
# Linux/x86_64 5.15.40 Kernel Configuration
#
CONFIG_CC_VERSION_TEXT="gcc (Alpine 10.3.1_git20211027) 10.3.1 20211027"
CONFIG_CC_IS_GCC=y
CONFIG_GCC_VERSION=100301
CONFIG_CLANG_VERSION=0
CONFIG_AS_IS_GNU=y
CONFIG_AS_VERSION=23700

@yzhao1012
Copy link
Contributor

Thanks @jandubois !
I've added /media/sr0/boot/config-virt into Pixie's config search path, and verified that Pixie now works fine on rancher-desktop, on Ubuntu Linux.

I still need to verify on mac. Since initial examination shows that /media/sr0/boot/config-virt does not exist on my m1 macbook air running with Lima VM.

@jandubois
Copy link

I still need to verify on mac. Since initial examination shows that /media/sr0/boot/config-virt does not exist on my m1 macbook air running with Lima VM.

I don't know why, but on the M1 it seems to be under vda:

lima-rancher-desktop:~# head /media/vda/boot/config-virt
#
# Automatically generated file; DO NOT EDIT.
# Linux/arm64 5.15.40 Kernel Configuration
#
CONFIG_CC_VERSION_TEXT="gcc (Alpine 10.3.1_git20211027) 10.3.1 20211027"
CONFIG_CC_IS_GCC=y
CONFIG_GCC_VERSION=100301
CONFIG_CLANG_VERSION=0
CONFIG_AS_IS_GNU=y
CONFIG_AS_VERSION=23700

I'll create an issue on the alpine-lima repo to see if we can do something to make sure the boot CD is always mounted on /media/cdrom to have a predictable location; I don't know why this isn't the case right now.

@jandubois
Copy link

@yzhao1012 Could you include instructions here on how you got Pixie installed on RD, especially once you have it working on macOS? Thank you!

@yzhao1012
Copy link
Contributor

@yzhao1012 Could you include instructions here on how you got Pixie installed on RD, especially once you have it working on macOS? Thank you!

Re:
Once rancher-desktop is installed and launched, follow https://docs.px.dev/installing-pixie/install-guides/community-cloud-for-pixie to install Pixie.
That instruction just uses local kubectl config to figure out the target kubernetes cluster and how to reach it. Then deploy Pixie with a predefined packages from public GCR repos.

@yzhao1012
Copy link
Contributor

We have added 2 kernel config paths used by rancher-desktop's CDRom-boot kernel image, under @jandubois' information.
See 57e1dfd

You should be able to test this after this week's Vizier release.

@tharinduwijewardane
Copy link
Contributor

tharinduwijewardane commented Jun 12, 2022

@yzhao1012 I got this exact error (with rancher desktop and dockerd runtime) and I tried to build vizier and deploy it by following the development guide
(Pls note I have to manually set the DOCKER_ID and GROUP_ID in latest run_docker.sh script.)
But I keep getting the following output.

Output of `skaffold run -f skaffold/skaffold_vizier.yaml`

Generating tags...

  • gcr.io/pixie-oss/pixie-dev/vizier/pem_image -> gcr.io/pixie-oss/pixie-dev/vizier/pem_image:2022-06-12_04-50-10.549_UTC
  • gcr.io/pixie-oss/pixie-dev/vizier/kelvin_image -> gcr.io/pixie-oss/pixie-dev/vizier/kelvin_image:2022-06-12_04-50-10.549_UTC
  • gcr.io/pixie-oss/pixie-dev/vizier/metadata_server_image -> gcr.io/pixie-oss/pixie-dev/vizier/metadata_server_image:2022-06-12_04-50-10.549_UTC
  • gcr.io/pixie-oss/pixie-dev/vizier/query_broker_server_image -> gcr.io/pixie-oss/pixie-dev/vizier/query_broker_server_image:2022-06-12_04-50-10.549_UTC
  • gcr.io/pixie-oss/pixie-dev/vizier/cloud_connector_server_image -> gcr.io/pixie-oss/pixie-dev/vizier/cloud_connector_server_image:2022-06-12_04-50-10.549_UTC
  • gcr.io/pixie-oss/pixie-dev/vizier/cert_provisioner_image -> gcr.io/pixie-oss/pixie-dev/vizier/cert_provisioner_image:2022-06-12_04-50-10.549_UTC
    Checking cache...
    WARN[0002] Retrieving Bazel dependencies can take a long time the first time subtask=-1 task=Build
  • gcr.io/pixie-oss/pixie-dev/vizier/pem_image: Error checking cache.
    getting hash for artifact "gcr.io/pixie-oss/pixie-dev/vizier/pem_image": getting dependencies for "gcr.io/pixie-oss/pixie-dev/vizier/pem_image": getting bazel dependencies: running [bazel query kind('source file', deps('//src/vizier/services/agent/pem:pem_image.tar')) union buildfiles(deps('//src/vizier/services/agent/pem:pem_image.tar')) --noimplicit_deps --order_output=no --output=label]
  • stdout: ""
  • stderr: "Another command holds the client lock: \npid=74\nowner=client\ncwd=/pl/src/px.dev/pixie\n\nWaiting for it to complete...\nAnother command (pid=74) is running. Waiting for it to complete on the server (server_pid=84)...\nAnother command (pid=79) is running. Waiting for it to complete on the server (server_pid=84)...\nAnother command (pid=76) is running. Waiting for it to complete on the server (server_pid=84)...\nLoading: 0 packages loaded\nLoading: 20 packages loaded\n currently loading: @com_llvm_lib//\nINFO: Repository base_image instantiated at:\n /pl/src/px.dev/pixie/WORKSPACE:92:20: in \n /pl/src/px.dev/pixie/bazel/pl_workspace.bzl:68:28: in pl_container_images\n /pl/src/px.dev/pixie/bazel/pl_workspace.bzl:38:16: in _container_images_setup\n /pl/src/px.dev/pixie/bazel/container_images.bzl:50:18: in base_images\n /pl/src/px.dev/pixie/bazel/container_images.bzl:30:19: in _gcr_io_image\nRepository rule container_pull defined at:\n /home/tharindu/.cache/bazel/_bazel_tharindu/4c31fb537ca0f31ab15bbd6a8445d3b6/external/io_bazel_rules_docker/container/pull.bzl:294:33: in \nLoading: 34 packages loaded\nERROR: An error occurred during the fetch of repository 'base_image':\n Traceback (most recent call last):\n\tFile "/home/tharindu/.cache/bazel/_bazel_tharindu/4c31fb537ca0f31ab15bbd6a8445d3b6/external/io_bazel_rules_docker/container/pull.bzl", line 240, column 13, in _impl\n\t\tfail("Pull command failed: %s (%s)" % (result.stderr, " ".join([str(a) for a in args])))\nError in fail: Pull command failed: 2022/06/12 04:55:09 Running the Image Puller to pull images from a Docker Registry...\n2022/06/12 04:55:09 Image pull was unsuccessful: reading image "gcr.io/distroless/base@sha256:34e96e21516698913035a62ef1ce484d91184de8a44209a33c7e134547e20dee": error getting credentials - err: exec: "docker-credential-osxkeychain": executable file not found in $PATH, out: \n (/home/tharindu/.cache/bazel/_bazel_tharindu/4c31fb537ca0f31ab15bbd6a8445d3b6/external/go_puller_linux_amd64/file/downloaded -directory /home/tharindu/.cache/bazel/_bazel_tharindu/4c31fb537ca0f31ab15bbd6a8445d3b6/external/base_image/image -os linux -os-version -os-features -architecture amd64 -variant -features -name gcr.io/distroless/base@sha256:34e96e21516698913035a62ef1ce484d91184de8a44209a33c7e134547e20dee)\nERROR: /pl/src/px.dev/pixie/WORKSPACE:92:20: fetching container_pull rule //external:base_image: Traceback (most recent call last):\n\tFile \"/home/tharindu/.cache/bazel/_bazel_tharindu/4c31fb537ca0f31ab15bbd6a8445d3b6/external/io_bazel_rules_docker/container/pull.bzl\", line 240, column 13, in _impl\n\t\tfail(\"Pull command failed: %s (%s)\" % (result.stderr, \" \".join([str(a) for a in args])))\nError in fail: Pull command failed: 2022/06/12 04:55:09 Running the Image Puller to pull images from a Docker Registry...\n2022/06/12 04:55:09 Image pull was unsuccessful: reading image \"gcr.io/distroless/base@sha256:34e96e21516698913035a62ef1ce484d91184de8a44209a33c7e134547e20dee\": error getting credentials - err: exec: \"docker-credential-osxkeychain\": executable file not found in $PATH, out: \n (/home/tharindu/.cache/bazel/_bazel_tharindu/4c31fb537ca0f31ab15bbd6a8445d3b6/external/go_puller_linux_amd64/file/downloaded -directory /home/tharindu/.cache/bazel/_bazel_tharindu/4c31fb537ca0f31ab15bbd6a8445d3b6/external/base_image/image -os linux -os-version -os-features -architecture amd64 -variant -features -name gcr.io/distroless/base@sha256:34e96e21516698913035a62ef1ce484d91184de8a44209a33c7e134547e20dee)\nLoading: 34 packages loaded\nERROR: /pl/src/px.dev/pixie/BUILD.bazel:137:16: no such package '@base_image//image': Pull command failed: 2022/06/12 04:55:09 Running the Image Puller to pull images from a Docker Registry...\n2022/06/12 04:55:09 Image pull was unsuccessful: reading image "gcr.io/distroless/base@sha256:34e96e21516698913035a62ef1ce484d91184de8a44209a33c7e134547e20dee": error getting credentials - err: exec: "docker-credential-osxkeychain": executable file not found in $PATH, out: \n (/home/tharindu/.cache/bazel/_bazel_tharindu/4c31fb537ca0f31ab15bbd6a8445d3b6/external/go_puller_linux_amd64/file/downloaded -directory /home/tharindu/.cache/bazel/_bazel_tharindu/4c31fb537ca0f31ab15bbd6a8445d3b6/external/base_image/image -os linux -os-version -os-features -architecture amd64 -variant -features -name gcr.io/distroless/base@sha256:34e96e21516698913035a62ef1ce484d91184de8a44209a33c7e134547e20dee) and referenced by '//:pl_cc_base_image'\nINFO: Repository openjdk-base-musl instantiated at:\n /pl/src/px.dev/pixie/WORKSPACE:92:20: in <toplevel>\n /pl/src/px.dev/pixie/bazel/pl_workspace.bzl:68:28: in pl_container_images\n /pl/src/px.dev/pixie/bazel/pl_workspace.bzl:38:16: in _container_images_setup\n /pl/src/px.dev/pixie/bazel/container_images.bzl:68:18: in base_images\n /pl/src/px.dev/pixie/bazel/container_images.bzl:30:19: in _gcr_io_image\nRepository rule container_pull defined at:\n /home/tharindu/.cache/bazel/_bazel_tharindu/4c31fb537ca0f31ab15bbd6a8445d3b6/external/io_bazel_rules_docker/container/pull.bzl:294:33: in <toplevel>\nERROR: An error occurred during the fetch of repository 'openjdk-base-musl':\n Traceback (most recent call last):\n\tFile \"/home/tharindu/.cache/bazel/_bazel_tharindu/4c31fb537ca0f31ab15bbd6a8445d3b6/external/io_bazel_rules_docker/container/pull.bzl\", line 240, column 13, in _impl\n\t\tfail(\"Pull command failed: %s (%s)\" % (result.stderr, \" \".join([str(a) for a in args])))\nError in fail: Pull command failed: 2022/06/12 04:55:22 Running the Image Puller to pull images from a Docker Registry...\n2022/06/12 04:55:22 Image pull was unsuccessful: reading image \"gcr.io/pixie-oss/pixie-dev-public/docker-deps/library/openjdk@sha256:25b910311bfe15547ecab6895d5eb3f4ec718d6d53cced7eec78e4b889962e1f\": error getting credentials - err: exec: \"docker-credential-osxkeychain\": executable file not found in $PATH, out: \n (/home/tharindu/.cache/bazel/_bazel_tharindu/4c31fb537ca0f31ab15bbd6a8445d3b6/external/go_puller_linux_amd64/file/downloaded -directory /home/tharindu/.cache/bazel/_bazel_tharindu/4c31fb537ca0f31ab15bbd6a8445d3b6/external/openjdk-base-musl/image -os linux -os-version -os-features -architecture amd64 -variant -features -name gcr.io/pixie-oss/pixie-dev-public/docker-deps/library/openjdk@sha256:25b910311bfe15547ecab6895d5eb3f4ec718d6d53cced7eec78e4b889962e1f)\nERROR: /pl/src/px.dev/pixie/WORKSPACE:92:20: fetching container_pull rule //external:openjdk-base-musl: Traceback (most recent call last):\n\tFile "/home/tharindu/.cache/bazel/_bazel_tharindu/4c31fb537ca0f31ab15bbd6a8445d3b6/external/io_bazel_rules_docker/container/pull.bzl", line 240, column 13, in _impl\n\t\tfail("Pull command failed: %s (%s)" % (result.stderr, " ".join([str(a) for a in args])))\nError in fail: Pull command failed: 2022/06/12 04:55:22 Running the Image Puller to pull images from a Docker Registry...\n2022/06/12 04:55:22 Image pull was unsuccessful: reading image "gcr.io/pixie-oss/pixie-dev-public/docker-deps/library/openjdk@sha256:25b910311bfe15547ecab6895d5eb3f4ec718d6d53cced7eec78e4b889962e1f": error getting credentials - err: exec: "docker-credential-osxkeychain": executable file not found in $PATH, out: \n (/home/tharindu/.cache/bazel/_bazel_tharindu/4c31fb537ca0f31ab15bbd6a8445d3b6/external/go_puller_linux_amd64/file/downloaded -directory /home/tharindu/.cache/bazel/_bazel_tharindu/4c31fb537ca0f31ab15bbd6a8445d3b6/external/openjdk-base-musl/image -os linux -os-version -os-features -architecture amd64 -variant -features -name gcr.io/pixie-oss/pixie-dev-public/docker-deps/library/openjdk@sha256:25b910311bfe15547ecab6895d5eb3f4ec718d6d53cced7eec78e4b889962e1f)\nERROR: /pl/src/px.dev/pixie/src/stirling/source_connectors/perf_profiler/java/agent/BUILD.bazel:73:16: no such package '@openjdk-base-musl//image': Pull command failed: 2022/06/12 04:55:22 Running the Image Puller to pull images from a Docker Registry...\n2022/06/12 04:55:22 Image pull was unsuccessful: reading image \"gcr.io/pixie-oss/pixie-dev-public/docker-deps/library/openjdk@sha256:25b910311bfe15547ecab6895d5eb3f4ec718d6d53cced7eec78e4b889962e1f\": error getting credentials - err: exec: \"docker-credential-osxkeychain\": executable file not found in $PATH, out: \n (/home/tharindu/.cache/bazel/_bazel_tharindu/4c31fb537ca0f31ab15bbd6a8445d3b6/external/go_puller_linux_amd64/file/downloaded -directory /home/tharindu/.cache/bazel/_bazel_tharindu/4c31fb537ca0f31ab15bbd6a8445d3b6/external/openjdk-base-musl/image -os linux -os-version -os-features -architecture amd64 -variant -features -name gcr.io/pixie-oss/pixie-dev-public/docker-deps/library/openjdk@sha256:25b910311bfe15547ecab6895d5eb3f4ec718d6d53cced7eec78e4b889962e1f) and referenced by '//src/stirling/source_connectors/perf_profiler/java/agent:image-musl'\nLoading: 50 packages loaded\nLoading: 51 packages loaded\nLoading: 51 packages loaded\nLoading: 51 packages loaded\nLoading: 51 packages loaded\nLoading: 51 packages loaded\nINFO: Repository com_github_antlr_grammars_v4 instantiated at:\n /pl/src/px.dev/pixie/WORKSPACE:10:8: in \n /pl/src/px.dev/pixie/bazel/repositories.bzl:222:13: in _pl_deps\n /pl/src/px.dev/pixie/bazel/repositories.bzl:158:16: in _cc_deps\n /pl/src/px.dev/pixie/bazel/repositories.bzl:98:28: in _bazel_repo\n /pl/src/px.dev/pixie/bazel/repositories.bzl:39:17: in _http_archive_repo_impl\nRepository rule http_archive defined at:\n /home/tharindu/.cache/bazel/_bazel_tharindu/4c31fb537ca0f31ab15bbd6a8445d3b6/external/bazel_tools/tools/build_defs/repo/http.bzl:353:31: in \nINFO: Repository 'com_github_antlr_grammars_v4' used the following cache hits instead of downloading the corresponding file.\n * Hash '9858e4a9944cac85830e6cf8edd9d567227af96d8b75f0b31accc525ec842c30' for https://github.com/antlr/grammars-v4/archive/e53d7a1228505bfc80d8637808ef60e7eea92cc2.tar.gz\nIf the definition of 'com_github_antlr_grammars_v4' was updated, verify that the hashes were also updated.\nERROR: An error occurred during the fetch of repository 'com_github_antlr_grammars_v4':\n Traceback (most recent call last):\n\tFile "/home/tharindu/.cache/bazel/_bazel_tharindu/4c31fb537ca0f31ab15bbd6a8445d3b6/external/bazel_tools/tools/build_defs/repo/http.bzl", line 111, column 10, in _http_archive_impl\n\t\tpatch(ctx, auth = auth)\n\tFile "/home/tharindu/.cache/bazel/_bazel_tharindu/4c31fb537ca0f31ab15bbd6a8445d3b6/external/bazel_tools/tools/build_defs/repo/utils.bzl", line 167, column 22, in patch\n\t\tctx.patch(patchfile, strip)\nError in patch: Error applying patch /pl/src/px.dev/pixie/bazel/external/antlr_grammars.patch: Cannot rename file (near line 6), new file name (sql/mysql/Positive-Technologies/MySQLLexer.g4) already exists.\nERROR: /pl/src/px.dev/pixie/WORKSPACE:10:8: fetching http_archive rule //external:com_github_antlr_grammars_v4: Traceback (most recent call last):\n\tFile "/home/tharindu/.cache/bazel/_bazel_tharindu/4c31fb537ca0f31ab15bbd6a8445d3b6/external/bazel_tools/tools/build_defs/repo/http.bzl", line 111, column 10, in _http_archive_impl\n\t\tpatch(ctx, auth = auth)\n\tFile "/home/tharindu/.cache/bazel/_bazel_tharindu/4c31fb537ca0f31ab15bbd6a8445d3b6/external/bazel_tools/tools/build_defs/repo/utils.bzl", line 167, column 22, in patch\n\t\tctx.patch(patchfile, strip)\nError in patch: Error applying patch /pl/src/px.dev/pixie/bazel/external/antlr_grammars.patch: Cannot rename file (near line 6), new file name (sql/mysql/Positive-Technologies/MySQLLexer.g4) already exists.\nERROR: /pl/src/px.dev/pixie/src/carnot/funcs/builtins/sql_parsing/BUILD.bazel:21:14: no such package '@com_github_antlr_grammars_v4//': Error applying patch /pl/src/px.dev/pixie/bazel/external/antlr_grammars.patch: Cannot rename file (near line 6), new file name (sql/mysql/Positive-Technologies/MySQLLexer.g4) already exists. and referenced by '//src/carnot/funcs/builtins/sql_parsing:cc_library'\nINFO: Repository openjdk-base-glibc instantiated at:\n /pl/src/px.dev/pixie/WORKSPACE:92:20: in \n /pl/src/px.dev/pixie/bazel/pl_workspace.bzl:68:28: in pl_container_images\n /pl/src/px.dev/pixie/bazel/pl_workspace.bzl:38:16: in _container_images_setup\n /pl/src/px.dev/pixie/bazel/container_images.bzl:62:18: in base_images\n /pl/src/px.dev/pixie/bazel/container_images.bzl:30:19: in _gcr_io_image\nRepository rule container_pull defined at:\n /home/tharindu/.cache/bazel/_bazel_tharindu/4c31fb537ca0f31ab15bbd6a8445d3b6/external/io_bazel_rules_docker/container/pull.bzl:294:33: in \nERROR: An error occurred during the fetch of repository 'openjdk-base-glibc':\n Traceback (most recent call last):\n\tFile "/home/tharindu/.cache/bazel/_bazel_tharindu/4c31fb537ca0f31ab15bbd6a8445d3b6/external/io_bazel_rules_docker/container/pull.bzl", line 240, column 13, in _impl\n\t\tfail("Pull command failed: %s (%s)" % (result.stderr, " ".join([str(a) for a in args])))\nError in fail: Pull command failed: 2022/06/12 04:57:08 Running the Image Puller to pull images from a Docker Registry...\n2022/06/12 04:57:08 Image pull was unsuccessful: reading image "gcr.io/pixie-oss/pixie-dev-public/docker-deps/library/openjdk@sha256:d7048f5a32ca7598f583c492c960496848cc9017fdb55942370f02603c83561d": error getting credentials - err: exec: "docker-credential-osxkeychain": executable file not found in $PATH, out: \n (/home/tharindu/.cache/bazel/_bazel_tharindu/4c31fb537ca0f31ab15bbd6a8445d3b6/external/go_puller_linux_amd64/file/downloaded -directory /home/tharindu/.cache/bazel/_bazel_tharindu/4c31fb537ca0f31ab15bbd6a8445d3b6/external/openjdk-base-glibc/image -os linux -os-version -os-features -architecture amd64 -variant -features -name gcr.io/pixie-oss/pixie-dev-public/docker-deps/library/openjdk@sha256:d7048f5a32ca7598f583c492c960496848cc9017fdb55942370f02603c83561d)\nERROR: /pl/src/px.dev/pixie/WORKSPACE:92:20: fetching container_pull rule //external:openjdk-base-glibc: Traceback (most recent call last):\n\tFile \"/home/tharindu/.cache/bazel/_bazel_tharindu/4c31fb537ca0f31ab15bbd6a8445d3b6/external/io_bazel_rules_docker/container/pull.bzl\", line 240, column 13, in _impl\n\t\tfail(\"Pull command failed: %s (%s)\" % (result.stderr, \" \".join([str(a) for a in args])))\nError in fail: Pull command failed: 2022/06/12 04:57:08 Running the Image Puller to pull images from a Docker Registry...\n2022/06/12 04:57:08 Image pull was unsuccessful: reading image \"gcr.io/pixie-oss/pixie-dev-public/docker-deps/library/openjdk@sha256:d7048f5a32ca7598f583c492c960496848cc9017fdb55942370f02603c83561d\": error getting credentials - err: exec: \"docker-credential-osxkeychain\": executable file not found in $PATH, out: \n (/home/tharindu/.cache/bazel/_bazel_tharindu/4c31fb537ca0f31ab15bbd6a8445d3b6/external/go_puller_linux_amd64/file/downloaded -directory /home/tharindu/.cache/bazel/_bazel_tharindu/4c31fb537ca0f31ab15bbd6a8445d3b6/external/openjdk-base-glibc/image -os linux -os-version -os-features -architecture amd64 -variant -features -name gcr.io/pixie-oss/pixie-dev-public/docker-deps/library/openjdk@sha256:d7048f5a32ca7598f583c492c960496848cc9017fdb55942370f02603c83561d)\nERROR: /pl/src/px.dev/pixie/src/stirling/source_connectors/perf_profiler/java/agent/BUILD.bazel:66:16: no such package '@openjdk-base-glibc//image': Pull command failed: 2022/06/12 04:57:08 Running the Image Puller to pull images from a Docker Registry...\n2022/06/12 04:57:08 Image pull was unsuccessful: reading image "gcr.io/pixie-oss/pixie-dev-public/docker-deps/library/openjdk@sha256:d7048f5a32ca7598f583c492c960496848cc9017fdb55942370f02603c83561d": error getting credentials - err: exec: "docker-credential-osxkeychain": executable file not found in $PATH, out: \n (/home/tharindu/.cache/bazel/_bazel_tharindu/4c31fb537ca0f31ab15bbd6a8445d3b6/external/go_puller_linux_amd64/file/downloaded -directory /home/tharindu/.cache/bazel/_bazel_tharindu/4c31fb537ca0f31ab15bbd6a8445d3b6/external/openjdk-base-glibc/image -os linux -os-version -os-features -architecture amd64 -variant -features -name gcr.io/pixie-oss/pixie-dev-public/docker-deps/library/openjdk@sha256:d7048f5a32ca7598f583c492c960496848cc9017fdb55942370f02603c83561d) and referenced by '//src/stirling/source_connectors/perf_profiler/java/agent:image-glibc'\nERROR: /pl/src/px.dev/pixie/BUILD.bazel:137:16: no such package '@base_image//image': Pull command failed: 2022/06/12 04:55:09 Running the Image Puller to pull images from a Docker Registry...\n2022/06/12 04:55:09 Image pull was unsuccessful: reading image \"gcr.io/distroless/base@sha256:34e96e21516698913035a62ef1ce484d91184de8a44209a33c7e134547e20dee\": error getting credentials - err: exec: \"docker-credential-osxkeychain\": executable file not found in $PATH, out: \n (/home/tharindu/.cache/bazel/_bazel_tharindu/4c31fb537ca0f31ab15bbd6a8445d3b6/external/go_puller_linux_amd64/file/downloaded -directory /home/tharindu/.cache/bazel/_bazel_tharindu/4c31fb537ca0f31ab15bbd6a8445d3b6/external/base_image/image -os linux -os-version -os-features -architecture amd64 -variant -features -name gcr.io/distroless/base@sha256:34e96e21516698913035a62ef1ce484d91184de8a44209a33c7e134547e20dee) and referenced by '//:pl_cc_base_image'\nERROR: /pl/src/px.dev/pixie/src/stirling/source_connectors/perf_profiler/java/agent/BUILD.bazel:66:16: no such package '@openjdk-base-glibc//image': Pull command failed: 2022/06/12 04:57:08 Running the Image Puller to pull images from a Docker Registry...\n2022/06/12 04:57:08 Image pull was unsuccessful: reading image "gcr.io/pixie-oss/pixie-dev-public/docker-deps/library/openjdk@sha256:d7048f5a32ca7598f583c492c960496848cc9017fdb55942370f02603c83561d": error getting credentials - err: exec: "docker-credential-osxkeychain": executable file not found in $PATH, out: \n (/home/tharindu/.cache/bazel/_bazel_tharindu/4c31fb537ca0f31ab15bbd6a8445d3b6/external/go_puller_linux_amd64/file/downloaded -directory /home/tharindu/.cache/bazel/_bazel_tharindu/4c31fb537ca0f31ab15bbd6a8445d3b6/external/openjdk-base-glibc/image -os linux -os-version -os-features -architecture amd64 -variant -features -name gcr.io/pixie-oss/pixie-dev-public/docker-deps/library/openjdk@sha256:d7048f5a32ca7598f583c492c960496848cc9017fdb55942370f02603c83561d) and referenced by '//src/stirling/source_connectors/perf_profiler/java/agent:image-glibc'\nERROR: /pl/src/px.dev/pixie/src/stirling/source_connectors/perf_profiler/java/agent/BUILD.bazel:73:16: no such package '@openjdk-base-musl//image': Pull command failed: 2022/06/12 04:55:22 Running the Image Puller to pull images from a Docker Registry...\n2022/06/12 04:55:22 Image pull was unsuccessful: reading image \"gcr.io/pixie-oss/pixie-dev-public/docker-deps/library/openjdk@sha256:25b910311bfe15547ecab6895d5eb3f4ec718d6d53cced7eec78e4b889962e1f\": error getting credentials - err: exec: \"docker-credential-osxkeychain\": executable file not found in $PATH, out: \n (/home/tharindu/.cache/bazel/_bazel_tharindu/4c31fb537ca0f31ab15bbd6a8445d3b6/external/go_puller_linux_amd64/file/downloaded -directory /home/tharindu/.cache/bazel/_bazel_tharindu/4c31fb537ca0f31ab15bbd6a8445d3b6/external/openjdk-base-musl/image -os linux -os-version -os-features -architecture amd64 -variant -features -name gcr.io/pixie-oss/pixie-dev-public/docker-deps/library/openjdk@sha256:25b910311bfe15547ecab6895d5eb3f4ec718d6d53cced7eec78e4b889962e1f) and referenced by '//src/stirling/source_connectors/perf_profiler/java/agent:image-musl'\nERROR: /pl/src/px.dev/pixie/src/carnot/funcs/builtins/sql_parsing/BUILD.bazel:21:14: no such package '@com_github_antlr_grammars_v4//': Error applying patch /pl/src/px.dev/pixie/bazel/external/antlr_grammars.patch: Cannot rename file (near line 6), new file name (sql/mysql/Positive-Technologies/MySQLLexer.g4) already exists. and referenced by '//src/carnot/funcs/builtins/sql_parsing:cc_library'\nERROR: Evaluation of subquery "deps(//src/vizier/services/agent/pem:pem_image.tar)" failed (did you want to use --keep_going?): errors were encountered while computing transitive closure\nLoading: 101 packages loaded\nLoading: 101 packages loaded\n"
  • cause: exit status 7

I am just trying to build and deploy vizier with a macbook. Using minikube to build and deploy was also not successful since minikube uses a separate docker runtime and mounting host file system to minikube vm was problematic in mac os.

@zasgar
Copy link
Member

zasgar commented Jun 15, 2022

Related issue: lima-vm/lima#450

@zasgar zasgar added priority/important-longterm Important over the long term, but may not be staffed and/or may need multiple releases to complete. triage/needs-information Indicates an issue needs more information in order to work on it. area/datacollector Issues related to Stirling (datacollector) kind/compatibility Compatibility related to either K8s, architecture, library versions labels Jun 15, 2022
@htroisi
Copy link
Contributor

htroisi commented Jun 16, 2022

You should be able to test this after this week's Vizier release.

@yzhao1012 - I just tested this on my Intel Macbook and it now works for me (I was previously able to reproduce the issue). Given that you've confirmed that it also works on M1 Macbook and Linux, should we add Rancher Desktop as a supported environment to Pixie's Requirements page?

@yzhao1012
Copy link
Contributor

Related issue: lima-vm/lima#450

with rancher desktop and dockerd runtime

Dockerd runtime wont work.
Actually, you have to use the VM driver, like what Hannah and I tested with rancher-desktop.
As docker (or any container-based kubernetes desktop environment) is not compatible with Pixie (PEM specifically).

The reason is that PEM needs to access the host kernel's system filesystem to compile BPF code.
Container-based kubernetes desktop environment hides the host filesystem from PEM and breaks it.

@rainan16
Copy link

rainan16 commented Jul 4, 2023

@yzhao1012 - I just tested this on my Intel Macbook and it now works for me (I was previously able to reproduce the issue). Given that you've confirmed that it also works on M1 Macbook and Linux, should we add Rancher Desktop as a supported environment to Pixie's Requirements page?

@htroisi Is this confirmed to work on Macbook M1/M2? I still get the same error on my Macbook Pro M2 using Pixie 0.8.2+Distribution.401c92c.20230530203620.1.jenkins and the instructions from Install guide for Community Cloud, which installed gcr.io/pixie-oss/pixie-prod/vizier/pem_image:0.14.0 in my RD cluster.

Logs: pixie_logs_20230704095739.zip

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
area/datacollector Issues related to Stirling (datacollector) kind/compatibility Compatibility related to either K8s, architecture, library versions priority/important-longterm Important over the long term, but may not be staffed and/or may need multiple releases to complete. triage/needs-information Indicates an issue needs more information in order to work on it.
Projects
None yet
Development

No branches or pull requests

8 participants