Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Buildkite Agent v3.19.0 fails to download files from GCS with backslashes in the name? #1159

Open
philwo opened this issue Feb 4, 2020 · 1 comment

Comments

@philwo
Copy link
Contributor

@philwo philwo commented Feb 4, 2020

Hi,

I'm debugging a mysterious pipeline failure that seems to occur since we upgraded our Buildkite Agents from v3.14.0 to v3.19.0.

We have some build artifacts that are in a sub-folder on Windows, e.g. bazel-bin\bin\bazel.exe. When Buildkite Agent uploads them, they get stored like this on GCS: https://storage.googleapis.com/bazel-trusted-buildkite-artifacts/148f94cc-c9bd-4600-b73c-7b41d5bdc8e6/bazel-bin%5Csrc%5Cbazel.exe (the backslashes remain and aren't converted into forward slashes).

Screenshot 2020-02-04 at 21 23 40

This is also visible in the GCS console:

Screenshot 2020-02-04 at 21 33 56

Since v3.19.0, Buildkite Agent fails to download these artifacts, at least when it's running on Linux (haven't tested what happens when the downloading agent runs on Windows).

With v3.14.0:
Screenshot 2020-02-04 at 21 19 13

With v3.19.0:
Screenshot 2020-02-04 at 21 18 56

The issue seems to be that the agent now converts the backslashes to forward slashes when it builds the URL for the download. This is the one it tries to download from:

https://www.googleapis.com/storage/v1/b/bazel-trusted-buildkite-artifacts/o/edf99f72-5065-4b49-815e-afb715d8fad3%2Fbazel-bin%2Fsrc%2Fbazel.exe?alt=media

This would be the correct one (note that %2F vs. %5C):

https://www.googleapis.com/storage/v1/b/bazel-trusted-buildkite-artifacts/o/edf99f72-5065-4b49-815e-afb715d8fad3%2Fbazel-bin%5Csrc%5Cbazel.exe?alt=media

Our pipeline is now failing. I can see a workaround here, of course - we can upload the binaries without the path in their name, but this seems like a bug? 🤔

@philwo philwo changed the title Buildkite Agent v3.19.0 uploads files to GCS with wrong filename on Windows? Buildkite Agent v3.19.0 fails to download files from GCS with backslashes in the name? Feb 4, 2020
@yob

This comment has been minimized.

Copy link
Contributor

@yob yob commented Feb 5, 2020

PR #1113 shipped in agent 3.17.0 and sounds like it might be related.

Maybe @lox can provide some more context about that change and why it might result in what you're seeing.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Linked pull requests

Successfully merging a pull request may close this issue.

None yet
2 participants
You can’t perform that action at this time.